Do we eventually say that we are satisfied that the premise of a conclusion can stand on its own, or do we really just not continue this exercise infinitely because we would rather do something else with our time?
I think there are two ways in which this can be understood. Your question isn't one about logic, because logic doesn't say anything about the premises or where they come from. Instead, it's either a question of epistemology or a practical issue of how to make philosophical arguments.
The epistemological issue
The idea is this: if we depend on some premises in order to reach conclusions, how are we justified in taking these premises? So, it's an issue of the structure of knowledge. This question goes all the way back to Ancient Greece to Pyrrhonian scepticism. The basic idea is called Agrippa's Trilemma or Münchhausen Trilemma. Wikipedia will do here.
The article already mentions foundationalists and coherentists. Foundationalists (about justification) believe that we can in fact go back after every premise all the way until we get some sort of basic truth. Some "modest" foundationalists will make a weaker claim, but we can take out of it that this is how we can structure justification.
Then there are coherentists. Those generally believe that justification is holistic, so that we can't go back to basic truths, but that instead a good justification belongs to a non-contradictory net. In the trilemma, this is kind of the circular solution.
Those are the basic two directions. Apart from that, other theories try to combine the two, circumvent the problem, or try to get rid of the need for justification. There's also infinitism, which thinks that there are justificatory chains (just like foundationalism) that just, well, don't come to an end.
The practical issue
I'll keep this short, mainly because there aren't any set positions for this. (Although there's another epistemological issue related to this, but I'll bracket that.)
I believe this very much depends on the claim that we're making and what we want to achieve with our argument. If we want to think about issues for ourselves, then simply using what we're already holding as plausible can be just fine. If we want to join the discussion about an issue then we should be clear about our starting points and not be too far off from the discourse. If we want to specifically convince someone then we should go back until we think that there's a consensus about the starting point (or until we are okay with with dissent).