What you are asking is how reasoning is justified. This is part of a more general problem about epistemological justification, but applied specifically to reasoning and logic. Charles Dodgson (aka Lewis Carroll) published a paper called "What the Tortoise said to Achilles" to illustrate this problem. It shows that you cannot justify a simple logical rule like modus ponens using logic without circularity.
Broadly there are two classes of response, internalist and externalist. Internalists hold that justification must be internal to the cognitive agent. One form of this is a kind of psychologism in which what justifies reasoning is the fact that the agent perceives the reasoning as being intuitively obvious. Descartes appears to follow this route, in that he frequently speaks of "clear and distinct ideas". The problem is that all kinds of things seem intuitively obvious and subsequently turn out to be false. It seemed obvious to people for a long time that the postulates of euclidean geometry were indubitably true; it seems obvious at first that if the universe is unbounded it must be infinite; it seems obvious that if a system is determinate it must be predictable.
Externalists, by contrast, hold that reasoning is justified by reference to facts outside the cognitive agent. One form of this is reliabilism, which is roughly the idea that reasoning is justified by the fact that it works well in practice, or else we would have discarded it. This rather assumes that we are indeed good at reasoning, which is moot, given that we know humans are subject to all manner of cognitive biases and in particular are notoriously bad at reasoning with uncertain information. Another version is to say reasoning is justified as being a successful evolutionary adaptation, i.e. being able to reason more or less correctly is justified by its survival value. Again, this is problematic because our reasoning may be just a side effect of some primitive ability possessed by our pre-human ancestors that helped them to overcome the difficulties of surviving in Africa, but doesn't of itself provide us with any confidence that it is good for solving problems in higher mathematics.
There is some more general information about internalist vs. externalist conceptions of justification in the Stanford encyclopedia article.