Formal reasoning is powerful for two reasons. The first is the simplest to see: it is easily conveyed in a highly objective manner. When trying to discuss things which must be true for all, the more objective the better.
The second advantage of formal reasoning is that it can be written using formal languages. This permits the fascinating ability to discuss the validity of a claim through symbol manipulation alone. People can agree that a particular set of symbol manipulations correspond to true semantic meaning, and then determine if the semantic meaning is valid merely by looking at the symbols.
My favorite example of where this can be useful is the handling of infinity in mathematics vs. the handling of infinity in Pascal's Wager. The rules of set theory permit a very specific set of behaviors for infinite values, such as the cardinality of the set of all natural numbers. It states that the cardinality of the set of natural numbers is fundamentally smaller than the cardinality of the set of real numbers. Contrast that with Pascal's wager, where Pascal effectively argues that since one infinity appeared in the equation, it clearly had to dominate. It is now trivial to develop mathematical arguments for or against Pascal's wager, using various infinities. It is trivial to discuss maximizing value in the wager by taking the derivative of value and applying l'Hopital's rule (itself a wedging of two infinities against eachother). The only reason these are trivial is because their definitions and proofs are so rigorous that we accept them as valid even if the situation is absurd.
The limit to what formal languages can do was explored by Tarski. You are correct in being bothered that people don't want to discuss why formal reasoning is so important. Tarski demonstrated that the semantics of a formal language must be defined in a metalanguage, suggesting eventually you arrive at a non-formal language to describe how to interpret all the formal languages. However, despite this shortcoming, there is still value in them. They permit arguments like "either you must accept this formal reasoning, or you must come up with your own explanation of why 2+3=5 without using the Peano axioms at the basis of modern math."