Let me try to summarize my understanding of your questions before I respond to it.
You seem to be asking about the creation of theories, and how they are motivated by fundamental assumptions. Once a theory has come to be, it is (usually) expressible in math, and that means ultimately in sets, the ground of math. Now, since the theory is motivated by fundamental assumptions, shouldn't we be able to identify those assumptions in the theory, and be able to express them with sets, just like we can express theorems with sets?
But before I answer those question, let me try to tackle a second question you raised. There is also the question of why we are using these particular assumptions, and not other ones? What justifies our assumptions?
Here I would answer that if you want to look back and evaluate your assumptions, it matters why you want to make a theory in the first place. Some people make theories because they think they are beautiful, in which case the assumptions are justified by the perceived beauty of the formal system. Other people want to be able to make machines, in which case the functionality of those machines would justify the assumptions.
What I'm getting at here, is that the assumptions that created a formal system will always be in a realm outside of the formal system itself. Now I haven't myself run through the technical proof of Gödel's incompleteness theorems, but nevertheless I smell Gödel here: Suppose you have a formal system that can 1) determine whether things are true of false 2) express the assumptions that motivated the creation of the theory. Then you would be able to use that formal system to evaluate the truth of those assumptions. I'm claiming that this is essentially the same as something the 2nd incompleteness theory explicitly prohibits: "Second incompleteness theorem
For any consistent system F within which a certain amount of elementary arithmetic can be carried out, the consistency of F cannot be proved in F itself." Ref. 1.
Further, if you had a formal system that was able to express the assumptions that caused the creation of itself, that means you should also be able to mathematically express the concept of beauty, or functionality, or any other reason why someone looked at the world, thought about it, and decided to turn what they were thinking about into a formal system.