The ZFC axioms are the basis of modern mathematics. But Gödel's 2nd Theorem says that it is impossible to prove that these axioms are consistent. Hence, it is possible (if ZFC is inconsistent) that some of the theorems proven by mathematicians using the ZFC axioms are false.
So how is it correct for mathematicians to claim that any theorems proven in ZFC have been proven?
Let's start lower down. By the incompleteness theorems, PA (first-order arithmetic) can't prove its own consistency. Do we have to worry that PA is inconsistent? Fortunately not: we have a stronger system (ZFC) which proves the consistency of PA. But that's not much help -- if you doubt the consistency of PA, this just means you should doubt the consistency of ZFC as well. What we really want to say is: we have very good reason to believe that PA is consistent. Namely, we've been working with PA for many years and uncovered no inconsistency. And the axioms seem to be saying pretty common-sense things about a class of objects (the natural numbers) whose existence most of us take for granted.
Now move up to someone who doubts the consistency of ZFC. I could point to some higher system (say, ZFC + "there exists a strongly inaccessible cardinal") which proves the consistency of ZFC. But again, that wouldn't help, since if you doubt the consistency of ZFC you doubt the consistency of the second system. It's much better to say: we've been working with ZFC for a long time and uncovered no inconsistency. And ZFC seems to be saying some pretty common-sense thing about a class of objects (sets) whose existence most mathematicians now take for granted.
They are proven with the implicit assumption of the ZFC axioms.
If the foundations of a building collapse the whole building falls down.
Luckily, foundations in mathematics don't work this way: if ZFC was proven inconsistent tommorow it's unlikely that the girl manning the till at my local store will be worrying whether the change adds up correctly; or that a student worrying over how to integrate sin x may now be thinking does this have an answer different to what it did yesterday.
There isn't one way to justify results; there's a whole host of them; and they are inter-justificatory; this is a position called coherentism.
This has already happened once in the early days of Set Theory: Russell's Paradox - which was eventually resolved by working around it.
Very few theorems of mathematics can be proved "only" from ZFC axioms.
Theorems of arithmetic are proved from Peano axioms and the fact that these axioms in turn are provable from ZFC adds nothing to our "basic" understanding of natural numbers.
What set theory provides in terms of "foundations" is twofold :
a simple and unified language common to (quite) all mathematical theories
a powerful way to build up models for (quite) all mathematical theories.
This second "use" of set theory is the part that cab be "philosophically questionable" :
if we are searching for some sort of "certainity" regarding the existence of the natural numbers system, the fact that we can provide a model for the natural numbers axioms within ZFC, can only "move" the problem one step back.
In any case, the major interest in set theory relies in its own mathematical content, like every mathematical theory.
Gödel's result applies to theories which "include" a model of the arithmetic of integers, so incompleteness and questions of consistency need not apply to those theories which do not include integer arithmetic.
For example, Euclidean geometry can be proven to be both complete and consistent. Other geometries are also of this type - e.g., Hilbert's axioms and Tarski's axioms.
This (apparently) follows from the fact that the continuum, R, and therefore the plane R2 of geometry, is formalizable as a recursively enumerable theory that is incapable of defining integers as real numbers. I'm not entirely clear on the details of these results, but this is what I have read.
Therefore, it is not correct to say that "all" of mathematics is of questionable consistency.
@MoziburUllah is correct, but it is not so ambiguous as all that. At least not lately.
Since starting to dwell on its roots, math really has accepted a slightly less ambiguous formulation of than coherence we have had historically. The current model of math (along with the main forms in which formal logic is taught) is dual. It really only involves two proof positions -- deduction of implication via grammar of combination, and construction of 'worlds' or models via a grammar of description.
(These two positions, along with the distinction between them, go back to Euclid. So people who want to limit or minimize the assumption of coherence can easily just ignore the intermediate positions, and claim they were just spurious nonsense. This is what modern formalists do.)
Consistency may not be available via formal construction, but people's real intuition of validity or consistency is the one based upon models, not the one based on formal proofs. Then from a point of view focussed mainly on mathematics as the psychology of shared intuition, we really do not believe, deep down, that formal language is what really proves things. It is only a way of checking ourselves when we act upon our other intuitions, which we presume reliable, as long as we do not pursue them too far.
From that point of view, we can look at the stripped-down 'universes' of Von Neumann's V, or Goedel's L, (or even Conway's "Surreal Numbers") and agree that our system has a model. Then even if our formal manipulations ultimately do not really apply directly to the objects we are studying, our chosen base model does contain something isomorphic to them.
So, in any case where we are explicit enough about what we are saying to define isomorphism clearly, we can validate that what we are doing would not have a contradiction in it, by mapping its interpretation back onto our chosen base model.
This is the formalist escape from the ultimate weakness of formalism: to claim mathematical claims "only exist up to isomorphism" and to ignore the reliance of the existence of the base models upon intuition.
