Long comment
Two good "semi-technical" introduction to the issue:
T.Franzen, Gödel's Theorem. An Incomplete Guide to Its Use and Abuse (2005) and
F.Berto, There's Something About Gödel (2009).
A complete treatment:
Regarding the never-ending hierarchy of theories, see:
- T.Franzen, Inexhaustibility: A Non-Exhaustive Treatment (2004).: Introduction, page 1:
Gödel presented two results in his 1931 paper, usually referred to as the first and the second incompleteness theorem. The proof of the first incompleteness theorem shows that for every consistent formal axiomatic theory in a wide class of such theories, there is at least one statement which can be formulated in the language of the theory but can neither be proved nor disproved in the theory. [...] The second theorem states that for a wide class of such theories T, if T is consistent, the consistency of T cannot be proved using only the axioms of the theory T itself.
The second incompleteness theorem also has a positive aspect, which was emphasized by Gödel (Collected Works, vol. III, p.309, italics in the original): "It is this theorem [the second incompleteness theorem] which makes the incompletability of mathematics particularly evident. For, it makes it impossible that someone should set up a certain well-defined system of axioms and rules and consistently make the following assertion about it: All of these axioms and rules I perceive (with mathematical certitude) to be correct, and moreover I believe that they contain all of mathematics. If somebody makes such a statement he contradicts himself. For if he perceives the axioms under consideration to be correct, he also perceives (with the same certainty) that they are consistent. Hence he has a mathematical insight not derivable from his axioms."