2

Roger Penrose famously from Gödel's Incompleteness Theorem, that human mind is not computable, because mathematical intuition is not computable (a mathematician can prove more than any formal system, and therefore any computer).

But when I read through the proof or derivation of Gödel's First Incompleteness Theorem (say, in Penrose's The Emperor's New Mind or at plato.stanford.edu), it just happens to have been written by a human. In priciniple, a Large Language Model could write the same book or article (maybe not now, but almost surely in 10 years).

Why is a mathematician's mind not computable and the LLM is computable given that 1) they both produce proofs as sequences of natural language mixed with mathematical formalism, 2) any proof produced by a human can be produced, in principle, by an LLM (being the same sequence of natural language mixed with formal mathematics)?

Improve this question
6
  • The issue is exactly that: is the mind only a "collection" of formal systems? in which case GIT applies, or instead the capability of human mind to "transcend" the intrinsic limitations of formal systems via a "global intuition" shows that it is not?
    – Mauro ALLEGRANZA
    Commented May 31, 2023 at 11:27
  • Large language models just use statistical inference. That is why. It's like building machines to do simple repetative jobs, & concluding no human workers will be needed in the near future.
    – CriglCragl
    Commented May 31, 2023 at 11:35
  • The ability to figure out the GIT is quite different from the ability to learn to write an explanation of it.
    – Professor Sushing
    Commented May 31, 2023 at 12:10
  • 3
    > Large language models just use statistical inference. Yes. But to say that human brain does more than that is a non trivial claim.
    – Tomasz Garbus
    Commented May 31, 2023 at 12:19
  • 1
    How about this for a test. Train an LLM on all the mathematical papers written before the GIT, then ask the LLM if it can prove incompleteness.
    – Professor Sushing
    Commented May 31, 2023 at 12:26

1 Answer 1

Reset to default
2

Penrose's argument doesn't apply to LLMs for the simple reason that it assumes the AI is never wrong. The AI is based on a formal system, it only believes things that are provable in that system, and you the reader are supposed to believe ("see", in a fashion only human beings can) that that formal system is consistent. You may have noticed that LLMs are frequently wrong, and I don't see that changing in 10 years. I think the design of the things is fundamentally incompatible with being always right about everything.

(If you're wondering why Penrose holds the AI to a higher standard than human beings, who obviously are sometimes wrong, then you've spotted the reason why most people don't accept Penrose's argument.)

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .