8

A slightly flippant question, but curious to see what my platonist rivals might have to say!

One of the proported reasons that Open-AI was having business politics trouble was the suggestion that their newest Transformer-architecture driven Foundation model was powerful enough to carry out complex mathematical reasoning.

To a Mathematical Formalist, it should come as no surprise that a semantically sophisticated language model would have what it needs to properly do mathematical proof. There is no dedicated mathematical realm - number and set talk is just a kind of communication protocol.

So… checkmate?

14
  • 5
    "semantically sophisticated language" seems to imply that language has meaning and there is "something" about which the language speak of. What is this "something"? what sort of reality? Dec 6, 2023 at 11:17
  • 2
    I don't think LLMs are capable of anything that wasn't already in principle possible by e.g. exhaustive proof search / enumeration. The interesting thing about them is whether they make automated mathematical reasoning practical in a way it wasn't before, but I don't see a case for that difference being philosophically significant. Dec 6, 2023 at 11:31
  • 5
    i find your question very vague. what is "complex mathematical reasoning"? computers have always been super powerful combinatorics machines, and that is no big deal. those ai things like openai can combine and recombine already existing knowledge ad infinitum, but they cannot formulate brand new concepts, they cannot unify distinct areas of maths, and they cannot give revolutionary new meanings to old concepts, no reasoning. a big part of it is marketing also, the more "super evil superpower" their software seems, the more $ they will receive to keep writing a few lines of python code per day. Dec 6, 2023 at 13:59
  • 7
    LLMs can't even carry out arithmetic reliably.
    – hobbs
    Dec 7, 2023 at 2:04
  • 2
    LLMs don't carry out reasoning at all. They are very fancy next word predictors. Dec 7, 2023 at 15:54

4 Answers 4

8

As a constructivist brother who places as much credence in Platonic Forms as he does in the Irish tuatha da dannan or the Norwegian troll, let me dispute the premise that LLMs do math or have much in the way of semantic awareness.

Human mathematicians are enriched with two cognitive processes that LLMs have no analogue for since the transformer model deals only with syntactic awareness when it statistically evaluates sequences of tokens in a corpus. First, human beings work with chunks of information that go beyond arbitrary short-sequenced token collections. In fact, human grammars are processed by tokens and accompanying morphology at several levels: the morpheme, the lexeme, the phrase, and the sentence. The result of this is that an LLM is blind to all of the concomitant forms of semantics for each type of compositional element. Second, human beings ground those grammars in three major ways: operational, denotational, and axiomatic semantics. LLMs are capable of none of these types of semantics, and obviously without the systems for grounding the elements, no mathematics is actually "being done".

That is not to say that LLMs don't appear to be reasoning. LLMs and various prompt engineering strategies such as chain-of-thought can even craft language that can be run through an automated proof checker. However, the mechanisms by which an LLM understands text is limited to what the NLP community calls text processing, which might best be understood as character and string manipulation grammars and don't go beyond. More sophisticated models of reasoning such as phrase structure grammars, case grammars, and semantic networks have an expanded capacity for semantic action that LLMs do not possess. And those are old techniques which are relatively unsophisticated unlike type-theoretic semantics which finally begin to approach the vision Richard Montague set out in his seminal work marrying natural language to mathematical logic.

With that crash course in formal semantics relevant to grading classes of semantics related to mathematics, what can be shown is that an LLM is missing a fundamental requirement to "do math". LLMs have no teleological aspirations, and therefore are not capable of playing the game of math because they do not understand the rules of math. From the SEP's article Formalism in the Philosophy of Math (SEP):

One common understanding of formalism in the philosophy of mathematics takes it as holding that mathematics is not a body of propositions representing an abstract sector of reality but is much more akin to a game, bringing with it no more commitment to an ontology of objects or properties than ludo or chess.

LLMs simply don't have the operational, denotational, and axiomatic semantics in their engines to receive, manipulate, and succeed in the axiomatic reasoning necessary to do math. Nor do they have anything that remotely resembles the basic natural language ontology (SEP) to have even the simplest of type theories to accommodate the hyperonymy, synonymy, or identity required to relate mathematical primitives. LLMs at best manifest a statistical awareness of the syntax of what a mathematician is likely to say, and therefore possesses mere faint traces of semantic awareness.

As any use-based theory of acquisition of grammar presumes, semantics lies in the usage of the propositions of math, and LLMs simply are not aware of propositional content in any way. Without awareness of the propositional semantics, then the notion they can meaninful engage in the manipulation of propositional content is not possible. They do not have the ability to engage in speech acts nor do they possess any game-theoretic abilities. Therefore, there are many reasons to see Platonic reasoning as metaphysically flawed (such as it's rejection of simplest empirical presumptions of burden of proof about existence granting its ontological extravagance), but LLMs are not evidence against it.

17
  • 6
    Whereas an automated proof checker might be seen as an idiot mathematician (GPS comes to mind as an exemplar), an LLM is like a brain-dead parrot incapable of doing little more than replicating the surface syntax blind to the deeper structures that inhere to it.
    – J D
    Dec 6, 2023 at 15:35
  • 1
    Probably worth noting that "appear to be reasoning" and actually reasoning are different things entirely. Dec 7, 2023 at 15:55
  • "In fact, human grammars are processed by tokens and accompanying morphology at several levels: the morpheme, the lexeme, the phrase, and the sentence." my understanding of the structure of current LLMs is that they are able to maintain and represent the context at all of these levels and above (paragraph scale). So I do not see this feature as distinguishing humans from these models.
    – Dave
    Dec 7, 2023 at 18:11
  • 1
    "LLMs have no teleological aspirations, and therefore are not capable of playing the game of math because they do not understand the rules of math" Isn't the heart of this question whether the game/rules of math are fundamentally syntactic rules that could, in principle, be learned by an LLM? I'm not sure how or whether having teleological aspirations is relevant to that question.
    – Dave
    Dec 7, 2023 at 18:14
  • @Dave All use of rules is inherently goal-driven.
    – J D
    Dec 7, 2023 at 20:49
4

Dougherty[95] is a continuation of an examination of a topic in the theory of large cardinals (those which are critical points of elementary embeddings) that is some many years old, the abstract for which makes note that:

Some of the main results here were discovered through examination of bit patterns from extensive calculations in the finite algebras; this may be the first serious example of computer-aided research in the theory of large cardinals.

Modulo algebraism-minded formalism, the paper says:

The question of whether the large cardinal is necessary for this purely algebraic result [about the embedding operation] was resolved by Dehornoy [1], who proved the result without using any large cardinals; in fact, Dehornoy’s proof goes through in a very weak system of arithmetic (Primitive Recursive Arithmetic).

So we have known for some time that even (some of) the higher flights of set theory are amenable to formalistic formulation(!). That is, computers have been useful in learning about something that otherwise seems the most realism-minded from the outside and at the outset (the doctrine of very large cardinals); advances in the intervening time period cannot have but helped us to algebraize the matter all the more (see e.g. Dimonte[18] for extensions of Dougherty's "program" (if you will) to rank-into-rank embeddings above I3).

I should also like to note that some "this diagram commutes" reasoning is used in at least some of the theory of large cardinals (see Jech's great big book on set theory, pg. 342, for a commutative diagram in the illumination of ultrapower theory). Category theory (where diagram chasing has made its name, as far as I know) is sometimes taken to be algebraism/structuralism par excellence, or along such lines anyway.

Now, if even the theory of large cardinals can be processed by computers/in a formalism-minded manner, does this mean that even the most seemingly Platonistic zone of mathematical inquiry can be converted into a set of problems of mere symbolic manipulation? Compare the situation to that of the aesthetic philosophy of AI image generators: just because an AI can produce what look like works of art, is then visual art reducible to formal manipulation of imagery? (C.f., then, AI music generators, of course.) Or if AI sex machines could deliver physically satisfying encounters, would this collapse romantic attitudes to the mechanics of satisfaction? I don't mean these questions rhetorically, though: I don't really know the answer to them all. The stage we're at with respect to these matters is not conclusive enough (I'm sympathetic to e.g. eliminative materialism and physicalism anyway, so I suspect that it's possible for those questions to have reductionism-friendly answers, although there will still be some metaphysical circularity at the deepest levels, as far as I can tell from trying to evaluate whether e.g. logic or mathematics is "more fundamental").

4

IMO we may consider the link between math and language (maybe more... maybe math is language).

Consider a ChatGPT answering our questions. What is it doing? Is it speaking? Or it is only simulating a human speaker?

The same for an "engine" doing formal math. If formal math is an abstract model of mathematical thinking, we have a machine simulating a human mathematician. If instead formal math is all that there is in math thinking, then we have an intelligent machine doing mathematics.

1

Consider the following algorithm to prove a theorem T:

Loop:
    Generate a random string of symbols, S
    Check S with a formal proof checker
    If S is a correct proof of T:
        Output S and halt
    Otherwise:
        Keep looping

This algorithm is pretty dumb, isn't it? And yet it can write formal proofs.

Are we enlightened about the nature of mathematics by the existence of this algorithm? I'm not. Are we in checkmate? I actually don't know what you mean by that, but I personally don't feel checkmated by this algorithm.

Now I happen to know a few researchers who work in the domain of formal provers and who dedicated quite some time to exploring potential uses of ChatGPT to generate formal proofs or just solve mathematical problems.

So far their conclusion is: if you have an extremely carefully engineered prompt, and I do mean extremely carefully engineered, then it is possible to ask ChatGPT to solve simple math problems, and reasonably hope that at least some of the time it won't output total garbage.

So in practice they are literally, to this day, using the following algorithm:

Loop 100 times:
    Feed ChatGPT a very thorough prompt that describes the problem and encourages ChatGPT to not output total crap and describe what a mathematical reasoning should look like and show a few examples of solved problems that are similar to the problem at hand and insist that the reasoning should not be too self-inconsistent
    Include a randomness parameter so that ChatGPT doesn't always give the same answer
    Get output S from ChatGPT
    Check S with a formal proof checker
    If S is a correct answer to the problem:
        Output S

With this algorithm they get pretty good results if the problem is simple enough.

My philosophical conclusion is that even a blind person may hit a target once in a while. At least they may, provided they have a hundred darts, and someone is behind the target shouting "the target is over here!!".

2
  • 1
    Good answer! I’m being a bit hyperbolic for effect; but the specifics of the search loop are a good starting point. The idea is something like “the second is more successful, but that’s not the same thing as recreating properly mathematical cognition”. Formalism still has to challenge that point, even if the success isn’t disputed.
    – Paul Ross
    Dec 18, 2023 at 8:39
  • @PaulRoss ChatGPT is still fairly recent, but there are several similar questions about "cognition" of the games Chess and Go. AIs are definitively not conscious nor sentient, but depending on your definition of "cognition" and "understanding" it may or may not make a lot of sense to say that a go-playing AI has an understanding of the game. Recently I wrote an answer to this question: Consciousness and Understanding of Physics, Mathematics and Philosophy
    – Stef
    Dec 18, 2023 at 14:44

You must log in to answer this question.