3

In the book Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought, Hofstadter argues that analogy is a fundamental mechanism of neurocognition.

Now, I've been pondering for a long time how to test if artificial intelligence is capable of this function or not. My personal experiment (based on something I was thinking about) is that Chat-Gpt is capable of analogy making.

This seems to go against the grain?

Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought (1995): "The ability to make analogies is a fundamental aspect of human thought, but it is not clear that AI systems can replicate this ability."

John McCarthy What is Artificial Intelligence?: (2007) "AI systems do not really understand analogies"

Perhaps I'm missing something going against such heavy weights. How do philosophers suggest testing this?

I've seen structural analogies here but they aren't as sophisticated as the one I proposed.

9
  • 1
    Anything that can do it visually? That boat is like the smell of coffee
    – user66697
    Jul 8, 2023 at 18:44
  • 1
    oh sorry, i didn't mean to give you the impression i wasn't. yes, fine, just excited by your question ha
    – user66697
    Jul 8, 2023 at 19:02
  • 2
    I would question whether prompting an AI to discuss an analogy is evidence that the AI is using analogy or any other neurocognitive process.
    – nwr
    Jul 8, 2023 at 19:51
  • 1
    turing test on analogical thinking?
    – user66697
    Jul 8, 2023 at 20:20
  • 1
    @loop yes it seems to have passed my turing test of analogical thinking atleast :/ Jul 8, 2023 at 20:42

4 Answers 4

1

I found a good blog post on exactly this topic: Can GPT-3 Make Analogies?, which summarises research.

It was suggested to me that it might be appropriate to think of Large-Language-Models like ChatGPT, as comparable to Kahneman's 'System 1', as basically the selection of habits that have been succesful. But what we need to jump this hurdle, is to also have a 'System 2' equivalent. This is evidenced by hiw ChatGPT does references, where it uses the pattern of genuine references, while failing to integrate knowledge of what they do and how they function, which relates to structuring information in non-habitual ways.

It's an interesting challenge, to think about whether beavers building dams, or birds building nests, are intelligence, or what they lack. With the edge case being, that play behaviour seems to be gene-driven like dam-making & nest-building but, relates to cognitive flexibility and environment exploration in a different more systematic way. I suggest that play and playfulness, are going to be necessary for the development from simple 'habitual' analogies (eg as illustrated by How to Use ChatGPT's Analogy Feature to Turbocharge Your Presentations), into the kind of truly creative ones Hofstadter is talking about (I think of the line of poetry "the cypress is like the ghost of a dead flame", that we discussed here: How is asymmetry of metaphor an important part of object-oriented ontology?).

How can we verify an AI is doing this kind of creative flexible innovative analogy, rather than a rote comparison that had basically become part of our linguistic habits? (I think of how we don't even notice that 'space shuttle' involves a descendent analogy from weaving) I think we are looking for something that goes beyond dam-building and nest-making but with language, and demonstrates some deeper appreciation, of a new analogy and an innovative way to apply it, something which isn't already an established trope or habit in language.

4
  • Thanks. "I started working as a research assistant for Douglas Hofstadter at MIT in 1983." Is there some classification system of anology? Also is there a seperate metric for following anologies? Jul 10, 2023 at 10:46
  • @MoreAnonymous: There is en.wikipedia.org/wiki/Metaphor_identification_procedure I was just listening to a podcast about en.m.wikipedia.org/wiki/Metaphors_We_Live_By which relates them to cognitive maps, which may be irreducible to non-metaphor descriptions.
    – CriglCragl
    Jul 10, 2023 at 11:45
  • Ah thanks! Would you mind sharing the podcast? I'm into those too. I'd be up for recommendations Jul 10, 2023 at 12:03
  • @MoreAnonymous: Sure. It was this Very Bad Wizards ep250: Metaphors All the Way Down player.fm/1BTmlqJ Particularly good episode I thought.
    – CriglCragl
    Jul 10, 2023 at 15:44
3

A key issue here is how to diagnose whether AI is capable of something. Some people have a "behaviorist" (or "operationalist") approach to this, i.e., they would say that "AI is capable of XXX" if it looks like it can do it. I'd furthermore distinguish a formal from an informal behaviorist approach. "Formal" means that criteria for diagnosis have to be defined transparently before the experiment in such a way that it can unambiguously be decided whether in the experiment criteria are fulfilled or not. (The Turing test for example is informal in the sense that it doesn't define exactly the criteria by which observers decide whether a system is intelligent or not.)

An advantage of formal criteria is that they are (ideally) objective; they don't rely on interpretation. Experiments and conclusions from them will be reproducible (even though due to variation not necessarily reproduced) and transparent.

A disadvantage of formal criteria is that algorithms handle formalities, and can more or less easily be trained to fulfill specific formal demands, arguably without actually proving "intelligence" or "understanding" in this way. Formal criteria can often be fulfilled in ways that would subvert what they are supposed to show, so to say.

A general objection against behavioral approaches (and particularly formal ones) is that concepts such as "intelligence" and "understanding" can be understood as not exclusively referring to observable behavior, but rather to some states of consciousness that cannot be observed from the outside, at least not by just looking at behavior.

There is a lot to be said for this objection, but it obviously makes it very difficult to make any (even negative) statements of the kind that this or that AI system has (or doesn't have) "real intelligence" or "understands" this or that.

In any case I think that the distinction is an important one to have in mind. We can apply behaviorist criteria, but then we have to keep their limitations in mind, namely what behaviorist observation cannot tell us.

Regarding handling analogies by large language models (LLMs), understanding of how they work is crucial. The language models are based on a very large body of training texts, many of which use analogies. They will come up with analogies because they are trained to behave as the body of training tests does (trying to reproduce the observed distribution of follow up words, sentences etc.), obviously prompted/modified by interlocutor's interventions (trying to reproduce conditional distributions, reinforced by objective functions that assess output according to training assessments given by human beings) and whatever of the general environment of the discussion is used by their model.

They may in principle also come up with creative analogies by potentially mixing closely related trained analogies up, and this may work (or not), maybe even by learning something like a typical structure of texts around their use of analogies. I'd for sure expect them to be able (in a behaviorist sense) to make up new analogies, however many of them may not "work" that well. This can be amended (if somebody wants to construct an LLM that is particularly good at analogies) by providing human assessment of as many as possible invented analogies as additional training.

None of this requires an "understanding" by the system to explain it; the complexity and richness of the training data and the complexity of the encoded model to extract information from the training data are enough. To what extent you still want to call these features "capability" or "understanding" is really up to you, or more precisely, how behaviorist you want to be about these concepts. Note in particular that the system will always in some sense reproduce the variety of training data generated by humans, and the specifically human assessments of what "works" used for training the objective function of reinforcement learning. Any LLM "creativity" is the result of random generation (according to learnt conditional distributions) plus assessment by an objective function designed and potentially updated by human training. (People who follow a behaviorist diagnosis paradigm may say, "maybe humans do it like this, too, so where's the difference?")

For some reason there doesn't seem to be much discussion around these days of Searle's Chinese Room experiment, but in my view this is very relevant.

5
  • Are these LLMs trained on material found on the internet at large? I would trust their output as much as I would eat food I found in the road. Is that a helpful analogy?
    – Scott Rowe
    Jul 9, 2023 at 13:43
  • 1
    @ScottRowe Large numbers of documents are needed, however designers may want to be selective to some extent. Wikipedia writes, e.g., "ChatGPT's training data includes software manual pages, information about internet phenomena such as bulletin board systems, and multiple programming languages. Wikipedia was also one of the sources of training data for ChatGPT." Jul 9, 2023 at 14:11
  • 2
    @ScottRowe en.wikipedia.org/wiki/ChatGPT#Training Human trainers/raters are used to improve reliability. Anyway, personally I don't think "trust" is the right attitude toward these systems (but then it may not a good attitude toward human beings either). Jul 9, 2023 at 14:15
  • 1
    @ScottRowe It's getting worse. People who are getting paid to find training material have started using the output of LLMs as training material. Much less effort and a good money maker.
    – gnasher729
    Jul 9, 2023 at 22:20
  • @gnasher729 LLMs drinking the Kool-Aid?
    – Scott Rowe
    Jul 10, 2023 at 2:25
-1

LLMs today have zero intelligence. They create sentences that are intended to look similar to what correct answers would look like, and that's it. Sometimes what looks similar to a correct answer is indeed correct. Sometimes it's absolute nonsense. Sometimes it is highly defamatory nonsense.

2
  • Being able to beat humans at Jeopardy is not nothing.
    – CriglCragl
    Jul 10, 2023 at 0:27
  • No intelligence needed for that. My computer stores about 2000 records which I can’t remember and nobody claims that makes it intelligent. But making up non-existing court cases or non-existing sexual harassment cases, that’s a sign of intelligence, right?
    – gnasher729
    Jul 10, 2023 at 10:26
-1

how to test if artificial intelligence is capable of this function or not.

You define your variables and terms. Then you develop an experiment relative to those variables and terms. And then you run the experiment. And then afterwards you argue whether or not the AI satisfied the variables and terms as you defined them.

See also: Operational definition

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .