I've quite often seen AI respond to John Searle's Chinese room argument by accepting the systems reply: while the man in the room doesn't understand Chinese, the room (the system) as a whole could - or at least the Chinese room argument doesn't conclude that it could not.

Yet the systems reply doesn't seem to address the main problem the CRA poses for AI. One of Searle's premisses says that symbols are semantically vacant, are intrinsically meaningless, that in themselves they don't indicate what they mean or refer to. And all the computer gets is the symbol (i.e. tokenized shape).

Given that computers do process symbols, the sensory symbols received from digital sensors will not indicate what was sensed. How, then, is it possible for a computer to come to know and understand the world?

  • 2
    How indeed. I wonder why anyone thinks it might.
    – user20253
    Aug 22, 2018 at 11:40
  • 1
    I don't see what AI has to do with any of the points raised in this post. You can ask the same question about how a human comes to understand symbols, if at all.
    – user6559
    Aug 22, 2018 at 20:24
  • @user6559 Humans have instincts; if AI is to have instincts, we have to program those in.
    – wizzwizz4
    Jul 21 at 15:45

4 Answers 4


Let's look at "Chinese Room". The words traverse the optic nerve as a complicated neural pattern with no semantic significance. Once they hit the brain, the brain can assign meaning to the letters, words, and phrase. If they hit the brain of someone only literate in Chinese, they'd look like meaningless symbols.

Therefore, the room and the brain are alike in receiving meaningless input and assigning meaning. The processes in the room assign meaning to the symbols that Searle doesn't see any meaning in.

It's important to remember the context of the Chinese Room, the Turing Test. In this, a human communicates with two text interfaces (teletypes in the original), one another human and the other a computer, and attempts to determine which of the other is the human and which the computer. This has been widely been considered a test of artificial intelligence. (Turing did not claim that; he predicted, inaccurately, that the computers would be reasonably successfully in 2000.) Therefore, the comparison is between the Chinese room getting Chinese text and a Chinese speaker getting Chinese text.

  • 1
    "the room and the brain are alike in receiving meaningless input and assigning meaning". This is the issue that seems crucial to me. How does the brain get the meanings that it assigns? And where does it get them from? Not the input streams – they are meaningless. But they are also the only things the room (and brain) gets. Conclusion: the input streams must not be meaningless. We know where the meanings come from: sensors. We know what the meanings are about: the things sensed. But how is the meaning embodied in the stream? = THE question.
    – Roddus
    Aug 24, 2018 at 2:41
  • Let's take this up a level. This web page would have been completely meaningless to me about sixty-three years ago. (I remember looking at a newspaper and not being able to interpret any of the writing.) If there is intelligent life elsewhere in the Universe, it would be meaningless to them. There's not enough text here to analyze it and determine any meaning. However, you and I find meaning in it. The stream isn't arbitrary or random, but it does not in itself convey meaning without being processed by certain means. Sep 7, 2018 at 18:28
  • I agree that the stream needs to be processed by cerain means in order to reveal (if that's the right word) the meaning it contains. The really interesting part to me is understanding how the meaning is embodied in the stream (which will indicate how to extract it an make it permanent in storage). But I'm not sure I agree that meaning can be determined with enough text. For attacking encrypted data, the more ciphertext you've got the better, but I don't think this idea works for text and meaning. Bickhard and Terveen wrote good book on this (they called text, encodingism).
    – Roddus
    Sep 17, 2018 at 1:31

I guess much here depends on the definition of "knowledge" and "understanding" -- today, self-driving cars already learn about their environments. (I considered putting quotation marks around "learn" but it is common practice not to in this context.) So for a weak sense of knowledge and understanding, where we require only (say) that there is information in the system that correlates with the world and is used to take desirable actions in the world, today's systems already acquire it. Of course one might be interested in a stronger sense of knowledge and understanding, perhaps involving some conscious awareness. This is what Searle's argument is about, and we're not going to settle that discussion here...

  • 1
    I'd probably dispute the self-driving claim. The image training sets used for deep learning artificial neural nets are made of a gazillion images, but usually each is annotated by a human who enters codes against object types: tree, stop sign, pedestrian, van, child, etc. Good training sets are available online. Human annotation isn't really the system acquiring the knowledge or of AI systems "learning about their environments". It seems more part of a process where a human defines the causation of the system by using the human's knowledge.
    – Roddus
    Aug 24, 2018 at 0:17
  • As for conscious awareness, I know Searle has spent some time on this topic. But I really see consciousness as a separate issue from semantics. It's conceivable that a system could have a quality semantics that enables it to survive in the wild, but has no consciousness. In fact there's been a debate about what is the evolutionary benefit of consciousness. Whereas the evolutionary benefit of a semantics is fairly obvious. I see the CRA as part of the programme of trying to understand how a machine could get a semantics. Isn't this an earlier issue – before that of consciousness?
    – Roddus
    Aug 24, 2018 at 0:18

Algorithmically calculable answers must be part of Godel-incomplete systems, with true but unprovable statements. But a strange loop system can form a tangled hierarchy, a network of reinforcement and doubt, like language in use, where tentative uses for symbols are used then refined and meaning created relationally and through interplay and interaction.

When our brains cease to get input from the optic nerve, they don't close part of the brain. It (eventually) hijacks and bootstraps the area to continue to try and create a model of the world as isomorphic as possible to it - as demonstrated iteratively, by patterns of interaction, cross-reference, etc - ie by use.

Your premise that symbols are semantically vacant is untenable, in terms of viewing language in practice - they are imbued with it by use. Mathematical systems reveal consequences to statements already made, that is they don't generate meaning but unpack it. Creative mathematical thinking however, works forwards to consequences & backwards to axioms, then around again, in interaction with the world, creatively.

How, then, is it possible for a computer to come to know and understand the world?

Through interaction, trial and error, exploration, heuristics. Just like us. Whatever means it had to interact, they would be senses - including mental models & simulations, which we use for instance in motion prediction for game playing.

The Chinese room is only intelligent like a deck of cards, or an abacus, say. True Artificial General Intelligence would have to be in a strange loop, not such a flat hierarchy with explicit defined rules.

  • You make some interesting statements about strange loops, their formation, and their use. Do you have some pointers to explication? Aug 22, 2018 at 17:58
  • @CriglCragl You say "Your premise that symbols are semantically vacant is untenable, in terms of viewing language in practice - they are imbued with it by use". So symbols are imbued with semantic content by virtue of use of the (those same) symbols. But isn't what really happens this: brains get imbued with semantic content by way of use of external symbols (and other things)? The content doesn't inhere in the symbol, rather it inheres in the brain? This issue of sensory semantic content seems much more primitive than any mathematical understanding of human processes and structures.
    – Roddus
    Aug 24, 2018 at 2:25
  • @Roddus Niether the brain nor the symbol. The culture of use is between the two.
    – CriglCragl
    Aug 26, 2018 at 0:30
  • @DavidThornley We discussed it on here philosophy.stackexchange.com/questions/22926/… I esprcially recommend the Wittgenstein-Nietzsche-Rorty article I posted there. If you seriously consider Munchausen's trilemma, strange loops are the only answer. There is no 'correct place to begin', there are only reinforcing movements through thought-structures using various modes, stepping in and out (axiomatising), unifying (regression), and looping (circular reasoning).
    – CriglCragl
    Aug 26, 2018 at 0:40

I mean one possible way to attack it would be to point out that, just because you aren't aware of the semantic information doesn't mean that there isn't semantic information intrinsic to the subject.

So you can try to find patterns within the noise and relations between the patterns. So that you can create word clouds without knowing what the words mean. Like if a sentence works with all symbols the same except one you can infer that there is an equality in some sense between those two words. You might be able to infer important ideas by the number of connections or filler words by the combination of being used frequently but the ability to be left out of a sentence and still be valid according to the syntax.

Now a human observer is capable of inferring quite a lot from the simple fact that this person is a human as are the people who invented the language so there are certain quirks of the human condition that you could infer are so universal that you might start trying to map them.

But even if that language was not Chinese but completely alien to any human you might still try to find regularities, make predictions for the output, check whether the predictions and the actual responses match and refine your predictions because of that i.e. do science.

And to some degree that is already happening with machine learning. You give a program an input, they literally guess (generate a random number) and output that. Then the user tells them the real output and they compare the guess and the real value and change their guess. So idk if they are meant to guess a result of an addition like 2+2 then they might guess 3, receive 4 as answer compute 4-1 = 1 which is positive so they change their parameters to create a bigger number let's say 5 where again they compute 4 -6 =-2 which is negative so they adjust the parameters to get a smaller number but bigger than 3 and so one until they got to 4.

Now they technically still don't know how to add, they just know that 2 on one input and 2 on the other input equals 4 as the output. But from the fact that a swap of the input results in the same output you'd already necessitate an intrinsic symmetry. Now you could be wrong assuming those are numbers but in reality those are words, but you'd still have given them some sort of semantic meaning even if it were wrong.

So if you assume that this is not a 1 way communication but that the computer uses it's outputs to make inferences on the environment than it could also progressively improve it's guesses and assign meaning to things that are arbitrary and meaningless.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .