19

Searle's Chinese Room basically argues that a program cannot make a computer 'intelligent'.

Searle summarises the argument as

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

What are the counter-arguments? Including the standard and not so standard. This notion of 'intelligence' intrigues me and I wish to see what other people have written said. Point me to your favourite counter-argument.

I have read Levesque's "Is It Enough to Get the Behaviour Right?"(http://ijcai.org/papers09/Abstracts/241.html) (full paper here) response. As Levesque is an AI researcher I would like to broaden my appreciation of counter-arguments from other fields.

10
  • It's a Mechanical Turk in reverse, and it took a long time for machines operating purely on "instructions" just to master chess. To pass a Turing Test in a human language, the operator's "instructions" would need to be infinitely long, detailed, and effectively unreadable by any finite embodied "experiencing" human being. Not sure, but I suspect this answers isn't as misplaced or superficial as it at first seems. Symbolizing experience is an infinite regress. The hypothesized "instructions" are implausible. Commented Nov 17, 2021 at 16:22
  • 1
    @NelsonAlexander I agree with Searle, but I don't agree with your point that a machine's instructions would need to be infinitely long. After all we're made up of a finite number of atoms, we have a finite number of neurons and neuronal paths, etc. We're finite and we are self-aware and intelligent. Whatever the "secret sauce" of consciousness is, it need not be infinite.
    – user4894
    Commented Nov 17, 2021 at 23:00
  • Yeah, I almost didn't use that term, which is a conversation killer. But when I say "symbolizing experience is an infinite regress" I might analogize it to the Coast of England problem, where ever more accurate measurement or definition increases the "coastal length" towards infinity. The physical sciences must introduce "limits," but paradoxically the idea that we have "finite" atoms in our bodies does not preclude a (virtually) infinite number of ways to describe them. Nor, I suspect, is there a finite number of meaningful Chinese Q&As. Commented Nov 18, 2021 at 1:38
  • @user4894 you assume a material interpretation of the universe, which is a particular metaphysical perspective. I recognize the character limits, but it seemed good to me to acknowledge this rather than merely put it forward as "simply the case." It is coherent that within every conscious being lies an infinite and eternal substance (or essence or spirit or whatever particular non-material variety of existence you may want to qualify) which is the natural prerequisite for consciousness. There are many respected traditions of this conviction and if this is the case we're not in-fact finite. Commented Nov 22, 2021 at 23:06
  • @GoodOl'SaintNick "you assume a material interpretation of the universe" This is not anything I said or a logical consequence of anything I said, so you lost me entirely here. "It is coherent that within every conscious being lies an infinite and eternal substance (or essence or spirit or whatever particular non-material variety of existence you may want to qualify) which is the natural prerequisite for consciousness. " Seems like a stretch. Evidence? What else lives in the immaterial realm? The Baby Jesus? The Flying Spaghetti monster? Vishnu? 'Splain me please.
    – user4894
    Commented Nov 23, 2021 at 0:41

10 Answers 10

6

Searle's Chinese Room experiment doesn't make any sense, but some context is required to bring to light the contradiction inherent in his argument. At the time of its writing, he was attempting to fight an extreme position among AI theorists who supposed that all of the properties of understanding, whatever we agree that entails, could be reduced to or captured by a sufficiently powerful computer acting on symbols by completely formal methods i.e. mathematics.

In a nutshell, the CR experiment supposes it possible for a Chinese-speaker to pass meaningful messages in their native language to a man in the room who doesn't speak it and have them respond intelligently or meaningfully by following a purely algorithmic or formal process, whatever that may entail i.e. using a lookup table, following preset rules, etc.

This is a strange way to argue for intuition, or rather for intuition having the property of non-reducibility. Because if the man in the room can communicate with the man on the outside despite his lack of understanding, and the semantic gap is bridged by a completely formal process, then how is this anything but semantics being reduced to syntax? Exactly what Searle wants to argue against. Once again, the man on the outside understands what goes in and what comes out of the room, and the only means the man has to communicate is to follow a mechanical recipe for writing.

3
  • 4
    It doesn't make sense to you because you have misread Searle's argument that syntax is insufficient for semantics. He is not commenting upon intuition. Note the the person inside the box - the computer, i.e. the person doing the computation - does not understand Chinese. The "semantic gap" remains unbridged.
    – MmmHmm
    Commented Oct 28, 2016 at 17:09
  • 1
    @Mr.Kennedy I do not agree with this answer, but, being an argumentative sod, I do not agree with your comment either. In insisting that the CR could only work if its operator understood Chinese, Searle is tacitly assuming that the instructions themselves cannot give rise to semantics. If the CR argument is not to be immediately dismissed on the grounds of circularity, the 'syntax cannot beget semantics' argument must be made elsewhere.
    – sdenham
    Commented Apr 23, 2018 at 13:30
  • @sdenham: I would go even further than that. CR is wrong because it tacitly assumes that the distinction between syntax and semantics is essential rather than functional, or absolute rather than relative. When you bring in metamathematics and model theory, it starts to look rather silly.
    – Kevin
    Commented Jul 20, 2022 at 23:28
17

The Stanford Encyclopedia of Philosophy's entry on Searle's "Chinese Room" thought experiment identifies three major categories of objections and goes into some detail on each (and provides generous citations.) Here is their high-level summary of the categories of possible responses to the argument:

(1) Some critics concede that the man in the room doesn't understand Chinese, but hold that at the same time there is some other thing that does understand. These critics object to the inference from the claim that the man in the room does not understand Chinese to the conclusion that no understanding has been created. There might be understanding by a larger, or different, entity. This is the strategy of The Systems Reply and the Virtual Mind Reply. These replies hold that there could be understanding in the original Chinese Room scenario.

(2) Other critics concede Searle's claim that just running a natural language processing program as described in the CR scenario does not create any understanding, whether by a human or a computer system. But these critics hold that a variation on the computer system could understand. The variant might be a computer embedded in a robotic body, having interaction with the physical world via sensors and motors (“The Robot Reply”), or it might be a system that simulated the detailed operation of an entire brain, neuron by neuron (“the Brain Simulator Reply”).

(3) Finally, some critics do not concede even the narrow point against AI. These critics hold that the man in the original Chinese Room scenario might understand Chinese, despite Searle's denials, or that the scenario is impossible. For example, critics have argued that our intuitions in such cases are unreliable. Other critics have held that it all depends on what one means by “understand”—points discussed in the section on the Intuition Reply. Others (e.g. Sprevak 2007) object to the assumption that any system (e.g. Searle in the room) can run any computer program. And finally some have argued that if it is not reasonable to attribute understanding on the basis of the behavior exhibited by the Chinese Room, then it would not be reasonable to attribute understanding to humans on the basis of similar behavioral evidence (Searle calls this last the “Other Minds Reply”).

0
6

I quite like the chapter in Jack Copeland's Philosophy of AI book on the Chinese room. Even though it's an introductory book, it goes into quite some breadths and depth. The chapter has a summary of the arguments against the Chinese room, and the retorts from Searl and friends, plus counter retorts. Rest of the book also sets the context of the discussions very well and definitely goes into the various ways people understand and use the term 'intelligence' in the AI debates (you might find some you like).

A not too standard problem I have with the argument is its general argument form (non-standard problem because it's not specific to the Chinese room argument).

The Chinese room is a modal argument which is deployed in a lot in philosophy of mind. Examples of other modal arguments include:

  • the zombie argument from David Chalmers
  • the inverted spectrum argument
  • brain in vats
  • Descarte's evil demon argumen

They are generally of the form: it is possible that despite the appearance of X we cannot know that X because of an equally plausible alternative understanding of the situation which exclude X. In this case, It is possible that even if the instructions lets the box cogently parlay with a Chinese Turing tester, it is more than equally plausible that no real understanding is occurring (sneakily established by the fact that we'd all had prior agreement that the person inside has no understanding of Chinese, and he's the only one doing anything).

I am not sure how seriously to take modal arguments in general, because I feel they are invariably sneaky, and often leads to unfruitfulness due to the way things are set up. A successful modal argument will give you no greater reason to believe either of the options, and the rhetoric is usually set up such that you forget one of the options consists of a natural and familiar scenarios (sitting by the fire, seeing a person, receiving red qualia) and the another scenario which is speculative (deceived by evil demons, seeing a zombie, receiving green qualia instead of read). So each camp can't really engage the other side to have a fruitful discussion because each has a reason to believe that the other person's argument is flawed -- with no real criterion to evaluate either side of the claim. Sneaky IMHO.

4

Searle's Chinese Room arguments depends initially on the assumption that, if there is an intelligent system, some component of the system must be intelligent. Carrying this a long way to a conclusion, each lepton and quark in the Universe must be intelligent to some extent. The alternatives to that conclusion are either (a) there is nothing intelligent in the Universe, or (b) intelligence is an emergent phenomenon. I'm going with the latter.

Searle then attempts to answer all objections (such as emergent intelligence) by indulging in more or less relevant discussion before claiming it reduces to the original Chinese Room, which he showed was not intelligent. This is an excellent example of begging the question, in the classic definition (assuming the proposition to be proven).

The most relevant other discussion is the claim that a computer could not be intelligent in the human sense without having human-type senses to sense things like a human. How is a computer to know what a hamburger is without eating one? This seems like a good line of argument, but Searle does not pursue it.

Searle claims, without support, that intelligence must be biological, and states that we know intentionality is biological in nature. I haven't seen this claim anywhere else, so I don't think we do know that.

3
  • Your options a) and b) don't include the option in which intelligence is held by an immaterial mind rather than an emergent mind. I don't quite recall how that option is framed in the formal literature, but I know it's framed in the literature. Commented Dec 20, 2018 at 19:19
  • 1
    @elliotsvensson Good point. The question it raises is what it attaches to. Commented Dec 22, 2018 at 21:22
  • That's the biggest point. Either Searle's argument is flawed (see the other arguments), or humans aren't conscious, or humans are MAGIC. Commented Jun 18, 2019 at 16:10
2

The OP wants to know what counter-arguments have been made against the Chinese Room Argument.

In Minds, Brains and Programs John Searle responds to six counter-arguments. Here is a paraphrase of them:

  1. The Systems Reply. The human who does not understand Chinese when running the steps of the program is only part of the larger system that does understand.

  2. The Robot Reply. Put the computer doing the formal language processing inside a robot.

  3. The Brain Simulator Reply. Write a program that simulates the actual neuron firings in the brain of a native Chinese speaker.

  4. The Combination Reply. Create a program that combines all three of the previous responses.

  5. The Other Minds Reply. We don't know that other people understand Chinese except by their behavior either.

  6. The Many Mansions Reply. Eventually we will be able to build devices that do understand.


Searle, John. R. (1980) Minds, brains, and programs. Behavioral and Brain Sciences 3 (3): 417-457 http://cogprints.org/7150/1/10.1.1.83.5248.pdf

1

For instance, Daniel Dennett's objections. Dennett calls Searle's thought experiment a "boom crutch", something that hinders thinking, as opposed to intuitive pumps, that are considered to be helpful.

Dennett claims that Searle's argument is phrased in a specific way to limit your options, hence a "boom crutch". Searle assumes some highly advanced Symbolic rule-based AI. He wants you to believe that it could not understand meaning because it merely assembles some input symbol to an output symbol according to the rule-book. But, immediately, we are faced with objections.

Firstly, according to Dennett, the argument shows Searle's lack of understanding on how computers and how computation is achieved. It would take billions of years of writing code for a purely rule-based system* to be able to obtain a meaningful human-level conversation. Just how many rules would it have to have? One has to construct rules convincing enough so that each response is as if a real person produced an answer, leading the conversation in Chinese, on a conversational level. Try to construct all of these rules using Symbolic logic; it would be an unbelievable number.

Secondly, even if we achieved the device that utilizes a rule-based system and such system can talk about movies, weather, Greek cuisine and ask personal questions, did we not just implement the entire process of human cognition, along with understanding? The "system" that Searle presents cannot merely assemble input to output because it has to rely on billion years worth of rule sets on what to say, and what not to say. It has to draw logical inferences involving large portions of empirical data about the world. It has to have common-sense. And if it does, it has no problems with understanding anything whatsoever, semantics included. This kind of sophisticated rule system is impossible to localize in some rudimentary boxes with instructions. What one needs is computation to do that efficiently, if at all.

Building on Dan's argument, it is clear that John Searle wants you to forget about a crucial part of what makes the system; where and how the actual stuff must be going on. If only banal manual assembly of input to output was the case, there would be no possibility of meaningful human-level conversation. Literally, no one claims that a simple program that prints out "Hello World" understands semantics. In the same way, a man in the room is unable to have a convincing conversation. Moreover, a conversation that doesn't take years for him to process, if he's using boxes of billion years' worth of information. This is a charge of natural possibility. Searle wants you to imagine something trivial and draw obvious conclusions from it - that such system cannot understand, so "strong AI" altogether is implausible. However, to pass anything resembling a Turing test, it is non-trivial. The entire discussion of the rule-set system, which has to exist aside from speech assembler, is almost "deleted" from Searle's argument.

Otherwise, in what way does the AI get the rules from? Of course, the only option is this gargantuan set of rules written over billions of years. How can one squeeze that rule-set so obtaining information is feasible? Only by using a computer. If a computer is practically used, then the computer can yield understanding.

By shifting your attention to a dumb assembler, Searle makes you forget where the real work must be going on, i.e. in the complex computational unit responsible for (and capable of) human-level cognition because otherwise, it could not pass a Turing test, which Searle's experiment is a variant of. The speaker on the other end needs to be well convinced that he/she is speaking to a real person.

*- As opposed to, say, Machine-learning system.

11
  • So in your third paragraph, you seem to be saying that Dennett believes a computer program cannot carry on a meaningful conversation. That only strengthens Searle's claim that programs cannot understand. Commented Nov 17, 2021 at 22:07
  • @AmeetSharma Far from it. The simplistic "assembler" version of AI that Searle wants you to imagine cannot, and would not. Searle's argument, however, presumes that his AI can pass a Turing test, which, literally, makes an AI indistinguishable from a human. Basically, Searle wants you to forget about the humongous rule-set that makes such AI behave, and possibly even think, exactly like human. Commented Nov 17, 2021 at 22:39
  • Searle never said that a computer program can pass the Turing test. His only point is that passing the Turing test is insufficient for "understanding". I don't see the relevance of the humongous rule-set. Searle is arguing against the Turing test claim that behaving as if you understand means that you understand. Whether the rule set is 1 rule or humongous, the thought-experiment is the same. Commented Nov 17, 2021 at 22:43
  • Are you saying that a machine being able to talk about preferences, movies, and its hobbies does not have understanding? Because any simpler case won't do for Searle. Searle's experiment is a variant of Turing test, where third-party observer is unable to distinguish a robot from a normal person. Searle's thought experiment depends on you imagining such a simple a case, a banal case, and drawing an obvious, preconceived conclusion from it. Basically, Searle wants you to imagine something like a calculator, which, obviously, cannot understand. Commented Nov 17, 2021 at 22:48
  • "Are you saying that a machine being able to talk about preferences, movies, and its hobbies does not have understanding?" if it's simply running a computer program then it doesn't. That's what the Chinese Room Argument is trying to show anyway. You're making too much of the case being trivial. Suppose instead of a Chinese-speaking program, we have a program that simulates human behavior in all its respects. The argument is exactly the same. It'll be a bigger rule book. It's still just a program. The only thing the computer has is faster speed... but why would this make it understand? Commented Nov 17, 2021 at 22:57
0

As for your stated interest consider Searle's remarks here re: the ambiguity of "intelligence"

“Intelligence” is also ambiguous because it is ambiguous between real, honest to John observer-independent intelligence – as, for example, when a human being is thinking about something – and the observer-relative, derivative, metaphorical sense of intelligence – for instance, when we speak of my pocket calculator or computer as displaying intelligence.

Straw people, however, are easy to knock over:

Searle's Chinese Room basically argues that a program cannot make a computer 'intelligent'.

Searle is not arguing "that a program cannot make a computer 'intelligent'" (and this in whichever sense you mean program, intelligent or computer). Searle's Chinese Room demonstrates that semantics are not intrinsic to syntax. Big difference.

The Chinese Room Argument is a refutation of the computational theory of mind, not a refutation of "artificial intelligence". Searle is very clear that he is only arguing against what he identifies as "strong AI" and not "weak AI". Furthermore, he points out that we are biological machines in a sense of the words. As we know computational machines today (i.e. Turing machines), they are insufficient for anything more that syntactical manipulations and do not (read: can not) achieve semantic content. To understand Searle's argument, you will need to familiarize yourself with the distinctions between observer-relative and observer-independent, the epistemic and ontological senses of first- and third-person objectivity and subjectivity, as well the ambiguity of concepts such as "artificial", "intelligent", "information" and such.

To broaden your understanding of counter-arguments, first start with the actual argument you seek counter to:

Searle, John. R. (1980)
"Minds, Brains, and Programs"

Abstract.
This article can be viewed as an attempt to explore the consequences of two propositions. (1) Intentionality in human beings (and animals) is a product of causal features of the brain. I assume this is an empirical fact about the actual causal relations between mental processes and brains. It says simply that certain brain processes are sufficient for intentionality. (2) Instantiating a computer program is never by itself a sufficient condition of intentionality. The main argument of this paper is directed at establishing this claim. The form of the argument is to show how a human agent could instantiate the program and still not have the relevant intentionality. These two propositions have the following consequences (3) The explanation of how the brain produces intentionality cannot be that it does it by instantiating a computer program. This is a strict logical consequence of 1 and 2. (4) Any mechanism capable of producing intentionality must have causal powers equal to those of the brain. This is meant to be a trivial consequence of 1. (5) Any attempt literally to create intentionality artificially (strong AI) could not succeed just by designing programs but would have to duplicate the causal powers of the human brain. This follows from 2 and 4.

"Could a machine think?" On the argument advanced here only a machine could think, and only very special kinds of machines, namely brains and machines with internal causal powers equivalent to those of brains. And that is why strong AI has little to tell us about thinking, since it is not about machines but about programs, and no program by itself is sufficient for thinking.

Furthermore, see this article for a continuation of Searle's argument which demonstrates that syntax is not intrinsic to physics.

You might also enjoy this exchange in the NYR between Searle and Motzkin (note in particular the technical end note re: the Turing test & Turing machines)

The test is very much part of the behaviorism of the era in which Turing wrote his article; and like all such forms of behaviorism it makes a fundamental confusion between the way we would verify the presence of a mental phenomenon from the third person point of view with the actual first person existence of the phenomenon. As interpreted by Motzkin, the Turing test confuses epistemology with ontology.

Lastly, the Turing test is a high bar to get under and it really says nothing at all about the test subject. It says much more about the tester. The Turing test is by no means a litmus test for consciousness. Passing it has been described as little more than "a parlor trick" - the focus upon which has been derided as "obnoxious and stupid" by no less than Marvin Minsky. For exciting work being done in the field of machine learning, check out Jürgen Schmidhuber.

0

I am intelligent. My brain is arguably not intelligent. My brains cells are definitely not intelligent. The man in the Chinese room is the brain cell. You can’t expect him to understand Chinese. But the Chinese Room? It may be just a brain. Or it may “understand” Chinese.

0

If instructions available with the man in the room are finite, and man outside knows that a finite instruction table exists, then nothing has been established. A Turing machine has a finite program, so infinitely long instruction table is out of scope.

0

If Searle actually is arguing that Chinese Room implies that a program cannot make a computer 'intelligent', then it is a non-sequitur. It just shows that the Turing test is not an adequate test of intelligence as it can be passed via a mechanism with no understanding. I suspect the same would be true of any non-black box test of intelligence where we have to rely on external behaviour and cannot examine the internal workings. The Turing test being an inadequate test of intelligence does not mean that the subject is not intelligent, so the conclusion does not follow from the premise.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .