Does Searle's Chinese Room model computers correctly?

Question

Searle invented a thought experiment, the Chinese Room, which he proposes is an argument against Strong AI (that machines think) but not against Weak AI (that machines simulate thinking), he has a man in a room manipulating chinese symbols via an instruction book written in english.

My question is, Where does this instruction book come from? We're all aware that humans write the code that drives a computer, or writes code that writes more code to drive a computer (ie a compiler) etc.

My clarification (that is, if it is), of Searle's Chinese Room thought experiment, is to have a man (John) who doesn't understand Chinese in a room with two windows. At one window, someone (Mai) submits questions in Chinese, at the other window stands a man (Lao) who does understand Chinese; when a question is submitted, John takes the question walks across the room, and passes it through the window, and gives it to Lao, who reads it, answers it, and then John walks back to the first window, and hands the answer back to the Mai.

To Mai, it appears John understands Chinese (even, if he rather strangely refuses to speak it). She is not aware that he has a secret human collaborator, Lao.

I think this models what actually happens in a computer, much more clearly. But is this an accurate analogy?

Answer 1

What you indicate is that the tome which allows John to simulate communication in Chinese is a rather tremendous computational resource: one which is very close in complexity — assuming that its rules are complex enough to successfully years of conversation in the same way that a Chinese essayist might — to simply conferring with a Chinese person. And I think that you are substantially right. Even though the book itself is a static object, it encodes rules to simulate an interactive process (by its very construction!) and so is not very distinguishable, in terms of its value as a conversational resource, from a Chinese person.

Obviously a book is not a conscious Chinese person. I think we can say with some confidence that a book on its own isn't any sort of conscious entity. But a book such as Searle envisions would be an informatic resource of tremendous power; of absurdly enormous power, in fact. (And someone who was patient enough to successfully use it would be no slouch at computation, or at least the reliable execution of complicated rule-systems, either.) One of the major missteps by Searle is to imagine that we can even conceive realistically of what this book (or its user) would have to look like in order to achieve the purpose he projects for it. It would essentially have to be as complicated as a rather sizeable part of a human brain, another thing that we don't understand on any level other than an essentially syntactic one.

There's a reason why consciousness is such a mystery. We don't have the first clue of how to understand the world in such a way that we can have both predictive power, and also the ability to use that predictive power to describe what consciousness even consists of, aside from experiencing it ourselves and describing the symptoms of apparent consciousness in others. By saying that computers cannot be conscious because we know of no mechanism that would allow it to arise from mechanical evolution is also to summarize the problem we have with understanding our own consciousness.

Scott Aaronson, professor of computer science at MIT, has very similar things to say. The following is an excerpt from a set of notes from a lecture of his on artificial intelligence, from an excellent and much wider-ranging set of lectures.

In the last fifty years, have there been any new insights about the Turing Test itself? In my opinion, no. There has, on the other hand, been a non-insight, which is called Searle's Chinese Room. This is supposed to be an argument that even a computer that did pass the Turing Test wouldn't be intelligent. The way it goes is, let's say you don't speak Chinese. (Debbie [a Chinese-American colleague of Scott's] isn't here today, so I think that's a safe assumption.) You sit in a room, and someone passes you paper slips through a hole in the wall with questions written in Chinese, and you're able to answer the questions (again in Chinese) just by consulting a rule book. In this case, you might be carrying out an intelligent Chinese conversation, yet by assumption, you don't understand a word of Chinese! Therefore symbol-manipulation can't produce understanding.

[...] Like many other thought experiments, the Chinese Room gets its mileage from a deceptive choice of imagery — and more to the point, from ignoring computational complexity. We're invited to imagine someone pushing around slips of paper with zero understanding or insight — much like the doofus freshmen who write (a+b)²=a²+b² on their math tests. But how many slips of paper are we talking about? How big would the rule book have to be, and how quickly would you have to consult it, to carry out an intelligent Chinese conversation in anything resembling real time? If each page of the rule book corresponded to one neuron of (say) Debbie's brain, then probably we'd be talking about a "rule book" at least the size of the Earth, its pages searchable by a swarm of robots traveling at close to the speed of light. When you put it that way, maybe it's not so hard to imagine that this enormous Chinese-speaking entity — this dian nao — that we've brought into being might have something we'd be prepared to call understanding or insight.

Answer 2

I'm very familiar with the argument John makes with his Chinese Room argument, and he's extremely consistent about what he means it to portray: that our concept of what it means to understand language is mistaken when we try to apply the term to any machine which operates only syntactically. It's primarily a refutation of the notion that a Turing Test is sufficient to claim that a conscious understanding is present.

As a system administrator by day, and an aspiring philosophy student by night, I can with full confidence tell you that yes, John Searle is correct when he claims that computers operate purely syntactically. All they do is manipulate symbols, and we still require a human agent to imbue those symbols with meaning. Still, the realization that syntax alone can have such incredible power is the great lesson of our age.

The problem with the example you gave above is that it sidesteps the very point of the reductio that the Chinese Room makes.

In the original example, Mai would submit her answer to a great big box, and receives intelligible responses from this box (whose occupant she's agnostic of) in a reasonably rapid amount of time. From Mai's perspective then, the box has passed the Turing Test- Mai believes she's been understood by a conscious being. On John's side of it, he has a set of drawers which contain all sorts of responses and phrases for different questions, and the guidebook he carries simply directs him to an appropriate drawer based on the Chinese message he receives.

The intuition Searle latches on to here is that John doesn't understand Chinese, so Mai's belief that her words are being understood by a conscious being must be wrong. Trying to replay the thought experiment with Lao playing the role of the conscious, understanding responder thus just circumvents the whole argument without addressing the problem it presents.

There's plenty of deep disgreements to be had at this point: we could defend Mai by claiming that John+Box+guidebook together make a system which understands Chinese, for instance. Searle himself denies this position is coherent, but not everyone buys his opinion. There's also the issue Daniel Dennett raises, that Searle makes a category mistake when using the word "understanding". In Dennett's view, semantics are unnecessary to understanding language, and syntactic operations are all that there is to explain consciousness.

You could also try leveling the charge that Searle's mistake is in thinking that there could even be a set of rules which a living language such as Chinese could be reduced to. This argument however has the consequence that it denies any possibility that a Turing Test could ever succeed. As a result, leveling this charge requires that you already agree with the results of the thought experiment: that rule-following alone cannot account for our normative understanding of what constitutes "understanding".

Answer 3

Although I believe Searle is mistaken, I don't think you have found the problem. You are postulating that the input contains the content not just knowledge of Chinese in distilled form, especially with the walking-across-the-room example. But many machine learning algorithms simply take lots of examples and can then generate appropriate behavior (within limits, of course). So they, once trained, have no content, and not even distillations provided by humans. It is true that relevant properties of, say, Chinese, are implicit in the input data; but it nonetheless does not seem at all like what we intuit thinking to be, nor is it the obvious I-am-walking-messages-across-the-room cheat that you propose.

(Searle proposed the thought experiment at a time when the thought was that logic-based AI was going to be highly successful, which it hasn't been. The thought experiment doesn't quite fit the present practice.)

Answer 4

Isn't that rather what an email is/does? Transporting untranslated information from A to B, exactly the way the information was inputted? But that's exactly the point Searle makes with his analogy - the man who doesn't speak Chinese but actually acts (considering only the outcome of his actions) as if he did, because he has a book of rules that tell him how to. If I'm not missing out anything, then your analogy lacks very important premises/assumptions.

I should add that I know pretty much nothing about how computers work, but I spent some time on that chapter (meaning to say I am only talking about Searle's argument, not about computers in general.)

Answer 5

Where does this instruction book come from?

a hypothesis

Before the official opening of the Great Chinese Room John had a different book and a tall stack of real-life questions-and-answers in Chinese - the Complete Chinese Corpus. This other book instructed John how - using the stack of questions as a training set - to write the book he's presently using to make the Room appear to be talking to Mai . The person who wrote the first book didn't know any Chinese either, the knowledge of Chinese in the Room has emerged from comprehensive analysis of real-life usage of the language.

a variation with Lao

A similar ploy, but involving your hidden Chinese speaker - Lao. Just like before, John has a meta instruction manual on how to make the room understand Chinese but some of the rules in the manual make John interact with Lao. These interactions aren't as simple as passing tablets from and to Mai, no, John has to keep meticulous records of everything coming in, and any responses Lao makes and does some arcane manipulations of the records (in fact - building the Incomplete Chinese Corpus as he goes along). Among those manipulations is predicting what would Lao say, computing Room's confidence that the prediction is accurate and rating the predictions based on the actual reply. Eventually, when Room's confidence is higher than some threshold Lao is cut out of the loop for a transaction or two, and gradually his involvement in the project fades to bare minimum, just to control the quality now and then.

Answer 6

I once met a physicists who held the strong AI position. I was 16, hadn't encountered this position before, and it appeared utterly absurd to me. Even so I'm no longer convinced that this position is utterly absurd, I still don't understand why everybody tries to disprove John Searle. My feeling is that the setup of John Searle's thought experiment can be usefully translated into the context of modern computers:

A modern computer offers a certain amount of memory, more precisely a hierarchy of memory with increasing size but decreasing access speed. It also offers a certain processing power, more precisely a cluster of parallel processing units with increasing number but decreasing interconnection speed. This basic architecture is normally agnostic of the programs and input data that will be used to generate useful results with the resources provided by this computer.

In Searle's Chinese room thought experiment, John is assumed to take the role of this computer, and provide the memory and processing power for the computation. Actually, he is only assumed to provide the fastest memory, while the really huge but slow memory is external to John in the form of paper and pencil and the "magic" book. But even if John would take the role of the "entire" tape of a Turing machine, why should we expect him (or the tape) to understand the problem instance the Turing machine is currently working on? Well, one reason is that for a universal Turing machine, the program itself was also written on the tape, so the tape had access to all the relevant information over time (except the meaning and interpretation of its final output, but I doubt that this is important here).

My question is, Where does this instruction book come from? We're all aware that humans write the code that drives a computer, or writes code that writes more code to drive a computer (ie a compiler) etc.

In the above translation into the context of modern computers, the instruction book is part of the input. This might be an important point, because superficially it looks like the only input comes from Mai, who submits the questions in Chinese.

So where does the input come from? My guess is that the input comes from the current and past environment. However, we can't really look far enough into the past to learn where the "initial seed" came from. And in addition, we have the theory of evolution, which suggests that the "initial seed" might have been less important than it seems. Which brings me to another position that appeared utterly absurd to me when I first encountered it. Somebody suggested to me that this world might have been created by aliens. I found this ridiculous, because it begs the questions who created the aliens in the first place. However, after I watched a clip where Richard Dawkins seriously considered that possibility, I have to admit that it might indeed be a consistent position.

---

Even if it is not directly related to the question, what is my own guess how understanding and meaning can arise during complicated computations? It might be related to the structure of space and time, where communication of information between different points is necessary during a computation (because the amount of information stored near any given point is finite) and takes a certain finite amount of time. The consequence is that compressed messages with more or less clear meaning in the context of the computation are exchanged and understood (and sometimes remembered for later reference) during the course of a computation. (And because the computation which our universe with its space and time structure seems to execute is unlikely to end anytime soon, there is no need to worry about the meaning and interpretation of its final output.)

Answer 7

If we look at animals, humans are intelligent, apes, pigs, dogs, horses are intelligent, chickens quite possibly. Whether flies or ants are intelligent is very debatable, and a bacterium or virus, no way (in my humble opinion).

Searle's argument suggest that a human muscle operated "computer" might be able to simulate an intelligent conversation in Chinese, and we surely all agree that there is no way that Searle's room could be intelligent. We surely also all agree that there is no way that Searle's room could simulate an intelligent conversation in Chinese! It's just not a sufficiently advanced device for any possibility at all.

From that we are supposed to conclude that a highly advanced computer cannot possibly "understand" the chinese text. This is like saying that because the bacterium cannot understand Chinese, therefore a human can't. This is not a reasonable conclusion.

A modern computer could do what Searle's room does about 10 billion times faster, ignoring the fact that the man in Searle's room won't be willing to work more than 1/3rd of the time, which means that an answer that a modern computer could give in a second, would take Searle's room a thousand years. We are right now not capable of creating a rulebook that would make the Chinese conversation possible. By the time we can create it, computers will be thousand times faster.

Would anybody be convinced that Searle's room, which isn't just a room but the interior of a huge skyscraper, operating for one million years, cannot do something equivalent to human "understanding"?