Take the 2-minute tour ×
Philosophy Stack Exchange is a question and answer site for those interested in logical reasoning. It's 100% free, no registration required.

Searle invented a thought experiment, the Chinese Room, which he proposes is an argument against Strong AI (that machines think) but not against Weak AI (that machines simulate thinking), he has a man in a room manipulating chinese symbols via an instruction book written in english.

My question is, Where does this instruction book come from? We're all aware that humans write the code that drives a computer, or writes code that writes more code to drive a computer (ie a compiler) etc.

My clarification (that is, if it is), of Searle's Chinese Room thought experiment, is to have a man (John) who doesn't understand Chinese in a room with two windows. At one window, someone (Mai) submits questions in Chinese, at the other window stands a man (Lao) who does understand Chinese; when a question is submitted, John takes the question walks across the room, and passes it through the window, and gives it to Lao, who reads it, answers it, and then John walks back to the first window, and hands the answer back to the Mai.

To Mai, it appears John understands Chinese (even, if he rather strangely refuses to speak it). She is not aware that he has a secret human collaborator, Lao.

I think this models what actually happens in a computer, much more clearly. But am I right?

share|improve this question
    
I was going to post an answer, but I think this fits better as a comment. Everyone likes to attack Searle's Chinese Room because it's a particularly easy thought experiment to attack. Even Searle has conceded that it's not that great. However, his position of Strong/Weak AI has some very good arguments (outlined in his book - I'm not sure how many repliers actually read it). The Chinese Room is only a peripheral thought experiment, not a central tenet. –  David Titarenco Dec 11 '12 at 18:46
    
@Titarenco:What actually is his position on Strong/Weak AI? –  Mozibur Ullah Dec 11 '12 at 19:20

7 Answers 7

up vote 5 down vote accepted

What you indicate is that the tome which allows John to simulate communication in Chinese is a rather tremendous computational resource: one which is very close in complexity — assuming that its rules are complex enough to successfully years of conversation in the same way that a Chinese essayist might — to simply conferring with a Chinese person. And I think that you are substantially right. Even though the book itself is a static object, it encodes rules to simulate an interactive process (by its very construction!) and so is not very distinguishable, in terms of its value as a conversational resource, from a Chinese person.

Obviously a book is not a conscious Chinese person. I think we can say with some confidence that a book on its own isn't any sort of conscious entity. But it is an informatic resource of tremendous power; of absurdly enormous power, in fact. (And someone who was patient enough to successfully use it would be no slouch at computation, or at least the reliable execution of complicated rule-systems, either.) One of the major missteps by Searle is to imagine that we can even conceive realistically of what this book (or its user) would have to look like in order to achieve the purpose he projects for it. It would essentially have to be as complicated as a rather sizeable part of a human brain, another thing that we don't understand on any level other than an essentially syntactic one.

There's a reason why consciousness is such a mystery. We don't have the first clue of how to understand the world in such a way that we can have both predictive power, and also the ability to use that predictive power to describe what consciousness even consists of, aside from experiencing it ourselves and describing the symptoms of apparent consciousness in others. By saying that computers cannot be conscious because we know of no mechanism that would allow it to arise from mechanical evolution is also to summarize the problem we have with understanding our own consciousness.

Scott Aaronson, professor of computer science at MIT, has very similar things to say. The following is an excerpt from a set of notes from a lecture of his on artificial intelligence, from an excellent and much wider-ranging set of lectures.

In the last fifty years, have there been any new insights about the Turing Test itself? In my opinion, no. There has, on the other hand, been a non-insight, which is called Searle's Chinese Room. This is supposed to be an argument that even a computer that did pass the Turing Test wouldn't be intelligent. The way it goes is, let's say you don't speak Chinese. (Debbie [a Chinese-American colleague of Scott's] isn't here today, so I think that's a safe assumption.) You sit in a room, and someone passes you paper slips through a hole in the wall with questions written in Chinese, and you're able to answer the questions (again in Chinese) just by consulting a rule book. In this case, you might be carrying out an intelligent Chinese conversation, yet by assumption, you don't understand a word of Chinese! Therefore symbol-manipulation can't produce understanding.

[...] Like many other thought experiments, the Chinese Room gets its mileage from a deceptive choice of imagery — and more to the point, from ignoring computational complexity. We're invited to imagine someone pushing around slips of paper with zero understanding or insight — much like the doofus freshmen who write (a+b)2=a2+b2 on their math tests. But how many slips of paper are we talking about? How big would the rule book have to be, and how quickly would you have to consult it, to carry out an intelligent Chinese conversation in anything resembling real time? If each page of the rule book corresponded to one neuron of (say) Debbie's brain, then probably we'd be talking about a "rule book" at least the size of the Earth, its pages searchable by a swarm of robots traveling at close to the speed of light. When you put it that way, maybe it's not so hard to imagine that this enormous Chinese-speaking entity — this dian nao — that we've brought into being might have something we'd be prepared to call understanding or insight.

share|improve this answer
    
There is a reason why literary theorists have excavated so much out of what can seem in comparison so little. Compare the corpus of criticism on Shakespeares plays compared with corpus of plays themselves, and why they talk of the dialogue between text & reader. Nice quote from Scott; I entirely agree; though I think he's being a bit hard on Searle: A thought experiment can take a life of its own in the hands of others. –  Mozibur Ullah Dec 11 '12 at 16:11
    
I've seen this challenge before, and it's so bad that I can't even call it wrong. The claim you quote by Aaronson is that the model assayed can't be right because doesn't exhibit the same complexity we'd expect it to have?! That begs the question, doesn't bother to address the challenge to syntactic understanding, and just assumes semantics can't exist unless we have some magical limit of complex relations to hide it under. –  Ryder Dec 11 '12 at 17:37
    
@RyderDain: The whole point is that if you are a metaphysical naturalist, then any claim that computers are limited to syntactical manipulation applies equally well to ourselves. (If some of the people in the discussion are metaphysical naturalists and others aren't, then one may as well go home, because they'll only argue past each other; as I'm a naturalist I therefore argue from that premise.) On that premise, the only limitation the book-interpreter system has is complexity; caricaturing it merely as a bureaucrat with a simple rulebook basically masks the complexity of the system. –  Niel de Beaudrap Dec 11 '12 at 17:40
    
@NeilDeBeaudrap: Let's really go ad absurdum here: I could conceivably level the exact same charge against proponents of heliocentrism. It can't possibly account for the complex orbits we see from Earth, and it wouldn't resemble what we think we already know unless you tried to include galactic precession in the model as well! There's plenty of good arguments to be made why Searle's experiment fails, but throwing a lack of a presumably sufficient complexity at it isn't one of them. –  Ryder Dec 11 '12 at 17:46
2  
@RyderDain: I don't deny that semantics exist; I but my philosophy of mathematics is that the significance (to use a less laden term) of systems of notation arise from the functional relationships in them, and that semantics is nothing more than the process of making associations between the symbols and the functional signifance in a predictive manner. This, I think, is roughly what human brains do for symbols, sounds, facial expressions, etc., where some may be inbuilt. I see no reason why a program could not embody semantical associations in a manner similar to our brains. –  Niel de Beaudrap Dec 11 '12 at 18:08

I'm very familiar with the argument John makes with his Chinese Room argument, and he's extremely consistent about what he means it to portray: that our concept of what it means to understand language is mistaken when we try to apply the term to any machine which operates only syntactically. It's primarily a refutation of the notion that a Turing Test is sufficient to claim that a conscious understanding is present.

As a system administrator by day, and an aspiring philosophy student by night, I can with full confidence tell you that yes, John Searle is correct when he claims that computers operate purely syntactically. All they do is manipulate symbols, and we still require a human agent to imbue those symbols with meaning. Still, the realization that syntax alone can have such incredible power is the great lesson of our age.

The problem with the example you gave above is that it sidesteps the very point of the reductio that the Chinese Room makes.

In the original example, Mai would submit her answer to a great big box, and receives intelligible responses from this box (whose occupant she's agnostic of) in a reasonably rapid amount of time. From Mai's perspective then, the box has passed the Turing Test- Mai believes she's been understood by a conscious being. On John's side of it, he has a set of drawers which contain all sorts of responses and phrases for different questions, and the guidebook he carries simply directs him to an appropriate drawer based on the Chinese message he receives.

The intuition Searle latches on to here is that John doesn't understand Chinese, so Mai's belief that her words are being understood by a conscious being must be wrong. Trying to replay the thought experiment with Lao playing the role of the conscious, understanding responder thus just circumvents the whole argument without addressing the problem it presents.

There's plenty of deep disgreements to be had at this point: we could defend Mai by claiming that John+Box+guidebook together make a system which understands Chinese, for instance. Searle himself denies this position is coherent, but not everyone buys his opinion. There's also the issue Daniel Dennett raises, that Searle makes a category mistake when using the word "understanding". In Dennett's view, semantics are unnecessary to understanding language, and syntactic operations are all that there is to explain consciousness.

You could also try leveling the charge that Searle's mistake is in thinking that there could even be a set of rules which a living language such as Chinese could be reduced to. This argument however has the consequence that it denies any possibility that a Turing Test could ever succeed. As a result, leveling this charge requires that you already agree with the results of the thought experiment: that rule-following alone cannot account for our normative understanding of what constitutes "understanding".

share|improve this answer
3  
Definitely the clearest answer! I would add that I've seen no coherent response to a Dennett-style criticism, namely that intuition just fools us about what "understanding" is. That is, when we manipulate symbols in various complicated ways (without being aware of the steps, but maybe with some quality-prediction metric that lets us tell we're probably doing a good job), we call that understanding. If a machine does the same, we say it's a mechanistic process with no understanding. (In this view, we are for now a lot better than computers--but it's merely a quantitative difference.) –  Rex Kerr Dec 5 '12 at 0:15
    
@RexKerr I have to disagree: the standard response, that Dennett's criticism is valid only if understanding has no de facto relation to semantics, is perfectly coherent. It's only incoherent if you already deny, as Dennett and others do, that qualia-laden experiences are a valid account for any mental state we identify when we say "x understands p". I'm not going to set foot in that one. If you'd rather have it out in chat, let's take it outside. –  Ryder Dec 5 '12 at 10:47
    
Indeed, chat would be the place. I would just note in advance that qualia-laden experiences have been almost hopelessly misleading in understanding the basis of perception, and there is no reason to expect that they are any better at giving us insight into "understanding" and "thinking". –  Rex Kerr Dec 5 '12 at 11:54
    
@Dain: I actually accept Searles contention: That computers cannot learn/understand a language in the significant meaning of the term; I wanted to unwrap Searles reductio to show what was going on - communication between two concious minds - because to me (and as a sometime c++ programmer, and an assembly programmer in my mis-spent youth - they hold no magic for me) thats exactly whats going on. But you're also right that it only causes confusion; Searles reductio is a better illustration, as it looks physically closer to what a computer does: it has an instruction booklet. –  Mozibur Ullah Dec 7 '12 at 9:28
1  
@MoziburUllah: you are begging the question: you are assuming that no computer system can have a conscious mind, and you cannot reasonably claim that a computer cannot have a conscious mind without positing a model for how we can have a conscious mind. –  Niel de Beaudrap Dec 11 '12 at 16:03

Isn't that rather what an email is/does? Transporting untranslated information from A to B, exactly the way the information was inputted? But that's exactly the point Searle makes with his analogy - the man who doesn't speak Chinese but actually acts (considering only the outcome of his actions) as if he did, because he has a book of rules that tell him how to. If I'm not missing out anything, then your analogy lacks very important premises/assumptions.

I should add that I know pretty much nothing about how computers work, but I spent some time on that chapter (meaning to say I am only talking about Searle's argument, not about computers in general.)

share|improve this answer
    
I'm interested in how he got that book of instructions. Someone wrote it who knew chinese. –  Mozibur Ullah Dec 4 '12 at 19:31
    
I haven't read Searle, I only know of his argument by hearsay (wikipedia); I'm not being critical of Searles conclusion, but being more expansive in the hope of making the conclusion more obvious; but in fact perhaps it makes it more confusing... –  Mozibur Ullah Dec 4 '12 at 19:48
2  
@MoziburUllah Yes, someone did, but that only allows us to say that there is someone who does speak Chinese, it doesn't tell us anything about the computer's ability to actually learn Chinese. The people who wrote that book are the ones that wrote a program executing commands. –  iphigenie Dec 4 '12 at 20:47
    
@Iphegenie: all I'm doing really is replacing that instruction booklet with the person who wrote it. Personally, I agree with Searles conclusion. But simulated learning/understanding can be good enough. –  Mozibur Ullah Dec 4 '12 at 21:00
    
@MoziburUllah You're oversimplifying the focus on imitating the knowledge by acting out commands. Yes, walking from A to B, that much a computer can do. Learning how to speak Chinese, he can't, because learning a language is more than acting according to a script, and that's all Searle's saying. –  iphigenie Dec 4 '12 at 21:13

Although I believe Searle is mistaken, I don't think you have found the problem. You are postulating that the input contains the content not just knowledge of Chinese in distilled form, especially with the walking-across-the-room example. But many machine learning algorithms simply take lots of examples and can then generate appropriate behavior (within limits, of course). So they, once trained, have no content, and not even distillations provided by humans. It is true that relevant properties of, say, Chinese, are implicit in the input data; but it nonetheless does not seem at all like what we intuit thinking to be, nor is it the obvious I-am-walking-messages-across-the-room cheat that you propose.

(Searle proposed the thought experiment at a time when the thought was that logic-based AI was going to be highly successful, which it hasn't been. The thought experiment doesn't quite fit the present practice.)

share|improve this answer
    
I think you've misunderstood me. I accept Searles contention. I'm just unwrapping his reductio. I'm not looking for his mistake, as you put it. I'm postulating that understanding occurs in a conciousness, and I wanted to show it in a more direct manner. And I don't think computers, no matter how sophisticated they become at simulating it, will be concious in the significant term of the word. Nature is more subtle than that. You're saying present practise has gone beyond the turing model, which Searles experiment illustrates? –  Mozibur Ullah Dec 7 '12 at 9:42
    
@MoziburUllah - There are two parts to Searle's contention: (1) computers merely compute, and (2) we are conscious, which is different. You have altered (1) by making it so the computer does not even compute, but merely acts as a conduit for fully-formed information between conscious entities. But what is in question these days is that computing and consciousness are different in kind. To be instructive, you need to push computing as close to consciousness as you possibly can and show that it still is inadequate. Observing one bird will leave you doubting that a flock is made of birds. –  Rex Kerr Dec 7 '12 at 15:42
    
@Kerr: but I'm not so sure, it's possible to hide things by pushing them closer, along one dimension; but they're still just as far apart in another. –  Mozibur Ullah Dec 11 '12 at 19:32
    
@MoziburUllah - Indeed, but then you want to argue exclusively about that other dimension. Searle's argument is about like this: I can watch YouTube videos of cats on my computer. I have a LCD watch from the 1990s, on which I cannot watch YouTube videos. There is a difference in kind between my computer and my watch; to see this, consider reducing the watch to a single LCD element driven by one transistor, which a computer waves back and forth and turns on and off to build up a picture. Clearly, the computer is doing all the work, proving that computers are different in kind. –  Rex Kerr Dec 11 '12 at 20:18

As I wrote in chapter 20 of Parsing the Turing Test:

The idiosyncrasies of this arrangement—in particular, the existence of three separate batches of Chinese writing—are the result of Searle's attempt to duplicate the functionality of SAM, a “story understanding” program written by Roger Schank and his colleagues at Yale in the 1970's.

Personally, I think it's much more fun to think of the person inside the Chinese room executing the program on a Turing Machine using toilet paper and pebbles, as suggested by Joseph Weizenbaum in his Computer Power and Human Reason.

share|improve this answer

Where does this instruction book come from?

a hypothesis

Before the official opening of the Great Chinese Room John had a different book and a tall stack of real-life questions-and-answers in Chinese - the Complete Chinese Corpus. This other book instructed John how - using the stack of questions as a training set - to write the book he's presently using to make the Room appear to be talking to Mai . The person who wrote the first book didn't know any Chinese either, the knowledge of Chinese in the Room has emerged from comprehensive analysis of real-life usage of the language.

a variation with Lao

A similar ploy, but involving your hidden Chinese speaker - Lao. Just like before, John has a meta instruction manual on how to make the room understand Chinese but some of the rules in the manual make John interact with Lao. These interactions aren't as simple as passing tablets from and to Mai, no, John has to keep meticulous records of everything coming in, and any responses Lao makes and does some arcane manipulations of the records (in fact - building the Incomplete Chinese Corpus as he goes along). Among those manipulations is predicting what would Lao say, computing Room's confidence that the prediction is accurate and rating the predictions based on the actual reply. Eventually, when Room's confidence is higher than some threshold Lao is cut out of the loop for a transaction or two, and gradually his involvement in the project fades to bare minimum, just to control the quality now and then.

share|improve this answer
    
I have the impression that this answer brings you back to Plato, who suggests that the ideal "real thing" casts his "shadow" into the "world" (i.e. into the instruction book). But then the "real thing" is somehow also responsible for the meaning and understanding that arises by the use of the instruction book. Funnily, for all examples in the initial paper on PAC learning, there was (implicitly) such an ideal "real thing". However, I think the author did this so that he could prove some nice theorems, not because it would have been necessary. –  Thomas Klimpel Dec 10 '12 at 23:04
    
The only part of the scheme that is just remotely "ideal" is "the knowledge of Chinese". The rest is real (not in platonic, but in physical, material sense): the room, John sitting in the room, a book of instructions that John wrote, a book of instructions that John followed to write a book of instructions, a stack of examples etc. The "knowledge of Chinese" seems less tangible, but ultimately it's just that book of instructions + the state of the cupboards with Chinese tablets. –  artm Dec 11 '12 at 9:37
    
@artm: 'The person who wrote the first book didn't know any Chinese' is still giving John an instruction booklet, but this time for language learning: He's understood & examined enough human languages to deduce a (supposed) general theory of language acquisition & manipulation. It only moves Searles argument a step further down. I don't see a significant difference. Are you arguing for Searles position or against (Understanding is not syntax/mechanism)? I'm not talking about his illustration. –  Mozibur Ullah Dec 11 '12 at 11:14
    
I'm not trying to argue either for or against Searle's position, just trying to imagine where the book could have come from based on how similar software problems are often solved nowadays (re: Rex Kerr's answer). The original formulation is based on CS state of the day, my variations are more a kin to what they teach in machine learning today. Wether or not the room "understands Chinese" in one sense or another is still open to debate, and my Chinese Room should be subject to all the traditional criticism of the classic variant. –  artm Dec 11 '12 at 15:43

I once met a physicists who held the strong AI position. I was 16, hadn't encountered this position before, and it appeared utterly absurd to me. Even so I'm no longer convinced that this position is utterly absurd, I still don't understand why everybody tries to disprove John Searle. My feeling is that the setup of John Searle's thought experiment can be usefully translated into the context of modern computers:

A modern computer offers a certain amount of memory, more precisely a hierarchy of memory with increasing size but decreasing access speed. It also offers a certain processing power, more precisely a cluster of parallel processing units with increasing number but decreasing interconnection speed. This basic architecture is normally agnostic of the programs and input data that will be used to generate useful results with the resources provided by this computer.

In Searle's Chinese room thought experiment, John is assumed to take the role of this computer, and provide the memory and processing power for the computation. Actually, he is only assumed to provide the fastest memory, while the really huge but slow memory is external to John in the form of paper and pencil and the "magic" book. But even if John would take the role of the "entire" tape of a Turing machine, why should we expect him (or the tape) to understand the problem instance the Turing machine is currently working on? Well, one reason is that for a universal Turing machine, the program itself was also written on the tape, so the tape had access to all the relevant information over time (except the meaning and interpretation of its final output, but I doubt that this is important here).

My question is, Where does this instruction book come from? We're all aware that humans write the code that drives a computer, or writes code that writes more code to drive a computer (ie a compiler) etc.

In the above translation into the context of modern computers, the instruction book is part of the input. This might be an important point, because superficially it looks like the only input comes from Mai, who submits the questions in Chinese.

So where does the input come from? My guess is that the input comes from the current and past environment. However, we can't really look far enough into the past to learn where the "initial seed" came from. And in addition, we have the theory of evolution, which suggests that the "initial seed" might have been less important than it seems. Which brings me to another position that appeared utterly absurd to me when I first encountered it. Somebody suggested to me that this world might have been created by aliens. I found this ridiculous, because it begs the questions who created the aliens in the first place. However, after I watched a clip where Richard Dawkins seriously considered that possibility, I have to admit that it might indeed be a consistent position.


Even if it is not directly related to the question, what is my own guess how understanding and meaning can arise during complicated computations? It might be related to the structure of space and time, where communication of information between different points is necessary during a computation (because the amount of information stored near any given point is finite) and takes a certain finite amount of time. The consequence is that compressed messages with more or less clear meaning in the context of the computation are exchanged and understood (and sometimes remembered for later reference) during the course of a computation. (And because the computation which our universe with its space and time structure seems to execute is unlikely to end anytime soon, there is no need to worry about the meaning and interpretation of its final output.)

share|improve this answer
    
I'm not trying to disprove Searle. Perhaps because everyone is doing this, it's taken for granted that this is what I'm attempting to do! I'm glad you've noted that 'where the instruction booklet comes from' is an important question. In the context of computers we can always trace this; and in that context we can say, in an important sense it is part of the input. In the context of humans, I'm not so sure. Of course evolution is part of the answer. This brings us to a conundrum about the universe: what 'created' the laws of the universe that allowed evolution to occur. –  Mozibur Ullah Dec 8 '12 at 17:51
    
I've read about Lee smolins speculations on this, but in my mind, it pushes the problem only one stage further back. To say that they are given, or just as they are, to me, is no different when the first peoples looking at the course of the sun & the stars across the sky, said that it is so. And no deeper principles applied, or if they did we couldn't know them. It was over four millenia later that principle was found (dating it from Sumerian astronomy) –  Mozibur Ullah Dec 8 '12 at 17:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.