Why doesn't the Chinese room learn Chinese?

Question

I just can't see how John Searle's Chinese room makes sense. The room passes the Turing test. People outside the room think there's a human inside who understands Chinese. But, Searle explains, the room actually contains, in analogical form, all the essential elements of a electronic digital computer programmed to understand (according to Strong AI) written Chinese. But the monolingual English-speaking man in the room (the computer CPU) understands no Chinese. Cards inscribed with Chinese symbols fall into the room through a slot in the door. These are sensible Chinese questions. The rule book (the program) deals only with their shapes, not their meanings. It instructs the man to find certain Chinese characters among the spares in the room then to push them out through the slot. Unknown to the man, these are sensible Chinese answers. Neither the man nor the room understands the meanings of the shapes, since all they have is the shapes. From here Searle goes on to argue that computers will never understand language or the world.

What seems to me like a fundamental mistake is that Searle bases his argument on comparing a computer receiving Chinese symbols with a human receiving Chinese symbols. Then from the fact that the computer doesn't understand the meanings of the symbols, Searle argues that computers could never understand anything.

Well, humans can't understand the meanings of the symbols either. Humans first have to learn Chinese. Why doesn't the room try to learn Chinese? Without this, Searle's argument is pointless. Leaning Chinese entails developing memory structure. There's no structure in the Chinese room because there is nothing in the room to build it out of. The room's ontology needs structural elements added to it so that it then contains atoms of structure as well as symbols (the content of structure). Then the program can instruct the man to build memory structure. Digital computers can easily build memory structure and often do. Now with structural elements, the Chinese room can try to learn Chinese. And by the way, the CRA is unsound because Searle's premiss "... a digital computer is a syntactical machine. It manipulates symbols and does noting else" (John Searle, 2014, "What Your Computer Can't Know", in The New York Review of Books, October 9, 2014) is false. Also, it can be well argued that some structural elements are semantic.

It would be really great to get some comments and criticisms of my above reply to the Chinese room argument.

Answer 1

Even if the man inside the Chinese room memorised every single translation instance (theoretically every possible combination which is impossible given our limited memory, but it's a thought experiment, so this constraint doesn't matter), would he understand Chinese, since he doesn't understand the meaning of any of the cards he has been presented with?

Searle does not think so:

in the literal sense the programmed computer understands what the car and the adding machine understand, namely, exactly nothing. The computer's understanding is not just (like my understanding of German) partial or incomplete; it is zero. . . . In the linguistic jargon, they have only a syntax but no semantics.

It is hard to say, therefore, that the man in the Chinese room does understand Chinese, as he is just following a set of rules or algorithms, much the same as an AI or super-intelligent computer that appears conscious would be doing.

What's central to this is the idea of human consciousness and whether it can be, firstly, defined and, secondly, simulated on a computer. If we don't know what human consciousnesses is, we have no chance of simulating it on a computer. Perhaps human consciousness is nothing more than physical interactions of the laws of nature: chemistry, atomic forces, molecular biology, quantum fields, cellular and neuronal connections, and so on. Or maybe human consciousnesses is more than this. Subjective experience, therefore, can't be simulated by a machine, as it is not reducible to physical interactions and there is something more to consciousness that we can't explain sufficiently by scientific explanations alone. (See: qualia and the philosophical zombie argument). It could be due to the fact that our brains evolved to be the way they are now over billions of years, starting from the first single-celled life, and the emulation of such a process is much too complex — although we couldn't completely rule out this possibility.

If conciousness emerged and we can create similar conditions which led to this emergence, by building a machine or computer program that can emulate this emergence, could we then create and artificial consciousness? Very hard to know. If the atoms of an intelligent and conscious thing are silicon-based rather than carbon-based, and are arranged in a precisely identical arrangement, are we talking about the same phenomena? Consciousness in its varied forms?

Searle's comments about this:

"Could a machine think?" The answer is, obviously, yes. 'We are precisely such machines. "Yes, but could an artificial, a man-made machine, think?" Assuming it is possible to produce artificially a machine with a nervous system, neurons, with axons and dendrites, and all the rest of it, sufficiently like ours, again the answer to the question seems to be obviously, yes. If you can exactly duplicate the causes, you could duplicate the effects. And indeed it might be possible to produce consciousness, intentionality, and all the rest of it using some other sorts of chemical principles than those that human beings use. It is, as I said, an empirical question.

If you are of the opinion that it doesn't matter whether the computer 'really' knows it is conscious and can pass a sophisticated Turing test, then appearance is all you need to be convinced that an AI can really be conscious. Strong AI, to Searle, is human consciousness only and can't be replicated by a computer program. Very hard to define what is meant by consciousness, so his entire argument is underpinned by knowing at-bottom intuitively what we mean by "subjective experience of reality". Only you really know what it is like to be you, and no computer will ever be able to really 'know' how it exists in the same way. Without seeing the limits in the future of what our technology can achieve; however, we will not be able to unconditionally rule out a self-conscious super-intelligence or strong-AI.

Reference:

Cole, David, "The Chinese Room Argument", The Stanford Encyclopedia of Philosophy (Winter 2015 Edition), Edward N. Zalta (ed.), URL = https://plato.stanford.edu/archives/win2015/entries/chinese-room/.

Kirk, Robert, "Zombies", The Stanford Encyclopedia of Philosophy (Summer 2015 Edition), Edward N. Zalta (ed.), URL = https://plato.stanford.edu/archives/sum2015/entries/zombies/.

John R. Searle, "Minds, Brains and Programs' inThe Behavioral and Brain Sciences, vol. 3. Copyright @ 1980 Cambridge University Press.

Answer 2

There seem to be several things not understood in asking this question.

Searle gave an intuitive argument. He did not and still does not understand the details so there was a limit to what he could explain.

It doesn't actually matter if you used books or you used a filing system or a database or you used a state of the art AI, the results would be the same. So, whatever Searle said would be right or wrong regardless of how you built it. What would happen if you built such a room?

The room would be capable of answering any stock question correctly. This would be limited only by the size and access time of your information base. BTW, this is pretty much what Watson did when it played Jeopardy. It understood none of the questions or answers but only looked for associations. So, Watson is probably the best example of a modern, Chinese Room. If you asked a stock question like, "What US president was rumored to have chopped down a cherry tree?," Watson would find that US, president, and cherry tree were associated with George Washington and would answer correctly. But this isn't understanding, not even close.

The human mind is strong in terms of its ability to generalize which means applying known patterns and logic to conditions outside of previous experience. You could in fact trap Watson very easily. Let's take

Suppose you were in a room with a locked door. There is a button on the wall that will unlock the door. However, the button is too high to reach even if you jump. There are a sturdy, wooden table, a broom, and a rubber ball in the room. How might you try to escape?

This question is trivial for a human but Watson would be unable to give any kind of answer unless that question and an associated answer was in its information system. I've even heard of experiments that toddlers could solve but Watson would be incapable of. And, this will still be the case no matter how large you make the information base. So, Searle was mostly correct.

Answer 3

Consider the following about the Chinese Room Argument.

First, strong AI is a view that programs running on Turing machines (computers) not only produce correct results but also generate consciousness when run.

Second, assume there exists a program that passes the Turing test for Chinese when run on any Turing machine no matter how advanced or primitive that computer may be.

Third, let a human who does not understand Chinese simulate a Turing machine by following that successful program while being isolated from outside influence in a “room”. If strong AI is correct this should be a way for the human not only to give a correct answer but also to understand Chinese. This program would be a way for someone to learn Chinese.

Fourth, Searle claims the human will not understand or learn Chinese, but the human will be able to pass the Turing test using the program since the program is assumed to be able to do so.

Fifth, look at the program as the “mind”. Look at the computer as the “body”. The body runs the mind and this supposedly generates not only results, but also understanding of Chinese. It is this mind-body dualism that is the problem for a physicalist.

Given the above, I will try to answer this question.

“Humans first have to learn Chinese. Why doesn't the room try to learn Chinese?”

The room is only a way to isolate the human from outside influence. It is not the computer. There is nothing for the room to learn. The human needs to learn or understand Chinese by running the program. Running the program is what counts not which computer is used to run the program. The program can be moved around to different computers or different humans and the results should be the same. The question is does running the program anywhere generate not only a correct result, but also understanding of Chinese? If it does, then this would be an alternate way for someone to learn a new language.

Reference: John Searle, "Minds, Brains and Programs"

Answer 4

I believe that you've hit on what is typically referred to as the "systems reply", which is, in short, that the room system does understand Chinese. This seems plausibly true in terms of a functional definition of understanding, but getting to the point where one can conceive that the room system has a subjective experience of understanding is a much bigger conceptual chasm to jump. (though as far as I can tell, it cannot be ruled out, especially since I don't know to a high degree of certainty which of you dear readers have subjective experiences and which of you are zombies).

Answer 5

I've always thought that Searle's repudiation of Turing through his "Chinese room" analogy was flawed.

The claim that the room "doesn't understand Chinese", substantively on the basis that the man inside it doesn't understand Chinese, seems to me deeply flawed and quite unconvincing.

Firstly, it conflates a system with one of its minor components

• A CPU doesn't learn stuff (or at least, doesn't remember very much or for very long), but a computer with a CPU, sensors, outputs, and storage certainly can.
• One of my brain cells doesn't understand English; that doesn't mean my brain as a whole doesn't understand it.

Secondly, the room couldn't actually chat with a living human, because it operates at vastly too slow a time-scale.

And perhaps that's the real reason that the man inside the room never learns read & write Chinese: it's not one man, it's a role handed down through the generations. (And if somehow the devices and structure of the room generate responses fast enough to allow interaction with humans, that just means that the man in the room becomes even more marginal and irrelevant to the whole system's ability to understand stuff.)

It could be argued that since the room relies entirely on pre-written instructions and cannot learn new information or share experiences with its correspondents, it could never converse on a subject it hadn't been explicitly programmed for. In short, it would be an expert system that couldn't pass a Turing test.

Answer 6

This question comes down to qualia, is there a difference between a state as observed from outside, and subjectively from inside. The Chinese Room helps us thinking about that, but raises more questions than it answers.

We are moving into a new era with this question however, as we approach the point of being able to build such a room.

From the ground up, we have natural language processing like https://en.m.wikipedia.org/wiki/Watson_(computer)

And from simulating the human brain down, we have https://en.m.wikipedia.org/wiki/Human_Brain_Project

A human, or a room that has 'learnt Chinese' rather than just been given limited responses to preprogrammed situations, would not only give answers to well formulated questions, but be able to interpret incomplete or unclear questions, and reinterpret definitions on the go (Watson struggled with short questions, for instance, lacking context). The setup tries to say these wouldn't, but natural human language is full of them. Yes the room needs structures, parallel to human mental structures or learning, and will only be as capable as they are sophisticated. We have to look at the threshold between rote learning and understanding a language, like between brute forcing chess moves and being able to understand the game. What is understanding?

We do complex things, and take them for granted, which muddies the water of this, similar to how we found computer visual processing to be a great deal more difficult than expected, because so much is happening we aren't aware of. The best framework I know for understanding what may be making the difference, from the ground up, is the https://en.m.wikipedia.org/wiki/Strange_loop picture.

Answer 7

Let me state another question to perhaps change the view on this:

Can the man learn Chinese this way at all? Memorizing rules and already given answers and reproducing them is not the same as if he had understood. This goes a little bit in the direction of answering incomplete questions and if there is are qualia to the man inside the room (i.e. does the man have any representation of what he is doing in his mind other than the book of / memorized rules and historical answers, already produced). However, there are a few aspects that I would like to emphasize with this slight modification to the original question:

First, even if the man / system learns the rules by rote and could recall all the combinations of symbols (answers) he already produced, I would argue, he still does not understand. BUT is it possible that he develops a feeling (internal set of rules) for producing correct answers, aside from the rules in the book?

Second, can he become better? Can these new internalized / learned / self-developed rules enable the man to give semantically correct answers, that are not in the expected form. For example: can he give a different answer to a question he already answered earlier, still without understanding a single word Chinese? Different in the sense that the people outside would accept the answer as correct, but it's not the same wording he used before. For example, the question could be "Is snow white?". The correct and expected answer would be "Yes, it is.". Note that this answer is not only semantically but also grammatically correct. The new answer might be a simple "Yes", which would be accepted as correct I assume, but is not absolutely correct.

Third, does this set of new rules inside the man state something as a Super-Language to Chinese, or is it something completely different and new? Can he form new Ideas, Sentences or solve any inherent problem hiding inside the posed question, using his set of rules, so that the answer technically still is Chinese language and would be understood by the people outside, but is so strange to them, that they would not have come up with it theirselves? Or has the man inside the room developed a completely new language, which only he understands (since there is no one else with the same internalized rules)?

Fourth, who decides what is a correct answer? A Chinese-Teacher would say, "Shì (yes)" is not a (grammatically) correct answer, while average Kim Lee would accept it as being correct. This brings this discussion back to the whole field of Philosophy of Mind with all its own thoughts and discussions. So I'd like to stop here.

Finally I'd like to make a general annotation: The Turing-Test and the setup of this thought-experimemt breathes the spirit of Behaviourism, where only the visible (outward) result counts. As far as computers (symbol manipulating systems) are concerned it actually might seem like the only way to get to an answer to the question, do they understand what they are doing. But is it? Perhaps the question is misleading. Perhaps they do understand but in a different way? And how can we tell? Do you understand what you are doing? Completely? All the time? Does a man understand society, who convicts a person for stealing food as an act against society? What if the same man would stand right by a homeless person and does not give him food, although he could, since his doing is not unlawful and not helping is not even expected. Does he understand what the term society really means?

I'm looking forward to your replies and hope my answer helps you to get new insights.