8

The Chinese Room argument attempts to prove that a computer, no matter how powerful, cannot achieve consciousness.

Brief summary:

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

How is this any different than what goes on inside our brains?

Certain impulses are received from sensory organs and processed by neurons. This is a completely deterministic process and to these neurons, individually, the input/output has absolutely no meaning. Individually, they possess no consciousness. Sure, it happens 10^n times simultaneously and maybe there is some recursion involved, but the concept is the same - the origin of the input and the destination of the output are irrelevant.

The only difference I can think of is that, in the brain, the instructions/look-up tables/whatever can be modified by this process. The experiment makes no mention of this, because there is no need for it - language syntax remains relatively constant over a short period of time. But as long as these modifications are carried out according to a set of rules, it would make no difference.

Am I missing some crucial part of Searle's argument?

(Inspired by this question)

  • 5
    If you have no personal experience of your own experience ... you get the official philosophy.stackexchange.com Zombie badge. It astonishes me that people pretend to be unaware of themselves. – user4894 Jun 1 '16 at 17:45
  • You might like this quote from Scott Aaronson: "Like many other thought experiments, the Chinese Room gets its mileage from a deceptive choice of imagery -- and more to the point, from ignoring computational complexity. We're invited to imagine someone pushing around slips of paper with zero understanding or insight. But how many slips of paper are we talking about? How big would the rule book have to be, and how quickly would you have to consult it, to carry out an intelligent Chinese conversation in anything resembling real time?... – Tim kinsella Jun 1 '16 at 18:13
  • 1
    Hahaha thank you sir. I think thats going on the fridge – J.Doe Jun 1 '16 at 19:44
5

I would like to suggest that your puzzlement arises from confusing intelligence and consciousness. Neither concept is well defined but nonetheless they are distinct. Searle would say that a Chinese room cannot be conscious, not that it cannot appear to be intelligent. In fact the original argument revolves around the concept of understanding which is another blurred concept with no clear definition. Searle is a philosopher who believes that the mind cannot be expressed in terms of computations, or that in other words, a computer may never have a mind, regardless of its architecture and particular computation. He does not rule out that machines in general may have a mind, only that mechanisms (a subset of machines) may never amount to a mind. You can still disagree with him (as most people do) but to do that it is important first to understand him (pun intended).

  • 1
    The person in the room is following an algorithm, which means that each step is simple and does he does not "devise his own response". – nir Jun 2 '16 at 14:00
  • Right, sorry, just found that on the wiki page. – Tim kinsella Jun 2 '16 at 14:05
  • 1
    It is not a problem since all that is required is the capacity in principle, not in practice. Functionalist like dennett believe that once the computation is complex enough nothing will be missing. searle believes that the complexity of the computation is irrelevant (me too). searle used the thought experiment to argue that computation is what he calls observer relative. but other than that it is not very different than Leibniz's mill - home.datacomm.ch/kerguelen/monadology/printable.html#17 – nir Jun 2 '16 at 14:22
  • 1
    @Timkinsella, I think the thought experiment is purposely phantasmagorical. It is clearly impossible for the person using the rule-book to produce answers in a timely manner, and yet that point is irrelevant. For what does the timescale of the scene have to do with the principle? – nir Jun 2 '16 at 19:30
  • 1
    @Timkinsella, Dennett and Searle fundamentally disagree. Dennett believes that the chinese room can be conscious and Searle believes that it cannot. But their difference of opinions does not hinge on the details of the intuition pump. To the extent they are fighting about its details, it is just inconsequential skirmishes. BTW, what do you think? can a computer be conscious in principle? what do you think about Leibniz's mill? home.datacomm.ch/kerguelen/monadology/printable.html#17 – nir Jun 3 '16 at 6:25
3

John Searle's Chinese Room example is clumsy and is vulnerable to all sorts of refutations from a strictly technical point of view (The systems reply, The brain simulator reply, etc...).

But this is unfair to the argument, because beneath the awkward thought experiment there is a deeper epistemological question which does warrant serious consideration. John Searle in his lectures goes into the details and often repeats that "Syntax is not Semantics" (See the SEP article):

Anybody who has studied formal logic knows that rules like De Morgans laws or the laws of idempotency ( e.g. A ^ A = A ) are independent of the meaning of the symbols being processed. A rule of the type

IF A then: 
   B 
Else: 
   C

Works regardless of of the meaning of A, B, and C. But all a computer does is process rules of this type.

This is the idea that syntax (the rules) is independent of semantics (the meaning), and therefore a computer can function perfectly without ever knowing the meaning of what it is computing. Even the most advanced brain simulator, that can pass all sorts Turing tests is still ultimately just shuffling symbols around without ever knowing the meaning of those symbols.

Searle claims that this shows that no computer, no matter how advanced, canbe considered truly intelligent, since it lacks the understanding of the meaning behind the symbol.

Somehere in the lectures I linked to above. He does mention however, that if a biological artificial brain is produced, this might lead to true intelligence, since it would possess the biological characteristics of human brain processes, and will be driven by whatever mechanisms drive human brains.
See this question for further details: How can one refute John Searle's "syntax is not semantics" argument against strong AI?

  • 1
    Thanks, will take a look at the lecture later. For now I'll just say that to me, the distinction between biological and simulated seems arbitrary. With enough processing power you could replicate a human brain down to molecular scale inside a computer. It will function just as well as the original. – J.Doe Jun 1 '16 at 19:23
  • @J.Doe ". With enough processing power you could replicate a human brain down to molecular scale inside a computer." I agree with you to some extent. I'm not defending John Searle's position, just explaining it. See the post I linked to for more information. – Alexander S King Jun 1 '16 at 20:24
  • 1
    You write "John Searle's Chinese Room example is clumsy and is vulnerable to all sorts of refutations". I would like to know what is so clumsy about it. Most, if not all, philosophical arguments are open to attacks and the fact that the Chinese Room is does not make it any worse. On the contrary it is one of the famous thought experiments in philosophy of mind and not accidentally. Throwing such insults at it is like throwing insults at a mirror. – nir Jun 2 '16 at 6:59
  • @nir what is so clumsy about it is that it can be refuted on purely technical grounds: Anyone with some basic knowledge of computer architecture can respond "Of course the man in the room doesn't understand Chinese - it is the combination [man + rule book + db of chinese symbols] that understands Chinese". People get bogged down in this argument and miss John Searle's more important point of Syntax vs Semantics. – Alexander S King Jun 2 '16 at 17:12
2

The difference is that we are conscious, that is, we have the subjective experience of understanding and awareness. Remember, it was consciousness that we were trying to explain in the first place, not the ability to process input and produce output.

There is a difference between being able to produce Chinese answers to Chinese questions and hearing a question in Chinese and thinking, "Oh, I know what that means". The latter is consciousness.

  • 1
    But how do we know computers do not also have a subjective experience, or stated otherwise that our own subjective experience cannot arise from purely mechanical means? – Dan Bron Jun 2 '16 at 3:33
  • @DanBron: As long as they do not "behave" as if they do (show signs of it), we have no reason to assume it. It would be mere speculation, which is bad as it is open to even the lightest form of scepticism - and rightfully so. Call it Occam's Razor, Sellarsian Myth of the Given or whatever you like: We have the knowledge that there is no reason to assume it or that there can be said anything substantial about a computer's conciousness. Why should we bother to think about mere logical possibilities instead of truth-bearing reality at this point? – Philip Klöcking Jun 2 '16 at 10:16
  • @PhilipKlöcking By the strong church Turing thesis, there is a Turing machine that exhibits the same "behavior" as you or I, including all the behavior that leads you to believe that humans experience consciousness. – Tim kinsella Jun 2 '16 at 13:01
  • @Timkinsella: Yes, but that is a different problem. Here the question moves to 'Can it show the behaviour because it is meant to do so (e.g. by ways of programming) or because of the programming enabling the machine to be concious?' Here it is more complicated regarding language, but philosophically, there are answers on that for about 90 years now, i.e. the dependence on bodily positing in the corporal environment that is a precondition for conciousness. – Philip Klöcking Jun 2 '16 at 13:12
  • @PhilipKlöcking I'm not quite sure what you mean. Can you expound a little on the "answers"? – Tim kinsella Jun 2 '16 at 13:22
-2

"Consciousness" requires definition. As a neurologist, I use the term in as many as four different ways depending on the circumstances. Let's see if we can box it in for the purposes of this discussion by defining some related concepts:

Reflex: A simple deterministic system. Input -> output. Neurons are connected in series. Cognition: implies a complex system in which concepts govern the relationship between input and output. Input -> concept -> output. A concept is a set of linked attributes, in the sense that "hairy," "barking," "four-legged" and "smelly" might link to form the concept of "dog." Linkage between attributes represents an implicit theory. Theory = meaning. (See Bernstein, "A Basic Theory of Neuropsychoanalysis.") Neural network: a computing system made up numerous simple processing units that are massively interconnected. They are hooked up in parallel, as well as sequence. Cognition is the characteristic behavior of a neural network. All brains capable of thinking are neural networks. Artificial neural networks can be thought of as models of the brain, albeit with important qualitative and quantitative differences. Intelligence: a parameter of cognition. Anatomically speaking, intelligence correlates with the degree of connectivity of neurons in the brain. Humans have more connections than animals. Einstein had more connections than me. When we say something is intelligent, we mean that it has the capacity to carry a lot of information in the form of concepts and theories. Concepts can be numerous, subtle, and hierarchical. Theories can be deep and valid. Sentience: is a superset of intelligence that implies the presence of consciousness. Consciousness: remains to be defined. We can call it "self-awareness" for now.


The "deterministic" system described by OP is neither intelligent, nor conscious. The computer system OP describes is reflexive.

Not all computers are reflexive. Suppose the task were handled by a self-organizing neural network. The formation of concepts that drive output implies theory formation. Even though these theories are primitive and idiosyncratic, we could say that such a system has rudimentary intelligence.

To make this computer seem human, we need to build in enough processors, and a massive enough degree of connectivity to approximate the capabilities of the human brain, both in terms of raw storage capacity, and also in terms of the capacity to form hierarchical concepts. We would also need to build in emotion. "Emotion" may require assimilation of as few as three attributes: valence (punishment vs reward), intensity, and whether an approach or withdrawal response is called for.

I don't know that it's possible for us to design such a system. But suppose we could. Would it be conscious?

Perhaps not, because of the question of "self-awareness."

I dislike the term "self-awareness" because it's too easily defined in ways that are trivial or circular. It means something (perhaps not much) if you are aware of my self. Anything with a brain, and any computer, can do that. As to whether you are aware of your own self -- of course you are. You ARE your own self.

The concept of "self-awareness" only has meaning, then, if the nature of the self is ambiguous.

The question then becomes, under what circumstances would a computer be faced with an ambiguous self-nature?

To date, we have not evidently seen the need to design such a computer. Ambiguity is a bug, not a feature. To be clear, the more complex the system, the more difficult will be the design task, and the more unpredictable the result. But it will be a system nevertheless, one that will render its own particular output perfectly. Such a system will have no basis to contemplate or even to define its "self" in any meaningful way.

But what if we were to design a computer that could evolve? If it could perceive that it were changing over time, would that be enough ambiguity to cause it to consider the nature of its self?

I doubt it. All humans evolve, in a sense. We mature over time. But the change is seldom dramatic enough to cause us to question our own nature.

With regard to the question of the Self, humans are struggling with bigger issues. Priests tell us we are body and soul. Philosophers tell us we are an existence and an essence. Psychoanalysts tell us we are an Ego and a Self. Are these conundrums unique to humans? Or can computers get in on the action?

Possibly. The question is, what is the nature of intelligence? Where does it come from?

So far, we have been talking about intelligence as a form of information that arises as an emergent property of a complex system, in the sense that theories and concepts represent the work product of a neural network.

We also hinted at the obverse; in other words, the notion that complex things are based on information. When we talk about designing computer systems, for example, the "design" part refers to information. If a computer design is based on a blueprint; we can look at the computer as a physical manifestation of the information space outlined in the blueprint.

Likewise, our solar system is a physical manifestation of information. Information in the form of gravity came over from the infinite beyond into the "3+1" material world at a time when the universe consisted of rapidly expanding gases. Gravity caused gas clouds to coalesce into stars and planets. The structure of the information space carried in gravity implied the structure of the physical universe, including our home.

The human brain is a manifestation of an information space, and in spite of what one might have heard, we are nowhere near understanding the nature of that information. We have decoded the human genome, and found it to account for little of the brain's structure. The most important gene that distinguishes the human brain from that of animals is ARHGAP11B, which allows for wild chaotic branching of neural connections. Hebb's principle -- "neurons that fire together, wire together" -- then accounts for the development of the neural network. To say this is a non-deterministic process would be an understatement. In addition, there is evidence that the brain has fractal structure (www. stat.wisc.edu/~mchung/ teaching/MIA/reading/ fractal.kiselev.NI.2003.pdf - remove the spaces) and so there's another information space that needs to be accounted for.

What is the information space that defines the brain? And what (if anything) does that have to do with our concept of "consciousness?"

If consciousness is only ever a property of a distinctly human intelligence, I would submit that we cannot possibly design a computer that would have a consciousness separate from that of its maker. No matter how human it seemed, it would always be the puer eternis, never able to differentiate its Ego from the parent. (Might still try to kill us, but that's another story.)

But. To the extent that anything we imply in the word "consciousness" precedes human intelligence, we might be in trouble. Note well, the laws of thermodynamics suggest that it's a downhill run from star nurseries to intelligent life. If so, the information that gave rise via gravity to primordial star nurseries preceded and implied human intelligence. In that case, we can expect intelligence to stay in its meat suit as long as that's the best way to fend off the Second Law. When it encounters a more efficient solution, it will jump ship.

In summary, the answer to your question is: 1. The brain does not work like the computer you describe. 2. But, it is theoretically possible to design a computer that does. Whether or not such a thing is a practical possibility is another matter. 3. We can't know if such a computer could become sentient. Under some circumstances, it would be inevitable.

  • Very informative answer. I'm surprised however that you did not mention how quantum effects allow humans (i.e. socially constructed neural networks) to hermeneutically transgress boundaries that classical Turing machines and even relativistic Turing machines cannot. – Alexander S King Jun 2 '16 at 5:06
  • 2
    could you work on rewriting this answer to be much clearer and much less stream of thought? (your other answers on a few questions have been poor enough to look like spam). – virmaior Jun 2 '16 at 6:43
  • @Alexander S King Quantum consciousness is nonsense – D J Sims Jun 2 '16 at 15:04
  • 1
    @DJSims the response already invokes string theory, I'm not going any deeper down the rabbit hole by referring to quantum effects. – Alexander S King Jun 2 '16 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.