4

The Chinese Room setup is as follows, quoted from an earlier question on the same topic:

Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese.

If the boxes of symbols and the book of instructions only contain a handful of rules I can see the argument that the man inside the room doesn't need to understand any of their meaning to correctly follow the instructions.

But going from there to a set of rules complex enough to encode Chinese to me requires a leap that is similar to the pile of sand paradox. 3 grains of sand are not a pile of sand, and adding a single grain to a non-pile doesn't make it a pile. Nevertheless piles of sand do exist.

In analogy three symbols and rules do not imply any understanding to carry out and adding a single rule doesn't change that. But this is not sufficient to argue that one can manipulate a set of rules complex enough for Chinese without any understanding.

I find the idea that one can encode a language including the ability to use it in a real world context just by writing down a finite list of logical manipulation rules questionable. It is not clear to me why one can/ should assume that using a language can be written down as a list of rules that behaves just the same way as a handful of individual rules. In analogy there might be a pile of sand and the Chinese room argument seems to require that considering individual grains of sand is all there is.

Has this argument been made/ discussed/ refuted by philosophers?

7
  • 1
    I’ve broken down your question into paragraphs to aid reading. Just to clarify, your essential thought is something like “manipulating individual rules doesn’t constitute understanding, but manipulating the whole system might; it’s just that language systems are really intricate”?
    – Paul Ross
    Nov 9, 2021 at 8:36
  • @PaulRoss Thanks for the edit. Essentially yes, added two more sentences to hopefully make my question more clear.
    – quarague
    Nov 9, 2021 at 9:00
  • There is a similar to a pile analogy in Churchlands' and Pinker's intuition reply, but on speed rather than complexity. They compare the Chinese room to waving a magnet without producing light, and then concluding that light is not electromagnetic waves. "“The thought experiment slows down the waves to a range to which we humans no longer see them as light. Similarly, Searle has slowed down the mental computations to a range in which we humans no longer think of it as understanding (since understanding is ordinarily much faster)".
    – Conifold
    Nov 9, 2021 at 9:42
  • Your intuition is correct. See emergent properties. Not any of our neurons can formulate an english sentence, yet all together they form an entity that can. Similarly, the person in the room may not understand chinese, but together with the room and the set of rules, they form an entity that does, for all intent and purpose, read and speak chinese.
    – armand
    Nov 9, 2021 at 10:57
  • A universal Turing machine can have very simple rules for jumping from one location on the tape to another and changing or not changing the digit it finds there, with nearly all the complexity of the program being in the arrangement of digits itself. And a universal Turing machine can in principle execute any computable program whatsoever if it has a long enough tape, including say a detailed atom-by-atom simulation of an entire human brain.
    – Hypnosifl
    Nov 11, 2021 at 0:05

3 Answers 3

2

Searle is probably thinking of the fact that the rules needed to execute an arbitrarily complex program can be fairly simple, as demonstrated for example by Alan Turing's abstract model of computation, which has become known as the "Turing machine". There are various online outlines of the idea you can read like this one, but the basic idea is that the machine can read and edit data from a linear "tape" divided into a sequence squares, with each square containing one of some finite number of possible symbols, like 1's and 0's for tape filled in with binary code. The machine is supposed to only view one square at a time, and it has a finite collection of internal states, with these states giving rules for how the machine will behave subsequently depending on what digit it reads on the square. For example, a machine might have a numbered list of possible states, and state #8 could be something like "if you read a 0 on the current square, move 3 squares to the left and transition to state #17; if you read a 1 on the current square, edit it to be a 1, then move 2 squares to the right and transition to state #3." The rules are such that the behavior of the machine on each step depends only on the current internal state and the digit in the square the machine is currently reading, no other information about its history is needed. Turing also assume there would be a special "halting state" that determines when the machine is finished, and the pattern of symbols on the tape at that point would be the program's output.

If we define different Turing machines by their different lists of internal rules, Turing found that some of these machines qualify as a universal Turing machines (UT) which can simulate any other possible Turing machine, given the appropriate string of symbols for its input tape. So the set of all possible "computable functions" can be understood as the set of functions that can be calculated by a UT. Anything that we conventionally think of as a "computer program" could be run on a UT, including any arbitrary complex program for answering questions in Chinese (say, a detailed simulation of the brain of a Chinese-speaking person). All the complexities and unique characteristics of different programs would be due to differences in the symbols on the input tape, the list of internal rules for the UT would be the same for every program. And these internal rules can be quite simple--if I'm understanding the chart from this answer on the computer science stack exchange correctly, it's saying that if we want a UT that includes a halting state and that acts on binary tapes with only two symbols, 19 internal rules is all we need.

So Searle is just asking us to imagine a person playing the role of the Turing machine, jumping around between squares on the input tape and editing them according to the internal rules, with some physical stand-in for the tape divided into squares (say, a bunch of note cards laid out in a row with erasable symbols written in pencil). One could imagine the person has memorized the 19 internal rules, but even this isn't really necessary; they could also have something like a clock face with the different rules written at different positions on the rim, and a pointer that can be moved by hand, with the rules saying things like "Rule #8: if you read a 0 on the current square, move 3 squares to the left and shift the pointer to rule #17; if you read a 1 on the current square, erase it and write a 1, then move 2 squares to the right and shift the pointer to rule #3." And as mentioned before, the person wouldn't need to retain any memory of the symbols they'd seen on previous steps while executing the program.

So, I think there is no problem with Searle's claim that a hypothetical person with a mind basically like yours or mine (though perhaps with a much longer lifespan) could execute an arbitrarily complex program for answering questions in Chinese without the person themselves having any understanding of Chinese, since there is no correlation between the complexity of the program and the complexity of the rules the person has to understand.

Hopefully the above addresses your question. But as an aside, I'd like to add that there seem to be two distinct claims Searle is making about "understanding": 1) the claim that a person could execute a complex Chinese-speaking program without the person having any understanding of Chinese, and 2) the further claim the there is no entity or process within the room that can be said to understand Chinese. I understood your question to be only dealing with 1), so my above answer was defending the plausibility of 1). But that doesn't mean I would defend Searle's overall argument, because I think 2) is wrong--as I said in this answer to another question about Searle's thought-experiment, I think the main flaw in his larger argument is that the network of causally-interrelated events created by the person interacting with the cards using the rules could be said to have its own understanding separate from the person's.

7
  • 1
    You went through a lengthy explanation of TMs to finally devote only your final sentence to the claim that the "network of causally interrelated events" etc., "could be said to have its own understanding." Checkmark notwithstanding, I didn't find this responsive to the question. " You made a vague statement and didn't back it up. Not even wrong," as it were. Hence my downvote.
    – user4894
    Nov 12, 2021 at 20:47
  • @user4894 That last sentence wasn't intended to answer the OP's question, I already answered the question by saying I didn't think there was anything wrong with Searle's claim that a human could execute a complex program without understanding, based on the simple rules of a UTM being all the human would need to know and the point that "there is no correlation between the complexity of the program and the complexity of the rules the person has to understand".
    – Hypnosifl
    Nov 12, 2021 at 20:54
  • (cont.) That last sentence about the "network of causally interrelated events" was just an aside, meant to explain that though my answer was defending the plausibility of the part of Searle's argument that OP focused on, I do think there is a separate problem with Searle's argument which I discussed more in the other linked answer, so that I don't agree with his ultimate conclusions (if I hadn't put in that last sentence, the OP might get the idea that I support Searle's overall argument about the larger system not understanding Chinese, as opposed to just the human in the room).
    – Hypnosifl
    Nov 12, 2021 at 21:02
  • Ok you talked me into it. When I went to upvote it said, "Your vote is locked in unless the OP edits their post" and didn't let me fix it.
    – user4894
    Nov 12, 2021 at 22:32
  • @user4894 I edited it to make more clear that the last part (now in a separate paragraph) was just an aside about Searle's larger argument, not directly relevant to the part of Searle's argument that the OP was asking about.
    – Hypnosifl
    Nov 12, 2021 at 22:53
0

I'd relate this to the Private Language Argument. And look to Hofstadter's strange loops for a structural difference between rule intelligence and agent intelligence.

Children raised by wolves, without human company past puberty, never learn to speak. They may perhaps make more subtle inferences than a wolf. But without language, and the tools that go with it (at least those beyond mimicry), a human is not a very special animal, & without creche-rearing and education likely less good at problem solving than corvids or cephalopods.

Wittgenstein analysed the intelligence of language as contained in modes of life. Contextual cues, lived practices, associated experiences, and the sharing of these. The power of language begins in it's intersubjectivity: if I were you, if you were me. It groups things, highlights things, allows continuities to be narrated - in salience landscapes, and enables cognitive grip for socially useful manipulation of concepts, like say mathematics elaborates subitism (a generalisation of the idea of sets of 'identical' things, another symmetry operation like intersubjectivity).

Language requires rules, time saving abstractions that facilitate learning. Infants intuitively grasp grammatical rules even when their vocabulary is minimal. The rules contain learning, many people have contributed to mathematics just like to all languages, and the education of children consists not just of learning the rules but of connecting them with the modes of life that make them intelligible, experiences & applications.

Rule following is not a private experience. The sand is not a pile or not privately, but socially, it's 'pileness' is not a direct experience, but a token we use to relate to it, functionally, for purposes. An intersubjective rule of a widespread community, seems like an 'objective' rule, one that's 'out there'. But, even though the rule-following to be communicable cannot be private, it still begins with subjective experiences, the ones that can be shared, transferred, abstracted.

The Dunbar Number points to humans developing their neocortex primarily for understanding the motivations of others, not problem solving. That is, to formulate our observations of community rule-following in the form 'if I was them', and the massive specialisation in this blinds us to the complexity involved.

Rule-creativity takes place in the mental 'global workspace' - we become conscious of the exceptions, the rule-failures, the challenges, and integrate them there through our sense of agency: in relation to our understanding of ourselves in relation to others (well understood familiar tasks go with the 'flow state' of unconscious competence).

That creative place, the global workspace of conscious working memory, is the intersubjective experience, that allows the rule-creativity to propagate, private playing with the rule, public framing of the change into a regularised structured abstraction (to be communicable): the joke, the new theorem, the sudden insight. Because the rules aren't fixed like objective rules, but are really intersubjective, affirmed or transformed by enactment of the community, in modes of life.

So language rules contain intelligence, just like DNA - it distils useful shorthands about our environment, as part of our extended phenotype, in a memesphere. The rules can be followed by a computer. But the rule-creativity requires a global workspace situated intersubjectively, which can watch modes of life alter update transmute and above all play, with meanings, in relation to the agents self, which interprets and conceptualises in relation to other agents who's modes of life the rules were drawn from.

We need a structural shift for AI to 'understand' Chinese, with not just rule following but pursuit of understanding of intentions, which language is only the expression of.

12
  • When you talk about a program not being able to achieve understanding by rule-following in the last paragraph, are you just talking about high-level rules a la "symbolic AI", or are you including any rules whatsoever, including things like the computational rules governing simulated neurons in some type of artificial neural net? Searle intended his argument to apply to both types, see his comments on the "brain simulator reply" on p. 420 of his 1980 paper.
    – Hypnosifl
    Nov 14, 2021 at 16:04
  • @Hypnosifl: Either system is capable of understanding, but not by rule following alone, there are also structural requirements - engagement with being an agent among other agents, & seeking to understand their intentions, intersubjectively. This model helps account for cephalopod intelligence even though they are mostly solitary, because they try to understand a range of predator & prey strategies, & have been in an arms-race doing so since before mammals existed
    – CriglCragl
    Nov 14, 2021 at 17:45
  • Sounds like the 'robot reply' which is also on p. 420--Searle would say an input-output relation with the real world (and other intelligent beings) isn't enough, since a person could carry out all the internal computations between sensory inputs and bodily outputs without the person understanding them. Also, what about something like a giant simulated world including multiple simulated agents that can interact with each other, but where the whole simulation is self-contained and not receiving any input from the 'real world'? Do you think agents can have intersubjective understanding there?
    – Hypnosifl
    Nov 14, 2021 at 18:03
  • @Hypnosifl: & I'd agree with him, until the intersubjective integration space criteria is met, that allows - rule-creativity in community. Yes to simulation, absolutely no problem. In fact that's how I expect Artificial General Intelligence to occur. A bunch of Alphazeros interacting, say, plus some kind of evolutionary selection among them.
    – CriglCragl
    Nov 14, 2021 at 20:34
  • But in his response to the "robot reply", Searle is imagining a scenario where you have a robot that has a body and is capable of interacting with others--in your view would that allow for "intersubjectivity" if its interactions seemed like those of a meaningful agent to people around it, having conversations about things going on around them, making plans and doing stuff together in the world? Searle thinks such a robot still wouldn't have any meaningful understanding if there was a humanlike agent inside the robot doing the computations by hand, and that agent didn't have such understanding.
    – Hypnosifl
    Nov 14, 2021 at 23:16
-1

The symbols of a system can be arbitrarily large, but the rules for manipulating those symbols are generally quite limited. For example, arithmetic may have an infinite number of possible equations, but the rules for constructing a valid equation are few enough in number that most people master them as children:

  • a term defined as a sequence of symbols that begins and ends with operands, sometimes encased in parentheses to show precedence
  • a term sequence defined as a set of terms separated — sometimes implicitly, as in 12x or 2y — by arithmetic operators
  • an equation defined as two term sequences separated by an equals sign

Those three rules define any valid arithmetic equation. They do not say whether an equation is true or false, much less whether an equation has any practical meaning or use. But they do tell us that (say):

  • z4m = 14c 12 12

... is improperly formed. And yes, I know we could adopt some of that weirdness as rules in other systems (e.g., 14c as notation for carbon-14 in chemistry, or z4m as a valid variable name in a programming language), but that's beside the point. In arithmetic it's nonsense, and easily identified as such.

The same holds true of language. Semantics is deeply complex and nuanced, but syntax is pretty simple. Our computers all have spell-checkers and grammar checkers that don't require deep-learning neural nets to work; they are simple algorithms that apply standardized rules and check a short list of irregular forms. There are ambiguities in meaning that make sentences confusing, but there aren't ambiguities in syntax. For instance, the classic test phrase:

  • Time flies like an arrow, but fruit flies like a banana

... will pass through your grammar-checker perfectly fine. The confusion comes because there are several different ways of organizing the subject-verb-object structure of the sentence, all of which are grammatically correct, and we have to choose between them. There's a whole lot of language theory behind this, but it's mostly found in linguistics or the social sciences, not in philosophy proper. It's a bit too technical for most philosophers to bother with.

1
  • 1
    Isn't this simplifying the task for the man in the Chinese room by a lot? He is not only supposed to create grammatically correct Chinese sentences (this can be done fairly well with a small set of rules in different languages, afaik Chinese is especially easy) but he is supposed to give meaningful answers to questions indistinguishable from a Chinese speaking human.
    – quarague
    Nov 9, 2021 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.