8

Suppose you have a conscious AI and you run its code by hand on a piece of paper. Would that AI still be conscious? If not, what is it about transistors that breathes life into an otherwise soulless process?

Can this logic also be applied to the human brain? If a simulated paper-brain cannot be given consciousness by transistors, why would it be able to be given consciousness by neurons?

3
  • 1
    "why would it be able to be given consciousness by neurons". Stomachs actually digest and computer simulations of stomachs don't "digest" when run. Whatever the brain analogue is giving rise to consciousness (probably not just neurons), the same reasoning is applied by Searle. It takes the right physical mechanism. And computers aren't it, unless they become much more physically similar to actual stomachs or brains.
    – J Kusin
    Jul 29, 2023 at 20:21
  • If said stomach is programmed to be as realistic as the real thing then it can digest virtual/theoretical food (that is, food that inhabits the same realm). Can the same be said about the digital brain? Can a digitized consciousness be simulated the same way a digital digestion can?
    – Dimitris02
    Jul 29, 2023 at 20:39
  • It could be used to demonstrate just the opposite. If a bunch of stuff that knows nothing about chinese can be arranged in a way that make it look like it knows chinese to the point that everybody on the outside would conclude "there is someone in the room who knows chinese", it's an argument for emergentism, a key part of physicalism as it claims that from a bunch of unconcsious stufff properly arranged can emerge conciousness.
    – armand
    Jul 31, 2023 at 1:27

6 Answers 6

7

The short answer is that a defense of dualism is not what John Searle was doing with the thought problem. He was showing how knowledge is different from function -- IE it was an attack on functionalism.

Longer answer: Searle would agree with your further elaboration on the paper version of an algorithm being no different from an electronic one, and neither being plausibly identical with consciousness.

Searle's argument is a thought problem, designed to exercise our intuitions, not an experiment. It is therefor not definitive -- it is possible in principle that SOME kinds of algorithms could be identical to consciousness, and that perhaps the "running" of an algorithm (Searle would note that "running" is not a logical state, so this is an invention of some new criteria, not actually functionalism) matters, and maybe it is the method of running, not just the algorithm that gives rise to consciousness.

Searle’s thought problem however has had a significant impact on philosophy of mind. IIT is explicitly postulating the response above -- Algorithm PLUS running it on a substrate with a large Phi is needed to create consciousness, per IIT.

Searle was a biological emergentist, so he was a form of physicalist. His response to your casting the Chinese room as an argument for dualism, is he would have argued that we have good reason to think the mind is dependent on the brain, so he speculated that there was some aspect of the biology of brains that cause minds to emerge from them. Most physicalists today have followed Searle and are not functionalists, but instead are emergentists, like Searle was.

0
6

I once attended a lecture by John Searle (circa 1996, on the UC Berkeley campus) in which the Chinese room thought experiment and its consequences was a primary topic. At one point I asked him essentially the same question as asked here: "How is this different from dualism?"

Although I don't recall precisely how he answered this question, he did insist it was different. That is, he felt that the thought experiment did imply that consciousness was more than computation, but did not imply it was non-physical.

So I asked him a follow-up question similar to yours, essentially pointing out that the functions of a neuron are sufficiently well understood to be fully simulated on a computer, and therefore one ought to be able to (in principle) simulate a complete brain using only computational elements. I vividly recall his response: "If you simulate the weather, you don't get wet."

At the time, I found this answer shockingly superficial, although I struggled to articulate why, and didn't press any further.

I think Searle is wrong. I think, if you have an object (such as a room with a person inside following mechanical rules) that exhibits conscious behavior, then it is conscious. If you contend it cannot be conscious because none of the parts are conscious, then you have to either say the same of a brain made of physical neurons, or say that something in the brain utilizes something non-physical.

3
  • 2
    If it is indeed possible to completely characterize consciousness in terms of a set of behaviors then I agree that we must conclude that an object that exhibits (all) those behaviors is conscious. There's no great leap there. The issue is with the assumption that what we mean by "consciousness" can be characterized that way -- this is by no means clear or established. Neither is it clear that a machine could be made to exhibit those behaviors. I guess it would be the physicalist position that one could be, but that's speculative. Jul 30, 2023 at 18:55
  • 1
    It is indeed a shockingly superficial answer, and it can be found in the "Chinese Room" paper. One response would be to point out that a computer simulation of an Enigma machine really does encode and decode messages.
    – A Raybould
    Jul 31, 2023 at 18:42
  • @JohnBollinger As it remains speculative, Searle's goal, which is not to provide any sort of explanation of consiousness, but to rule out any along these lines, has failed.
    – A Raybould
    Jul 31, 2023 at 18:51
3

The conclusion to draw from the idea of the code of a conscious AI being run on some kind of steam powered Turing machine, with paper tape and wooden cogs and levers, is that the AI cannot be assumed to be conscious after all.

10
  • Why? Isn't the brain inevitably an organised lump of atoms, analogous to a Turing Machine? What magic differentiates it if not? Structure? Quantum processes? Isn't any possible mode of differentiation, subject to copying by an 'artificial' simulation? A fertalised egg cell can build a consciousness, with materials inputs. Surely a code could be highly analogous to that DNA strand & do the same process
    – CriglCragl
    Jul 30, 2023 at 11:45
  • @CriglCragl OK. Suppose we construct the steam powered, wood and paper Turing machine, and we get it to execute, say, one instruction an hour, so that it would perform a small AI task over millions of years- would you consider it to be conscious? I'm not trying to make a point here- just trying to elicit your thinking. Jul 30, 2023 at 12:35
  • Yes. Like this xkcd.com/505 There's reason to think interaction with the environment is important, which is akin to how calibration is needed for sensors, but in an ongoing way.
    – CriglCragl
    Jul 30, 2023 at 19:53
  • @CriglCragl, the well known interaction-with-environment argument is a fallacy as shown by said xkcd strip since it too can be included in the simulation.
    – nir
    Jul 31, 2023 at 18:40
  • 1
    @nir That would be an example of a simulated world, per Bostrom. They are speculative, but not fallacious.
    – A Raybould
    Jul 31, 2023 at 19:00
2

While imaginative thought experiments like the 'Chinese room' raise intriguing questions, we must be careful not to overextend their implications when it comes to complex neurological phenomena like consciousness.

The brain is not analogous to a simple software program - it is an extraordinarily sophisticated biological system that gave rise to mind through eons of evolution. Its 86 billion neurons and trillions of dynamic synapses produce emergent properties that cannot be reduced to the sum of their parts.

Just because a human lacks subjective experience while mimicking a program does not imply that same program running on neural networks would too. The brain's parallel processing capacity, structural complexity, and adaptive plasticity enable first-person conscious awareness to arise in ways that fundamentally differ from rule-based simulations.

Dualism often seems tempting given the deep mysteries of subjective experience. But the research indicates consciousness is inextricably tied to the brain's material workings.

2
  • I don't think a software is inherently reductionist. If you look at Conway's game of life it's clear that computers can display emergent behaviours given a set of simple rules. The practicalities of simulating a human brain are not very relevant. If you DID have the know-how and computing power to simulate one neuron-by-neuron why wouldn't the phenomenon of consciousness emerge?
    – Dimitris02
    Jul 31, 2023 at 12:23
  • Also I'm not sure which research you are referring to. All neuroscience tells us is that certain stimuli are located in certain areas of the brain and affect our behavior. There is nothing about why this stimuli is "felt" rather than simply take its course like a domino. Clocks, computers and the earth itself are all examples of big systems with inputs that affect outputs. Why aren't they conscious? What is it precisely about the brain's structure that makes its reactions to input not only happen, but felt to happen?
    – Dimitris02
    Jul 31, 2023 at 12:32
0

Suppose you have a conscious AI and you run its code by hand on a piece of paper. Would that AI still be conscious?

By a "conscious AI", I take you to mean a Turing-equivalent computing system that computes outputs that we accept as fully and unequivocally characterizing consciousness. If it were possible to reproduce the same computations by simulating that machine with different hardware, such as pencil and paper, then the simulation must also be accepted as conscious, by virtue of computing the outputs that we already stipulated we accept as characterizing consciousness.

Can this logic also be applied to the human brain?

Of course.

But if you mean to use this as an argument for physicalism then you have two huge holes to fill:

  1. You need a satisfactory definition of consciousness in terms of observable behaviors of a human brain.

  2. You need that simulated brain, fully reproducing the biophysical behavior of the human brain, and manifesting the behaviors accepted, per (1), as defining consciousness.

0

The "Chinese Room" argument is like saying that my brain cells are not conscious, therefore I am not conscious. And since the transistors making up a computer are not conscious, therefore a computer cannot become conscious.

Now it is absolutely true that we have never (yet) been able to create a conscious computer, but the "Chinese Room" argument that it is impossible is very, very weak.

But now lets assume there is indeed a computer with software that is conscious, and has an intelligence comparable to mine, and a speed comparable to mine. (I believe that computer would necessarily have very powerful hardware for a computer. And a conscious and intelligent mind inside and likely connected to a powerful computer could do things that I couldn't, like finding the one billionth prime number faster than it takes me to find the one hundredth). And now lets say that instead of using this very powerful computer, we use pen and paper to do the same operations.

The result would be conscious and intelligent but just absurdly slow. Probably slower than the original computer by a factor of 100 billion. Make me 30 million times slower, and it takes me a year to give an answer that I could normally give in a second. Make me 100 billion times slower, and it takes me 3,000 years. I would be indistinguishable for all practical purposes from a dead body. It would have taken me a million years to type this reply. And the same with the "pen and paper" computer.

3
  • 1
    This last paragraph raises a question. Why do you need to write down the inputs and outputs of the simulated brain for it to become conscious? If you don't know the answer to 2+2, does that mean that the answer doesn't exist until you find it? Likewise, while you are taking your time to calculate how a simulated brain would respond to a stimulus, the answer already exists. What you write down on the paper says more about what you know about the answer than the answer itself. And the same applies to computers. Computers do not create anything, they explore what already exists
    – Dimitris02
    Jul 31, 2023 at 12:44
  • @Dimitris02 Without hardware, no consciousness. “Pen and paper” hardware is taken directly from the question. Me writing down things is just me being used as very cheap and very, very, very slow computer hardware. My role is the same as the role of transistors in a computer.
    – gnasher729
    Jul 31, 2023 at 18:13
  • It seems the fact that a conscious human does some brain-dead work seems to confuse you. You are mixing up levels.
    – gnasher729
    Jul 31, 2023 at 18:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .