4

I've quite often seen AI respond to John Searle's Chinese room argument by accepting the systems reply: while the man in the room doesn't understand Chinese, the room (the system) as a whole could - or at least the Chinese room argument doesn't conclude that it could not.

Yet the systems reply doesn't seem to address the main problem the CRA poses for AI. One of Searle's premisses says that symbols are semantically vacant, are intrinsically meaningless, that in themselves they don't indicate what they mean or refer to. And all the computer gets is the symbol (i.e. tokenized shape).

Given that computers do process symbols, the sensory symbols received from digital sensors will not indicate what was sensed. How, then, is it possible for a computer to come to know and understand the world?

4
  • 2
    How indeed. I wonder why anyone thinks it might.
    – user20253
    Aug 22, 2018 at 11:40
  • 2
    I don't see what AI has to do with any of the points raised in this post. You can ask the same question about how a human comes to understand symbols, if at all.
    – user6559
    Aug 22, 2018 at 20:24
  • @user6559 Humans have instincts; if AI is to have instincts, we have to program those in.
    – wizzwizz4
    Jul 21, 2022 at 15:45
  • Knowing all about something is impossible. So, it is impossible for any entity to completely know its environment: any representation (knowledge or bytes in computer RAM memory) is a map of the terrain, not the terrain itself. So, the straightforward answer is: with a sensor (in humans, those are the five senses).
    – RodolfoAP
    Jan 4, 2023 at 15:48

6 Answers 6

3

Let's look at "Chinese Room". The words traverse the optic nerve as a complicated neural pattern with no semantic significance. Once they hit the brain, the brain can assign meaning to the letters, words, and phrase. If they hit the brain of someone only literate in Chinese, they'd look like meaningless symbols.

Therefore, the room and the brain are alike in receiving meaningless input and assigning meaning. The processes in the room assign meaning to the symbols that Searle doesn't see any meaning in.

It's important to remember the context of the Chinese Room, the Turing Test. In this, a human communicates with two text interfaces (teletypes in the original), one another human and the other a computer, and attempts to determine which of the other is the human and which the computer. This has been widely been considered a test of artificial intelligence. (Turing did not claim that; he predicted, inaccurately, that the computers would be reasonably successfully in 2000.) Therefore, the comparison is between the Chinese room getting Chinese text and a Chinese speaker getting Chinese text.

5
  • 1
    "the room and the brain are alike in receiving meaningless input and assigning meaning". This is the issue that seems crucial to me. How does the brain get the meanings that it assigns? And where does it get them from? Not the input streams – they are meaningless. But they are also the only things the room (and brain) gets. Conclusion: the input streams must not be meaningless. We know where the meanings come from: sensors. We know what the meanings are about: the things sensed. But how is the meaning embodied in the stream? = THE question.
    – Roddus
    Aug 24, 2018 at 2:41
  • Let's take this up a level. This web page would have been completely meaningless to me about sixty-three years ago. (I remember looking at a newspaper and not being able to interpret any of the writing.) If there is intelligent life elsewhere in the Universe, it would be meaningless to them. There's not enough text here to analyze it and determine any meaning. However, you and I find meaning in it. The stream isn't arbitrary or random, but it does not in itself convey meaning without being processed by certain means. Sep 7, 2018 at 18:28
  • I agree that the stream needs to be processed by cerain means in order to reveal (if that's the right word) the meaning it contains. The really interesting part to me is understanding how the meaning is embodied in the stream (which will indicate how to extract it an make it permanent in storage). But I'm not sure I agree that meaning can be determined with enough text. For attacking encrypted data, the more ciphertext you've got the better, but I don't think this idea works for text and meaning. Bickhard and Terveen wrote good book on this (they called text, encodingism).
    – Roddus
    Sep 17, 2018 at 1:31
  • My comment for posterity is that words do not traverse the visual system (other than metaphorically); words are constructed just like colors (with the help of additional portions of the neocortex. If they "traverse", then they must "come from somewhere". To see the brain "getting meaning" from "meaningless stream" is to jump a false dichotomy. There are stimuli which are shaped by the low-levels of neurons in the visual system (light, hue, saturation, shape and what not), and then those appear to the conscious mind as a grapheme. From there, the neocortex associates graphemes with experience.
    – J D
    Jan 4, 2023 at 21:18
  • and also Turing predicated his belief about the success of computers presuming principles of learning. To be fair to Turing, machine learning and connectionism have faced vocal opposition from symbolists who are primarily the sort of people who are semantic externalists, mathematicians and their Plato veneration.
    – J D
    Jan 4, 2023 at 21:20
2

I guess much here depends on the definition of "knowledge" and "understanding" -- today, self-driving cars already learn about their environments. (I considered putting quotation marks around "learn" but it is common practice not to in this context.) So for a weak sense of knowledge and understanding, where we require only (say) that there is information in the system that correlates with the world and is used to take desirable actions in the world, today's systems already acquire it. Of course one might be interested in a stronger sense of knowledge and understanding, perhaps involving some conscious awareness. This is what Searle's argument is about, and we're not going to settle that discussion here...

14
  • 1
    I'd probably dispute the self-driving claim. The image training sets used for deep learning artificial neural nets are made of a gazillion images, but usually each is annotated by a human who enters codes against object types: tree, stop sign, pedestrian, van, child, etc. Good training sets are available online. Human annotation isn't really the system acquiring the knowledge or of AI systems "learning about their environments". It seems more part of a process where a human defines the causation of the system by using the human's knowledge.
    – Roddus
    Aug 24, 2018 at 0:17
  • As for conscious awareness, I know Searle has spent some time on this topic. But I really see consciousness as a separate issue from semantics. It's conceivable that a system could have a quality semantics that enables it to survive in the wild, but has no consciousness. In fact there's been a debate about what is the evolutionary benefit of consciousness. Whereas the evolutionary benefit of a semantics is fairly obvious. I see the CRA as part of the programme of trying to understand how a machine could get a semantics. Isn't this an earlier issue – before that of consciousness?
    – Roddus
    Aug 24, 2018 at 0:18
  • 1
    @roddus you say that humans have to encode all the training materials used by self-driving cars, but humans also have to encode everything that is used to train other humans. Had I been born in a world without other humans, my understanding of the world would not have developed in the way that is has. Jan 4, 2023 at 16:02
  • @MarcoOcram The fundamental argument you are arguing against starts intuitionally with the metaphysical claim that electromechanical computers are bit-twiddlers which lack a vital force. The objectionable part to the bit-twiddler camp is that knowledge is derived of belief which is a human activity. What argument do you make to that computers are capable of belief?
    – J D
    Jan 4, 2023 at 21:12
  • @JD I don't make any such arguments- I was simply pointing out that humans acquire knowledge with the help of other humans. I agree that computers are bit-twiddlers without consciousness. Pocket calculators don't 'understand' maths. Jan 4, 2023 at 21:25
2

Algorithmically calculable answers must be part of Godel-incomplete systems, with true but unprovable statements. But a strange loop system can form a tangled hierarchy, a network of reinforcement and doubt, like language in use, where tentative uses for symbols are used then refined and meaning created relationally and through interplay and interaction.

When our brains cease to get input from the optic nerve, they don't close part of the brain. It (eventually) hijacks and bootstraps the area to continue to try and create a model of the world as isomorphic as possible to it - as demonstrated iteratively, by patterns of interaction, cross-reference, etc - ie by use.

Your premise that symbols are semantically vacant is untenable, in terms of viewing language in practice - they are imbued with it by use. Mathematical systems reveal consequences to statements already made, that is they don't generate meaning but unpack it. Creative mathematical thinking however, works forwards to consequences & backwards to axioms, then around again, in interaction with the world, creatively.

How, then, is it possible for a computer to come to know and understand the world?

Through interaction, trial and error, exploration, heuristics. Just like us. Whatever means it had to interact, they would be senses - including mental models & simulations, which we use for instance in motion prediction for game playing.

The Chinese room is only intelligent like a deck of cards, or an abacus, say. True Artificial General Intelligence would have to be in a strange loop, not such a flat hierarchy with explicit defined rules.

7
  • You make some interesting statements about strange loops, their formation, and their use. Do you have some pointers to explication? Aug 22, 2018 at 17:58
  • @CriglCragl You say "Your premise that symbols are semantically vacant is untenable, in terms of viewing language in practice - they are imbued with it by use". So symbols are imbued with semantic content by virtue of use of the (those same) symbols. But isn't what really happens this: brains get imbued with semantic content by way of use of external symbols (and other things)? The content doesn't inhere in the symbol, rather it inheres in the brain? This issue of sensory semantic content seems much more primitive than any mathematical understanding of human processes and structures.
    – Roddus
    Aug 24, 2018 at 2:25
  • @Roddus Niether the brain nor the symbol. The culture of use is between the two.
    – CriglCragl
    Aug 26, 2018 at 0:30
  • @DavidThornley We discussed it on here philosophy.stackexchange.com/questions/22926/… I esprcially recommend the Wittgenstein-Nietzsche-Rorty article I posted there. If you seriously consider Munchausen's trilemma, strange loops are the only answer. There is no 'correct place to begin', there are only reinforcing movements through thought-structures using various modes, stepping in and out (axiomatising), unifying (regression), and looping (circular reasoning).
    – CriglCragl
    Aug 26, 2018 at 0:40
  • @CriglCragl Is your use of GEB's Strange Loop just a poetic euphemism for emergence from complexity?
    – J D
    Jan 4, 2023 at 21:14
1

I mean one possible way to attack it would be to point out that, just because you aren't aware of the semantic information doesn't mean that there isn't semantic information intrinsic to the subject.

So you can try to find patterns within the noise and relations between the patterns. So that you can create word clouds without knowing what the words mean. Like if a sentence works with all symbols the same except one you can infer that there is an equality in some sense between those two words. You might be able to infer important ideas by the number of connections or filler words by the combination of being used frequently but the ability to be left out of a sentence and still be valid according to the syntax.

Now a human observer is capable of inferring quite a lot from the simple fact that this person is a human as are the people who invented the language so there are certain quirks of the human condition that you could infer are so universal that you might start trying to map them.

But even if that language was not Chinese but completely alien to any human you might still try to find regularities, make predictions for the output, check whether the predictions and the actual responses match and refine your predictions because of that i.e. do science.

And to some degree that is already happening with machine learning. You give a program an input, they literally guess (generate a random number) and output that. Then the user tells them the real output and they compare the guess and the real value and change their guess. So idk if they are meant to guess a result of an addition like 2+2 then they might guess 3, receive 4 as answer compute 4-3 = 1 which is positive so they change their parameters to create a bigger number let's say 5 where again they compute 4 -5 =-1 which is negative so they adjust the parameters to get a smaller number but bigger than 3 and so on until they got to 4.

Now they technically still don't know how to add, they just know that 2 on one input and 2 on the other input equals 4 as the output. But from the fact that a swap of the input results in the same output you'd already necessitate an intrinsic symmetry. Now you could be wrong assuming those are numbers but in reality those are words, but you'd still have given them some sort of semantic meaning even if it were wrong.

So if you assume that this is not a 1 way communication but that the computer uses it's outputs to make inferences on the environment than it could also progressively improve it's guesses and assign meaning to things that are arbitrary and meaningless.

1

There is no way a computer could acquire knowledge.

Knowledge can be defined as interpreted and understood information, that has a meaning. Computers can only process information, meaningless data, they cannot interpret or understand it, find any meanings. Computers don't know or understand anything.

A computer can be programmed to change its behaviour depending on the information it receives. This means that the programmer understands what the information means and can decide what the computer should do with it.

3
  • And what is your argument that computers aren't capable of belief?
    – J D
    Jan 4, 2023 at 21:21
  • Computers have no mental capabilities at all. They are only calculating machines. Jan 5, 2023 at 9:02
  • Can offer a small observation. When we humans understand, say, text in a book, the symbols (linguistic characters) are external to the brain. We sense the text shapes and neural impulses travel to then enter the brain and we understand the meanings of the text (assuming we know the language, jargon, idioms etc.). But we don't understand the neural impulses propagating along the fibers of our neurons. So presumably the thinking computer will not understand the so-called "symbols" (in fact instances of electrical binary differences) it receives from sensors then internally processes.
    – Roddus
    Jan 6, 2023 at 6:46
0

Same way we do. By acquiring data (through senses) and analyzing it.

You have to first have a precise definition of knowledge.

Data -> Information -> Knowledge -> Intelligence

Data is raw input. It always come from senses. Whatever senses your organism or machine have. In humans and computers both we always have it in form of electrical signals.

Information is arranged data. Sort it, average it, find the highest or lowest values, find mode and median and so on.

Knowledge is much more involved. Its about finding rules from information. Such as:

  1. Amount paid never exceed 1,000,000 in a month.
  2. Whenever customers buy jackets from our shops 65 percent of the time they also buy juices to consume while shopping.

We have never been able to make a computer program that can find rules on its own. All our so-called A.I. work is algorithmical deep down, therefore has no intelligence whatsoever. It don't even make a knowledge base on its own.

With or without our assistance, if an information base is built, then we have whats equivalent to symbols for our computers. The data now has meanings.

On basis of those meanings it can make better decisions about its actions and can have better predictions. Once your self-driving car understand how signals work it obviously drive better.

To understand how signals work (an example of knowing a rule) it must have an idea of all traffic stopping (maximum value of location variables of other cars), colours of signal lights etc. The location values of other cars and the colours of signal lights are symbols for computers.

Once data become symbols it has meanings. Those meanings can then be used in finding rules. Once rules are known to an organism / computer it has knowledge of the system and environment it work in.

8
  • I agree that "All our so-called A.I. work is algorithmical deep down". But it doesn't have to be. The algorithm (program) could simply facilitate the creation of internal structure where both the content of the structure and the structure itself is determined by the sensed environment. Us patents 6,414,610 and 5,748,955 include algorithms which do this by extracting redundancy (repetition) from input streams.
    – Roddus
    Jan 6, 2023 at 7:00
  • "Once data become symbols it has meanings". So the "data" is the raw signals (what people like Searle call "symbols") and the "symbols" are what AI calls "internal representations"?
    – Roddus
    Jan 6, 2023 at 7:09
  • @Roddus We haven't yet found any method to make softwares decide on their own what rules to look for, therefore softwares cannot make knowledge base (know about rules) on their own. Yes data is always raw signals, as raw as it get. Yes, internal representation is information, arranged data.
    – Atif
    Jan 6, 2023 at 8:46
  • Is a rule program code? So if a software found a rule then the rule would be realized as lines of program code added to the searching program? The program would, as Turing (1950) suggested, (a) modify itself, or (b) learn by virtue of having rules which were ephemerally valid?
    – Roddus
    Jan 6, 2023 at 8:56
  • As for now yes a rule is a program code. Computers are unable to find them. Lot of data mining is done, by humans, then such non-obvious rules / symmetries / definitenesses (if x happen y always happen or happen z percent of time) are found, by humans. Once found knowledge base start building up. Computers are very good in using the knowledge basis (think expert systems, computer-aided problems solving etc). Now, none of this is A.I. Intelligence is on a different, higher level altogether. Its to guess rules not even hidden in data i.e. predictions of missing rules.
    – Atif
    Jan 6, 2023 at 12:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .