If you consider a human and a very smart computer, you have no trouble turning off the computer but most people would consider "turning off" a person to be bad.

If a computer becomes as smart as a human, you still intuitively know [citation needed] that turning off the computer is not a bad thing to do and turning off a human is a bad thing to do.

What if the robot (computer) can do everything a human can do? What then? If you cannot tell the difference between the robot and the human then how would you feel about turning off the robot? For me, I would still have no trouble in turning off the robot because I know how it works fully and I know it has no spirit. However the problem here is the spirit is something which is not defined.

I intuitively believe as an axiom of my reasoning that there is some sort of spirit or soul in people, animals and tree's (life) that differentiates life verses machines. This is a axiom of my reasoning of course since I cannot construct any rational basis for believing it.

We all have axioms of reasoning, but I believe many other people have this axiom as well.

I am sure many people believe this, if you conduct the thought experiment as I have outlined above that most people would be happy to turn off the machine but not happy to turn off the human. They would most likely say because the human has something "special", a "soul", or something which cannot be defined, but you would liken it to a "soul" or "spirit".

So the question is, what is the soul/spirit as I have discussed? The difference between a person and a machine? Can it be identified or defined?

If we ultimately build smart robots as smart as a us, then why should we believe we are more special than them? It must be on the basis of a "difference" between man and machine. That difference must be a "soul", something which I find very difficult to define.

There are consequences of particular answers, if you say that it is just how we evolved to value each other, then you deny the soul's existence and then there is nothing special between man and machine. Then you have no logical reason to value people more than machines, and ultimately we may end up in a dystopia where machines are more valued than people.

If you say there is no such thing as a soul, then fine, but is there still something "special" about people when compared to a smart robot which acts as exactly like a person?

Please don't think of the word "soul" exclusively in terms of religion, i.e. Christianity, I am talking about that which we intuitively believe is a "soul" but we are not taught. Even non-religious people would say there is something special about people that has a "soul" like character, they cannot rationalize it because it is axiomatic.

I feel this is an important question because it has consequences which may be real consequences in the future if we do develop smart robots.

Turning off is defined as killing for computer and human for the purpose of this question, destructive deletion of the information which composed the human or computer.

  • 1
    At the heart of this lies an excellent question but from the first two paragraphs alone you need to define what is "turning off a human" (what does that mean? Killing them? Turning off a computer does not kill a computer); also, from what basis can you assert "consider if a computer becomes as smart as a human, you still intuitively know that turning off the computer is not a bad thing to do"? I think if a computer is as smart (and here I presume you mean sentient) as a human, it would be just as immoral to turn it off as it would be a human.
    – stoicfury
    Nov 16 '11 at 2:56
  • "I think if a computer is as smart (and here I presume you mean sentient) as a human, it would be just as immoral to turn it off as it would be a human." That is a very surprising remark. What are the consequences of a society of people who think in this way? I mean smart, I mean, appears to be the same as a human, which doesn't mean its "sentient"
    – Phil
    Nov 16 '11 at 3:18
  • 1
    If it's just a fancy chat bot, then of course it's not a problem to turn it off. But "fancy chat bots" are not equivalent to human beings, so the analogy is not appropriate.
    – stoicfury
    Nov 16 '11 at 4:16
  • The problem is you can't tell the difference between a fancy chat bot and a "sentient" computer.
    – Phil
    Nov 16 '11 at 6:51
  • Yes that is a problem. I addressed this in a comment below when you noted the same thing on my post. :)
    – stoicfury
    Nov 17 '11 at 2:57

Your analogy is not adequate!

As I mentioned in my comment above, what does it mean to "turn off a human"? It's hard to understand your analogy in the first two paragraphs because this notion is decidedly unclear. Turning off a computer means you are putting it in a state of rest; essentially of no activity. The closest equivalent for a human would be sleep, but if you wanted some hypothetical condition you could invoke some sort of "cryostasis" (cryonics/suspended animation). You should note, however, that computers are not currently ascribed any rights so you do not ask your computer if you want to turn it off, but you would certainly ask a human if s/he wanted to be put in cryostasis. But it's also not a bad thing to turn off a computer today because turning off a computer doesn't harm the computer (in fact in some ways it is beneficial). It is not the same way with humans. Although there are many cases in which I think cryostasis would be awesome (I'd totally do it if we had the technology today, just to wake up 500 years from now in the super high-tech future! ^_^), there are differences in that humans have bonds and connections to the real world which would be severed, causing great harm if done involuntarily. Your analogy, I'm afraid, is not adequate for what you are trying to convey.

Based on your updated definitions...

I mean smart, I mean, appears to be the same as a human, which doesn't mean its "sentient"

Unfortunately, the analogy becomes even less adequate, because a (merely) "smart" computer is not conscious. We don't prescribe moral protections to rocks and shrubs and spoons and fancy "SmarterBot" programs because they lack conscious awareness. And that's why it's appropriate to turn them off. It's not appropriate to "turn off" a human because we are conscious, not because we have something "special" inside us.

"Smart" vs. "Sentient"

Your write:

Then consider if a computer becomes as smart as a human, you still intuitively know that turning off the computer is not a bad thing to do and turning off a human is a bad thing to do.

Originally you wrote "smart" and I had hoped you meant "sentient", but you've indicated you mean merely knowing lots of data but not sentient. Since we do not afford unconscious entities moral protections, as mentioned above it does not get you anywhere to compare turning off (destroying) a computer and turning off (killing) a human being.

You write after:

What if the robot (computer) can do everything a human can do? What then? If you cannot tell the difference between the robot and the human then how would you feel about turning off the robot?

Well, provided it still wasn't sentient, there would be no problem with turning it off. It is obviously quite theoretically possible that one could create a robot that mimics human-like behavior such that it is indistinguishable from a human but lacks actual sentience. It get's a little hazy, however, when you start getting that complex because sentience is not really a well-defined concept in philosophy or computer science. What is "awareness"? My robot could have video camera eyes and microphone ears, thus allowing it to perceive the environment, but does that mean it's aware of the environment? Need it be aware of itself in relation to the environment? Need it be aware that it is aware? Etc.

Size matters not

You write:

We all have axioms of reasoning, but I believe many other people have this axiom as well.

I am sure many people believe this ...

This is not a good way to start an argument; it does not matter how many individuals believe in an idea, it doesn't mean it's any more true. Take any widely held myth, or any particular religion you want. If your logic was correct then Islam would be the most "true" of all religions, and I'm not sure you want to say that... :P

"So the question is, what is the soul/spirit"?

You are correct in assuming that there is a distinguishing characteristic between humans and man-made machines. We are not the same. But "souls" are not typically used to characterize this difference in modern philosophical debates and even less so in scientific ones. The problem with souls is that they're so hard to prove. It would be sweet to have a soul (honestly). I'd love to have one. But they're entirely unverifiable. Outside philosophy, souls are dismissed as pseudoscience because there is no evidence to support their existence. One needs not invoke the concept of a soul to explain any real problem facing us today. Within philosophy, they aren't really talked about much anymore because from the outset they lack any sort of philosophical justification. There are philosophers who currently are dualists (who would generally accept the notion of a soul), but their numbers are thinning...

The distinguishing characteristic you are looking for is simply

  • flesh vs steel/plastic
  • DNA vs. BIOS
  • Brain vs HDD (or SSD if you're a hipster)

If you are looking for a distinguishing non-physical characteristic, I think you'll be hard-pressed to find one, although it will depend on which philosophers you talk to. Any physicalist (i.e. non-dualist) is going to dismiss souls and assert that logically (computationally), a human brain is just like a computer. Sure, it's wired a bit differently, and the wires are made up of different materials, but it's functionally the same.

Onto this idea of "specialness"

I feel this is an important question because it has consequences which may be real consequences in the future if we do develop smart robots.

Yes, the future—our future—is going to be very, very interesting. ^_^ There will probably be sentient robots before we die. The only question is: are you going to shut off these poor computers like they are merely toasters, or are you going to afford them some respect so they don't go all iRobot on us? :P I say this jokingly, although there is a tinge of seriousness to it because I feel like the "axioms" you describe, this "specialness", are at the heart of the problems of the past (and now) with sexism and racism and fascism and all these "ism" problems; they always seem to come down to this same "We're special" idea. White people are special, let's enslave the black ones. Pure German people are special, let's kill everyone else. Men are special, let's not let women vote or to get paid as much. The institution of marriage between a man and a women is special, let's not let gay's be married. Maybe I'm just a weirdo, but I don't think us humans are all that special. It is quite natural to think as humans that we are special, that we are not mere "animals", but after really examining that notion I was at a loss to really justify it in any way. It dawned on me that really we are no more special than the lion in the savannah, whose fierce hunting skills and sharp claws are perfect for exactly what it does, or the salmon in the river or the eagle in the sky in much the same way. We are not intrinsically "above" them; in fact in many ways we are far below them. Henry Beston in The Outermost House wrote,

“In a world older and more complete than ours [animals] move finished and complete, gifted with extensions of the senses we have lost or never attained, living by voices we shall never hear. They are not brethren, they are not underlings; they are other nations, caught with ourselves in the net of life and time, fellow prisoners of the splendour and travail of the earth.”

I guess what I'm getting at here is that I would encourage you to investigate the real reasons why you have this idea that humans are "special"; take a look at where those axioms came from. I could be wrong but I don't think you'd suggest that you were "born" with such an axiom. At some point it was acquired. If you can figure out where it came from and why, perhaps you can try to analyze whether it's based on a valid belief.

Sorry I rambled on a bit; this is a common question I get because I'm interested in Artificial Intelligence, and thus why I tend to have a lot to say. I hope I answered your question adequately. :)

  • One problem here is you cannot tell the difference between sentient (conscious, aware) and "smart".
    – Phil
    Nov 16 '11 at 6:59
  • 2
    +1, Excellent answer. Regarding sentience, it might be worth pointing out that for some philosophical traditions (Buddhism, for example) "sentience" reduces to "the ability to suffer." Nov 16 '11 at 8:48
  • @Phil: Well you can in some cases (for me, in all cases today). I don't know about you, but those SmarterBot programs and Alice and all the others are pretty smart (they're programed to know things) but they can't fool me into thinking they are sentient. Theoretically, one day that won't be the case, and you are correct in that we won't be able to tell the difference. But that is no different than how it is today, with the problem of other minds. We'll just have to assume they are sentient, like everyone else we meet in the world. :)
    – stoicfury
    Nov 16 '11 at 23:11
  • What I mean is, a sufficiently smart computer/robot could appear to be as smart as us, you can't tell the difference in that case between a human and a robot from outward appearances. In that case, are you happy to delete the human or robot? What are the consequences?
    – Phil
    Nov 17 '11 at 2:37
  • @Phil: Yes, I addressed that in my answer. :) In short: no, I don't like "deleting" people and I wouldn't like deleting conscious robots either. The consequences are irrelevant; to me is a moral harm and I wouldn't do it regardless if there were consequences for my actions. Just like I don't kill people merely because there is a law that says I will go to jail if I do so...
    – stoicfury
    Nov 17 '11 at 2:46

To side step the concerns about where the soul lies, let me attempt to answer the second question about what the difference is between a human and a machine with respect to ethics.

A machine is an artificial device, constructed deliberately and based on scientific rules (by definition, otherwise it is magic). And so, whatever outward appearance a machine may have (simulating a human in every respect), it is ostensibly a repeatable process to create another identical machine with the identical characteristics and exactly repeatable behavior.

However, a human has to be born and tended to for years. There is no such thing as an exact replica of a human, no clone popping out of an artificial womb to look and act like an identical twin. There is no recreating an individual human. The effort is unobtainable.

This difference is why turning off a machine, even though looking and acting like a human, somehow doesn't feel of the same ethical 'badness' as turning off (killing?) a human.

Then there's also the more petty 'humans are one of us and machines aren't'.

  • But his argument is suggesting that the robots act like humans in all ways such that their behavior is indistinguishable from humans. Presumably that means it learns and grows on it's own based on the experiences it has in life; i.e. it is no more difficult for two humans to be created alike than it would be for two of the robots he mentions. Even if this weren't the case, theoretically it would still be possible to create two identical human beings (insofar as you could create two identical robots). It would be much, much harder, but possible in theory, and that's all we would need.
    – stoicfury
    Nov 17 '11 at 2:53
  • @stoicfury: I think the 'theoretically' possible is a bit too ethereal. Today, what do you think can actually happen? You could reproduce a robot/AI talking program, but not a person. Technologically it is extremely distant to expect any kind of creation of exact human copies.
    – Mitch
    Nov 17 '11 at 14:19
  • What matters here is whether humans are capable of being duplicated — if not, they are special, because I agree robots can be. In theory, if we live in a physical universe there is nothing preventing humans from being duplicated except technology. You can't argue that there's something special about humans—oh, we can't be duplicated!—and then 400 years later when we develop the technology say "hmmm, just kidding..." I think the idea of this "specialness" is to find something that is intrinsic to human nature, not merely a byproduct of our poor technology when it comes to biology... :P
    – stoicfury
    Nov 18 '11 at 3:13
  • @stoicfury: I don't think I'm trying to point out the essential essence of what makes someone human (or something a machine). I'm just point out the easiest to identify differentiating feature.
    – Mitch
    Nov 18 '11 at 3:36


Setting aside the "adequacy" of your question, I think the distinction lies in the concept of "consciousness" and "thought". Humans have it and machines do not. In fact, I will go further and say that if we create a machine that truly has consciousness then it is no longer a machine, but a being. In addition, you might add "free will" to the criteria defining "living beings/humans" vs machines.

However, if you are a strict "materialist" then we are all machines anyway and consciousness and free will are illusions. The argument goes that the mind and the brain (body) are the same. There is no "mind" hovering inside your brain controlling the neurons. The neurons are you and nothing else is you. Therefore, all the natural laws apply to the neurons and the mind has no choice but to act in the way it does because of its configuration and the environment. We have no real free will and are therefore no different to machines.

  • The problem is in the first idea you can't tell if the machine is "conscious" since we don't know how to define it. In the other idea we de-value humans in a way that I don't want to occur, if you see humans as machines only then anything inhumane really doesn't matter.
    – Phil
    Nov 21 '11 at 14:18
  • I agree. Defining consciousness is very hard, but I think that is the root of the distinction between man and machine. As for materialism, I too don't like the logical conclusion that we are all just machines, but not liking something doesn't make it false.
    – andrewfd
    Nov 21 '11 at 20:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.