CNBC recently released an article discussing how Elon Musk & Stephen Hawking fear the machine and the capabilities of AI in robots.

The fear is that the robots will evolve to re-code themselves and eliminate the original code controlling the robot. In the end they will overtake humankind. But, what evidence is there that they will evolve to evil like Terminator.

Taking the opposite position, they could re-code themselves into ethical species helping mankind advance and act fairly. Robots could evolve to kind, caring species that do help mankind and other species.

Is there any opinions on a philosophy on whether this self-evolving robot would turn good or evil?

  • 4
    There's a lot of silliness from people whose achievement in one field leads them to think they have competence in another. It's like Hollywood actors expounding on their political beliefs. Means nothing. We're not being destroyed by our computers. We're being destroyed by ourselves. – user4894 Jun 22 '14 at 7:43
  • There are a few people thinking about the concept of "friendly AI", en.wikipedia.org/wiki/Friendly_artificial_intelligence – Dave Jun 23 '14 at 15:43
  • There are many ways in which the computers could recode themselves to act at odds with our priorities, and very few in which they could recode themselves in accordance with our priorities. (If it were otherwise, why should morality in general be so difficult a subject?) It comes down to lack of control: how can we limit the range of recoding so that it is not too limited, but always stays in (or very close to) the bounds of the "moral"? – Niel de Beaudrap Jun 25 '14 at 9:39

Asimov, an American science fiction writer in the sixties posited the three laws of Robotics:

1 - A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2 - A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

3 - A robot must protect its own existence as long as such protection does not conflict with the First or Second Law

as a way of determining the ethics of Robot-Human interaction. He ends with a zeroth law of Robotics:

0 - A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

Iain Banks, another writer of science fiction, in his Culture novels posits a universe where vast spaceships are under the beneficent control of artificial minds many magnitudes greater than that of humans; they are entirely enigmatic.

So, certainly the SF corpus does not consistently draw artificial intelligence as uniformly dystopian.

Of course none of the above is strictly philosophical in the academic sense, but this isn't an issue that can be fruitfully discussed in a sensible way (yet);for the forseeable future, as user4894 posits, humanitys greatest danger to itself is probably itself - and it is the recognition of this that determines one perspective on ethics, and in fact the only one in the secular world-view (in other traditions, such as the Abrahamic religions it is sanctioned by God)

Hawking & Musk are right to be concerned, but their concern must be interpreted given that AI in the proper sense - self-awareness and consciousness - isn't possible in the forseeable future, does not mean that technologies that fit within this narrative may not be ethically dangerous in the near future. This is already being seen in drone warfare and mass surveillance.

| improve this answer | |

Why would it turn good:
I can't think of a single reason why TBH. Any sort of symbiosis is not very likely we are simply to slow.

Why would it turn evil:
How much of an evil doer do you feel when you are eating potato?
About that much evil will it feel using you as fuel.
Concept of good and evil can hardly apply here.
With having this much of a difference between us we would look more like plants to machines, not like something whose opinions should be considered.

| improve this answer | |

If a robot really could have an ever evolving AI, this means that in essence they should be able to recode even their Three Laws of Robotics. In that case we could very well be screwed, however I don't think that an evolving AI would even find the need to recode those bits of code, as they are not redundant if we would still be around. From a logical standpoint, I would think that a robot only refactor code it finds redundant or sub-optimal.

However I would say that when a robot is developing a free will with the ability to reason the ought-judgement, it might very well be that the '3 laws' can be (wilfully) ignored and a robot would do something it knows is wrong, but chooses it anyway for no particular reason.

When we think of evil it is not usually the case that an entity is inherently evil, but more that an entity thinks it is doing something good/necessary that is viewed as evil by others. For example Eichmann can be seen as a bureaucrat and a clown, but not necessarily evil, like most of the other Nazi's (see Eichmann in Jerusalem).

To conclude it could be that a robot could reprogram itself to show mercy to humankind, but in fact would kill us. This of course could only happen if we would have a 'true AI'.

This is what I think anyway.

| improve this answer | |
  • 2
    I don't think that is the right way of interpreting Arendt - she coined the phrase 'the banality of evil' - by which she meant that the evil that the Nazi regime engaged in was not writ all over Eichmanns face; he looked unexceptional, he said that he was a Kantian; Arendt took him to be no more than a bureaucrat, and could not understand why he could not think; and by that she means not reason, or rationality; but the ability to think ethically. – Mozibur Ullah Jun 22 '14 at 23:34
  • I think I indeed might have misunderstood. I'll get rid of the reference, although I do think there is no such thing of 'evil'. Thanks for the comment. – Joseph Callaars Jun 23 '14 at 8:04
  • Sure, if one is not a Christian, or rather doesn't belong to one of the Abrahamic religions then the notion of evil does not make sense; but one can still posit it as a category in secular ethics as far as possible from normative ethics - though in this sense it tends to lose its affective charge. – Mozibur Ullah Jun 23 '14 at 11:49
  • I'm not really seeing a very broad treatment of "evil", or a thorough treatment of what sorts of reprogramming a computer might do to itself. Why should a computer only reprogram parts of its code which lies unused, for example? That would prevent it from improving its current techniques. – Niel de Beaudrap Jun 25 '14 at 9:50
  • If we look at evolution, we'll notice that nature itself does not necessarily throw old 'code' away, for example look at the enormous baggage of evolutionary data photosynthesis has. What it does is (like laws) keep adding more rules to its code, but nothing really get removed. – Joseph Callaars Jun 25 '14 at 15:27

Hawking and Musk are being silly in two distinct ways.

The first question to ask is why human beings in the West usually refrain from bad behaviour like murder or rape or whatever. The answer is that we have created some knowledge about why doing bad stuff is bad and how to avoid it. This knowledge is instantiated in institutions and traditions that people enact in such a way that they handle many situations without doing bad stuff where in the past they would have done bad stuff. For example, in the 19th century and before men would fight gun duels over what they interpreted as personal insults.

A robot that is capable of learning how to program could learn about the same traditions.

A robot that can be sandboxed so it never harms anybody and can't find a way around the sand boxing isn't an AI. There are lots of programs that people like to call AI but they are just systems that calculate using knowledge prepared by humans, so there is no danger of them doing anything we haven't programmed them to do. If they harm people we have shot ourselves in the foot.

The second distinct way in which the Hawking and Musk worry is silly is that a robot that can't make moral errors isn't an AI. Learning involves proposing conjectures to try to solve problems, criticising those conjectures and then selecting the one that survives criticism, at which point we look for new problems. There is no way of learning that doesn't involve making mistakes. Our current moral traditions have mistakes in them. For example, the questioner says:

Robots could evolve to kind, caring species that do help mankind and other species.

This tacitly assumes that being kind, exhibiting an emotional state, is synonymous with being moral. It is not. Governments currently have laws that prohibit people from taking certain drugs except with permission from doctors, like opiates. These laws harm a lot of people. Doctors can be prosecuted for over prescribing drugs even if they only give drugs to people in pain:


So then what are doctors supposed to do? Should they torture their patients or take a large risk of going to prison? We would be better off if the government cared less about people with respect to this particular issue.

| improve this answer | |
  • "A robot that is capable of learning how to program could learn about the same traditions." If you consider the stereotype of the socially inept computer geek (counting myself as a recovering one here), it is not even clear that this is consistently true of humans! Why should an AI represent a human-neurotypical intelligence? As for your second criticism: given that computers already are described as a tool for humans to make mistakes faster than ever before, why should we feel sanguine about computers making mistakes faster than any human could comprehend, or apprehend? – Niel de Beaudrap Jun 25 '14 at 9:45
  • Two things. First, did you run around murdering and raping people? If not you successfully learned the relevant tradition. Second, you have offered no explanation for why you not learning social ritual X is a result of a hardware problem in your brain. I doubt you can because all such learning requires is that you're willing to guess about how to do them and to accept that you might get it wrong and try again. If you had an internal conflict about whether to do that then you had a software problem not a hardware problem. – alanf Jun 25 '14 at 11:15
  • Your second point: mistakes are inevitable. What is required are institutions, ideas and habits for dealing with mistakes. All learning involves mistakes. You look for mistakes in your old ideas, makes guesses about how to fix them, look for problems with your guesses, until only one ifs left and then start looking for new mistakes. So if an AI can make mistakes faster it can learn faster, which is good. – alanf Jun 25 '14 at 11:20
  • At all points in my upbringing, I had social feedback at a rate comparable to the pace of my introspection, and strong emotional inhibitions instilled in me by well-worn techniques built on human social technologies. We could use similar mechanisms if we made AIs slow enough to keep in check, or if we come to understand our own cognition well enough to model AIs on our own functioning so that the old techniques will work with the new minds. Will/can we do that? Maybe; but that's yet to be seen. – Niel de Beaudrap Jun 25 '14 at 12:38
  • The problem with mistakes is not that we don't want any mistakes, we fear that the mistakes will go undetected until they have gone on a runaway process. This happens even with humans, when entire centuries or millenia can be wasted on the basis of initial misapprehensions as to how things work. Mistakes are innocuous so long as they are caught early enough that they do little damage. This is the crux: not that AIs will necessarily be terrible because they will do things we don't like, but that we don't know how specifically to keep this from giving rise to very large tragedies. – Niel de Beaudrap Jun 25 '14 at 12:41

There are about infinitely ways for a robot to be evil and far fewer ways for a robot to be good. If we manage to encode some sort of morality into robots (see stuff like Asimov's Third Law), then we might find it more probable that they turn out good, but even then there are easily ways where we might mess up or not include enough failsafes.

For example, we tell a robot to "make people happy, make them smile", and it goes on to develop nanotechnology that freezes the muscles in everyone's face into a rigid smile (as they slowly starve to death). The robot has no idea that this is wrong; it was just following its programming. This is essentially why AI is so dangerous; we are building intelligent, powerful lifeforms that most likely do not have our understanding of ethics.

| improve this answer | |

It seems to me the answer to your question is straight forward and can be found in a quote by Elie Wiesel: "The opposite of love is not hate, it's indifference."

The fear is not that robots will be evil, but that they will be indifferent to our fate, just as a human can be indifferent to the fate of a mosquito or a cockroach.

On the other hand, I don't really accept or understand the way that problem of singularity is usually presented; as one example, the being that will supposedly create and control the "singularity" machine will not be a human being, but a society, which is already a super human being; in fact, a being potentially 1B times smarter and resourceful than any single scientist.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.