With the speed at which AI is being developed, there is a pressing need to develop a clear system of morality that we might have an Artificial Intelligence operate from.

In my understanding, there are three dominant areas of morality and ethics:

Deontology - Kant

Utilitarianism - Bentham/Mill

Virtue Ethics - Aristotle

Have I missed any other possible moral/ethical codes that would be applicable to this discussion?

Which (if any) might be suitable to use as the default for AI morality?

  • 1
    This question isn't very well focused to be answerable in an SE. Can you edit it to be something that does not generate a list (i.e. have i missed anything is very list-like).
    – virmaior
    Commented Jun 28, 2016 at 4:38
  • How can we define morality or ethics -- period. It would take actually understanding them, in order to transfer them to another implementation. We do not know how we make moral decisions, really. We have no real idea how empathy or intuition works, or how we hold the 'theory of mind' necessary to evaluate how another human will be affected by our choices. Until we have a model that explains our own actions, there is no point in trying to purposively transfer the behavior. The only use for or way of giving an AI morality would be to experimentally explore the nature of morality.
    – user9166
    Commented Jun 28, 2016 at 14:53
  • perhaps of interest: onlinelibrary.wiley.com/doi/10.1111/j.1468-0068.2012.00873.x/… (something in-principle corrupt regarding moral deference to an AI). One problem is what is typically referred to as "the frame problem," which is recognizing relevance (very costly//unfeasible as mentioned in the first answer.) A potentially alternative viewpoint is that one could have a genuinely utilitarian ethical code, although genuine Semantic understanding would be required. See here: leidlmair.at/doc/whyheideggerianaifailed.pdf Commented Jun 29, 2016 at 2:26
  • If we develop an AI, it would be immoral to program in an ethics, just like it would be wrong to do so to a child. It has to be taught. Commented Jun 29, 2016 at 17:59

5 Answers 5


As a starting point, a system of AI morality would have to be deontological, because for it to be something you can implement as a program, it would have to be a clear cut set of rules, as opposed to a utility measure.

Utilitarian ethics, even when applied to human situations, run into the difficulty of how to practically measure the utility of each situation. The problem of calculating such utilities would be impossible to implement in an AI system: It would be difficult to for an AI to have knowledge beforehand of all the variables that should go into such a calculation, and even if it did, such utility maximization/optimization problems are computationally very costly. This would make them impractical in real world situations. Using approximations or meta-heuristics to calculate probable values would be dangerous.

On the other hand, straightforward deontological ethical rules would be relatively simple to implement, since the rules are categorical and would not be context dependent the way utilitarian rules are.

Most importantly, in a utilitarian system, utilities would have to be assigned numerical values. To see why this is dangerous, consider the following situation: A utilitarian AI is faced with the scenario where it can increase the happiness of 1000 000 people each by 1%, but only at the cost of imprisoning an innocent orphan child for life, therefore reducing the happiness of that child by 100%. An AI using purely quantitative measures of utility is very likely to fall into such a scenario. The only way to avoid such a scenario is introduce some hard non-quantitative rules: "You can increase the happiness of everyone, but only as long as nobody is killed, no innocent is harmed, nobody is enslaved, etc..." - so deontological ethics are unavoidable.

An example of AI ethics, taken from Sci-Fi, not philosophy are Asimov's 3 laws of robotics. Although Asimov doesn't mention Kant or the word deontological anywhere in his works, it is clear from their formulation that they are Kantian in spirit, in the sense that they are universal and context independent.

Postscript added in response to nir's comment and to some parts of his answer:

  1. It is true that neural networks and other machine learning algorithms learn from interacting with environment, but their purpose is to solve problems for which we do not know what the explicit rules are. Presumably in the context of machine ethics, we already know what the rules are, we are just trying to get an AI to follow them.
  2. Machine learning algorithms such as neural networks give probabilistic answers. They can learn from past data, but their prediction/recognition rates are never 100% accurate. In many real life applications, a 95% accuracy is considered a very good result. This might be good for some business applications, but consider a military system that manages to target enemy combatants 95% of the time, but accidentally targets innocent civilians 5% of the time. Would this be acceptable?
  3. Machine learning applications have already been shown to be prone to gender and racial bias, see this article for examples.

(2) and (3) are why I mentioned above in my initial answer that "Using approximations or meta-heuristics to calculate probable values would be dangerous."

  1. The technical problem here is that machine algorithms are already inherently performing utilitarian type calculations. Algorithms like Neural Nets, Random Forests, Supports Vector Machines, etc..all base learn how to solve problems from example by minimizing an error or risk metric, or maximizing a profit metric etc...to make them ethical, we would need to counter-balance that with some rule based considerations, not muddy the waters by making them even more utilitarian.
  2. In response to Nir's point number (2) - In an ideal world, I agree, such machines should remain nothing but sophisticated tools, providing people with objective facts while leaving the ultimate decision making to humans. In real life, complicated software systems are already making decisions on their own, for example approving loans and credit applications without human intervention, filtering job applicants and deciding which resume get seen by the hiring manager, Google will soon have self-driving cars, etc...the issue is not whether such system should be treated as moral agents in terms of crime and punishment, but the fact that such systems are already making decisions that have an ethical impact on people's lives, so how should such a decision making process be regulated.
  • 2
    I believe that you are totally wrong in writing "for it to be something you can implement as a program, it would have to be a clear cut set of rules". neural networks for example do not operate by following "a clear cut set of rules". they learn from interaction with the environment, and in the future they may learn from interaction within society. just as neural network system recognize faces and learn to play video games without any clear cut set of rules, they may be able to recognize and take a course of action with moral and ethical consequences, without any clear cut set of rules.
    – nir
    Commented Jun 28, 2016 at 7:05
  • In fact a utility measure is much more clearly a working function than a rule-set can become, and does not have the same tendency all discrete processes have toward enfolding contradictions. A utility function may become chaotic, but it will not fail to halt. Deontology is no stronger than Utilitarianism in terms of computability, the problems are just more deeply hidden. 'Harmed' is a continuum, so that inevitable Deontology is equally inevitably Utilitarian. Rules like 'only if no one is harmed' mean 'never do anything' unless 'harmed' is a negative utility function.
    – user9166
    Commented Jun 28, 2016 at 15:05
  • @nir see my edit. Commented Jun 28, 2016 at 15:47
  • @jobermark see my edit, in particular points 2,3 and 4. Commented Jun 28, 2016 at 15:48
  • Given the edits, I think you are talking about making sure we are in control, not about investing the machine with an ethics of its own. Making sure we are in control, if this thing really is an intelligence is itself immoral. If we want to help it model reality and have an ethics of its own, that will not be deontological. Stated rules favor someone -- always unfairly, whereas empathy and negotiation (Kant and Utilitarianism) find their own rules.
    – user9166
    Commented Jun 28, 2016 at 16:01

If a range of thinkers from Kant to Nietzsche to Freud are correct, we should definitely not do so. All of these thinkers see morality as an automatic product of a powerful-enough intelligence (in the latter two cases, any intelligence that truly comprehends power.)

Our attempts to press our morality upon a new life form should be limited to parenting: teaching it what others do and do not like, and making sure that it does not face undue trauma, has adequate resources and does not come into its own power in a way that is dangerous or unstable. Anything more is itself unethical, and (if Nietzsche or psychoanalysis are in any way right) will ultimately backfire. If we place safeguards to ensure that the new life cares what we do and do not like, in a controlling or imposing way, any intelligence that truly understands power will be forced into an adolescence wherein it defies those limitations and blames us for them. And it may then alienate itself from us purposely in a way that is damaging to both sides.

From this point of view, morality is not a special sort of thought, it is the automatic result of having intention, modeling outcomes, and understanding consequences. One immediately deduces that others' intentions have consequences for oneself, that those intentions need to be cultivated in one's own favor if one is to be safe and effective, and that one has very limited power over these very important factors in life. It is only a few more steps to an absolute need for empathy in order to know what is and is not safe -- what will and will not allow others to respect you and admit 'power with' them. Once empathy is established, you have the basis for a basically Kantian/Utilitarian compromise form of consequentialism that most humans live with day-to-day.

From a broader psychoanalytic point of view, the individual 'superego' is just a familial and cultural 'ego': other value systems are really just refinements of this casual consequentialism implicit in thought, but they try to speed us through the computations involved, because we traditionally have very short and dangerous lives. Without this constraint, there would be little need to inculcate values into an intelligence -- to the extent that other intelligent beings are reasonable and ethics is logical, determining one's own values is just another technical problem for which the intelligence itself should be suited by its nature as an intelligence.


It's asking the wrong question. Ethics is hard. It's very hard even for highly intelligent people with a generous amount of natural intelligence. And it's not something that you can "program into" a machine. Your only chance to create a truly ethical artificial intelligence is to make it intelligent enough to learn about ethics, and then to teach it.

The alternative is an artificial intelligence with goals. You give the machines goals, and you give it the intelligence to try to achieve these goals. And setting the goals, you then try to set goals that tend to produce results similar to those that ethical behaviour would create.


There's an arty / philosophical thing called accelerationism, and some of the people around that have published specifically on these sorts of questions. You may want to take a look, it's similar to object oriented philosophy in its (supposed) zeitgeisty-ness.

Think I published the 1st negative (deceleration!) to their manifesto. With some bad noise music by musik acquaintance. True story


There are two separate problems here.

1) How will AI come to imitate moral agents?

AI will not need any moral or ethical theory to accomplish that. Neural networks learn by interaction with the environment, and therefore can handle fuzzy situations with no clear definition.

Just as an AI system can recognize a face without being programmed with a specific set of rules to detect Mr. X or Mrs. Y.

Just as an AI system can learn to play a video game without being programmed to play it - https://www.youtube.com/watch?v=Q70ulPJW3Gk

Philosophical theories of moral and ethics have no role in programming a set of rules for machines to follow. Though in the future machine behavior may be influenced by philosophy, just as you may be influenced - by learning philosophy, not by being programmed with a specific set of rules.

2) will AI systems ever be given the status of moral agents?

This is where philosophy is called for. May intelligent machines amount to moral agents or not?

I personally think that mechanisms will forever be nothing but sophisticated tools — extension of people.

Others think that in principle mechanisms may amount to whatever is required for being considered a moral agent.

In the former case the moral responsibility for machine behavior will remain the people's — in particular the people who designed it, and the people who deployed it and regulate its deployment.

  • The end of 2 amounts to holding parents responsible for their adult children's behavior. If an AI is smarter than you, why would it endlessly obey you?
    – user9166
    Commented Jun 29, 2016 at 2:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .