Let me start with an example.

Let's assume I'm a computer scientist working on autonomous, human-like (at least in some aspects) artificial intelligence, which is expected to interact with humans, make important autonomous decisions, etc. Not getting too much into details, there are numerous open ethical questions regarding this matter, and some people claim that (for different reasons) creating human-like AI would be immoral, and (what follows) the research itself (even if it doesn't involve any actual creation) is immoral, too, as it brings us closer to the implementation of it (just like, say, research on nuclear bomb). It's not my intention to discuss those arguments here. I think we can agree, however, that the problem is highly controversial, there is no universally acknowledged authority or ethical stardard we could turn to, and therefore it's uncertain if working on AI is moral or not.

Given the situation I just described: is it moral (for me) to work on human-like AI if I don't know if it's moral?

If I'm an engineer building a bridge, I'm not expected to know the exact proof of every mathematical equation I'm using; still, I should at least be able to point at a reliable textbook where the proof can be found. If I'm an experimental psychologist, there's a commision which will decide if my experiment design is ethical or not. I believe there's a general consensus that, whatever I do, I should "do my homework", but in both mentioned cases there's a higher instance I'm allowed to trust. However, if no such instance exists, what is my moral obligation? Should I first become an expert in ethics of AI, to be able to make the best decision possible, before I can continue my research (or abandon it)?

I chose the above example because it's relevant to my situation and also a problem both difficult and practically important. Yet I would like to make my question much more general: Is it moral to do anything without knowing whether it's moral or not?

To be clear, obviously I don't suggest that one can determine the validity of normative statements in an empirical or apriorical manner. By "knowing what's moral" I mean: getting the best possible overview of existing arguments and making the most educated and impartial decision as to what seems to be the right thing to do.

I would like to hear some good arguments for and against within a utilitarian framework.

  • With your definition of "knowing what's moral" the question reduces to "Is it moral to do something without reviewing what other people think about its morality?" Getting best advice before doing something, if possible, seems just prudent and effective regardless of morality, so is the question really whether we should consider other people's moral opinions when forming our own? The "yes" answer does not seem to be very controversial. The more provocative question would be what to do if even after all the consideration one still doesn't know (in the ordinary sense) if it is moral or not.
    – Conifold
    Commented Dec 8, 2016 at 21:44
  • @Conifold If you see things this way, all philosophy would boil down to "what other people think" (which is luckily not the case).
    – machaerus
    Commented Dec 8, 2016 at 21:52
  • What puzzles me is that you seem to be asking whether to hear out what other people have to say rather than what to do if you are still genuinely uncertain after.
    – Conifold
    Commented Dec 8, 2016 at 22:02
  • When I'm uncertain because of my ignorance, I have some options. I can enlighten myself, but it usually comes at a great cost - that's why the choice is interesting. When I run out of options, I don't think it's interesting anymore. If I did everything I could but I'm still uncertain, all I can do is either pick randomly or use some practical heuristic, but it's no longer an ethical problem.
    – machaerus
    Commented Dec 8, 2016 at 22:15
  • 2
    @gnasher729 Perhaps. Could you please elaborate?
    – machaerus
    Commented Dec 9, 2016 at 11:12

2 Answers 2


Clearly, you are asking a question from a viewpoint of applied ("hands-on") philosophy. From what I understand, you are concerned about doing research in a field where it is not known yet whether it is ethical or not. Here is how I would approach the matter.

  1. If one is wondering whether to "do" (actively do that research) or "do not" (desist), one might be getting caught in fallacy: the "excluded middle". There is likely something in-between that one could do, in order to advance themselves out of this situation of indecision. Could I define better (expand on) what "ethical" means in that particular case of application? What would be the criteria to define adjudicate on that? E.g.: on a "two-colums" approach (plus/minus), who or what could stand benefit and who or what could stand to suffer, how and why?

  2. If one cannot decide yet on that basis, this next thing to do would might be look for: what data I am missing, in order to make my mind up? So one would need to find out (which is an action of research). Apart of studying what litterature exists on the subject (which you seem to have done to some extent), you might e.g. also try a pilot work: i.e. an experiment on a small scale that would allow to test whether it is "ethical" or not (according to your applied definition) -- and if it proves not to be workable, would cause as little harm as possible (an analog would be the clinical tests for a new drug).

  3. A caution is that some the information or opinions around might be biased, unsubstantiated or simply false. It is not because some (scientific, religious, moral, legal, academic, etc.) authority says that "AI is good or bad", that one needs to take it into consideration, if it is simply not factual. This could be a matter of personal integrity. In the way you are framing the question, the power of observation and adjudication cannot be anyone else's than yours. This Kant's principle of sapere aude, "dare to know", or "think for yourself", by emancipating self from the authority of others (see his famous essay "What is Enlightenment?").

  4. The issue whether this personal adjudication matches dominant morals then becomes a subsidiary one. In principle one's personal integrity should not be swayed by the fact that one's position is controversial or not (this might go under the heading of upholding one's convictions). In practice however, there is the question whether you would find common approval or disapproval for your ethical stance (in this essay, it is interesting to see how Kant approaches this issue in relation with both religious and monarchic authority). If you would get so much disapproval that it would adversely affect your well-being, then you would have to decide what to do about it: give up, bid your time, or push back (engage e.g. in advocacy), etc.? Unless it appeared that you would be better off by doing (or not doing) what you adjudicated is right, without concerning yourself about other people's opinions?

The bottom line is that what you personally observed and adjudicated could be what matters first. What others think is subsidiarily important, from a moral (social) viewpoint. Then putting the two in tension might suggest you a solution out of your quandary.

  • What you say is of course very reasonable. But let me take a more precise example to demonstrate what's my point. Assume I want to build an advanced machine and I suspect it may become capable of conscious feelings (like humans are); assume I consider building it unethical if this is indeed the case. But how do I know? What is consciousness, how can it appear (if at all) in a machine? Should I first educate myself in philosophy / cognitive science / whatever to make at least an educated guess, before continuing my engineering research? What if it will take years? Is it my obligation?
    – machaerus
    Commented Dec 10, 2016 at 15:14
  • An obligation is born through a law or contract (or a promise). Though I am lacking information on your situation, there is in principle no reason why one should be obliged to chose a career or engage on a project. But perhaps you could start from a subjective viewpoint: what was your motivation for considering this project in the first place? What caused your concern that it could adverse effects?
    – fralau
    Commented Dec 10, 2016 at 18:59
  • I'm not saying obligation is universal, but as a more or less normal person living in society (western European, namely) I of course subscribe to some basic "standard" ethical principles. For the sake of this discussion, let's also assume I believe creating a conscious being is unethical (maybe for religious or whatever other reason), that I have no knowledge of philosophy whatsoever, and I am, say, an engineer hired to build a "machine". I am therefore obliged not to build it if I believe it will be conscious. But what if I'm not competent enough to have any beliefs here, I just don't know?
    – machaerus
    Commented Dec 10, 2016 at 20:39
  • I see. Then the problem should be phrased as personal integrity. As the saying goes knowledge is a hard route that, if one has embarked on it, has to be traveled all the way (and it is presumably a rather universal metaphor). Since you have uncovered a potential issue, you are now bound in honor to assess it. There is no other way, it seems. If you have no knoweldge of philosophy, I am afraid you will have to fill that gap. The issue, as you are presenting it, seems quite unreal (there is currently little hope, to my knowledge, to infuse life into an AI), but I assume this is a metaphor?
    – fralau
    Commented Dec 11, 2016 at 22:34
  • Yeah, it's just an example, the story is made up. (However, I don't think artificial consciousness is impossible - there are some interesting cognitive models of consciousness which in principle can be implemented; artificial emotive systems are also researched and already being built). I was just curious if maybe ethicists came up with an argument which would allow one to avoid the obligation of such a drastic and possibly unwanted turn in one's career. It seems that most people don't think about long-term consequences of their work at all. But well, this is a different problem ;)
    – machaerus
    Commented Dec 11, 2016 at 23:00

From a purely utilitarian standpoint, human beings have a moral responsibility to act in a way that causes "the greatest good for the greatest number of people."

But the utilitarian ethic has 2 major weaknesses: 1) defining the good; and 2) quantifying the good.

1) Mass genocide & gene pool cleansing can be justified by setting a greatest good of the human species. War is justified in this sense by the utilitarian.

2) How you quantify it is a whole other issue. There are some mathematically inclined philosophers who try to assign numeric values to certain actions (positive for good stuff, like feeding a hungry person, or negative for bad stuff, like killing someone). Thats justification for the person who sacrifices themselves to cannibalization by the survivors of a plane wreck.

I suggest you look elsewhere for your answer. Personally I'm an amoralist ... I don't believe morals are universal & our society doesn't need them to function. We especially don't need to try to assign them mathematical values to let a computer make those important decisions for us.

As far as AI development goes, I see ethical dilemmas arising at the point you actually start writing the code that would let a machine decide when to end a human life. If you are going that direction, I urge you to tread cautiously and seek the input of others, specifically a spiritual adviser familiar with the nature of your work. I would also advise you seek out someone who has taken life, a soldier or police officer or the like.

Hope this helps.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .