Within a utilitarian framework, how should I handle research where I don't know whether it is ethical?

Question

Let me start with an example.

Let's assume I'm a computer scientist working on autonomous, human-like (at least in some aspects) artificial intelligence, which is expected to interact with humans, make important autonomous decisions, etc. Not getting too much into details, there are numerous open ethical questions regarding this matter, and some people claim that (for different reasons) creating human-like AI would be immoral, and (what follows) the research itself (even if it doesn't involve any actual creation) is immoral, too, as it brings us closer to the implementation of it (just like, say, research on nuclear bomb). It's not my intention to discuss those arguments here. I think we can agree, however, that the problem is highly controversial, there is no universally acknowledged authority or ethical stardard we could turn to, and therefore it's uncertain if working on AI is moral or not.

Given the situation I just described: is it moral (for me) to work on human-like AI if I don't know if it's moral?

If I'm an engineer building a bridge, I'm not expected to know the exact proof of every mathematical equation I'm using; still, I should at least be able to point at a reliable textbook where the proof can be found. If I'm an experimental psychologist, there's a commision which will decide if my experiment design is ethical or not. I believe there's a general consensus that, whatever I do, I should "do my homework", but in both mentioned cases there's a higher instance I'm allowed to trust. However, if no such instance exists, what is my moral obligation? Should I first become an expert in ethics of AI, to be able to make the best decision possible, before I can continue my research (or abandon it)?

---

I chose the above example because it's relevant to my situation and also a problem both difficult and practically important. Yet I would like to make my question much more general: Is it moral to do anything without knowing whether it's moral or not?

To be clear, obviously I don't suggest that one can determine the validity of normative statements in an empirical or apriorical manner. By "knowing what's moral" I mean: getting the best possible overview of existing arguments and making the most educated and impartial decision as to what seems to be the right thing to do.

I would like to hear some good arguments for and against within a utilitarian framework.

Answer 1

Clearly, you are asking a question from a viewpoint of applied ("hands-on") philosophy. From what I understand, you are concerned about doing research in a field where it is not known yet whether it is ethical or not. Here is how I would approach the matter.

1. If one is wondering whether to "do" (actively do that research) or "do not" (desist), one might be getting caught in fallacy: the "excluded middle". There is likely something in-between that one could do, in order to advance themselves out of this situation of indecision. Could I define better (expand on) what "ethical" means in that particular case of application? What would be the criteria to define adjudicate on that? E.g.: on a "two-colums" approach (plus/minus), who or what could stand benefit and who or what could stand to suffer, how and why?

2. If one cannot decide yet on that basis, this next thing to do would might be look for: what data I am missing, in order to make my mind up? So one would need to find out (which is an action of research). Apart of studying what litterature exists on the subject (which you seem to have done to some extent), you might e.g. also try a pilot work: i.e. an experiment on a small scale that would allow to test whether it is "ethical" or not (according to your applied definition) -- and if it proves not to be workable, would cause as little harm as possible (an analog would be the clinical tests for a new drug).

3. A caution is that some the information or opinions around might be biased, unsubstantiated or simply false. It is not because some (scientific, religious, moral, legal, academic, etc.) authority says that "AI is good or bad", that one needs to take it into consideration, if it is simply not factual. This could be a matter of personal integrity. In the way you are framing the question, the power of observation and adjudication cannot be anyone else's than yours. This Kant's principle of sapere aude, "dare to know", or "think for yourself", by emancipating self from the authority of others (see his famous essay "What is Enlightenment?").

4. The issue whether this personal adjudication matches dominant morals then becomes a subsidiary one. In principle one's personal integrity should not be swayed by the fact that one's position is controversial or not (this might go under the heading of upholding one's convictions). In practice however, there is the question whether you would find common approval or disapproval for your ethical stance (in this essay, it is interesting to see how Kant approaches this issue in relation with both religious and monarchic authority). If you would get so much disapproval that it would adversely affect your well-being, then you would have to decide what to do about it: give up, bid your time, or push back (engage e.g. in advocacy), etc.? Unless it appeared that you would be better off by doing (or not doing) what you adjudicated is right, without concerning yourself about other people's opinions?

The bottom line is that what you personally observed and adjudicated could be what matters first. What others think is subsidiarily important, from a moral (social) viewpoint. Then putting the two in tension might suggest you a solution out of your quandary.

Answer 2

From a purely utilitarian standpoint, human beings have a moral responsibility to act in a way that causes "the greatest good for the greatest number of people."

But the utilitarian ethic has 2 major weaknesses: 1) defining the good; and 2) quantifying the good.

1) Mass genocide & gene pool cleansing can be justified by setting a greatest good of the human species. War is justified in this sense by the utilitarian.

2) How you quantify it is a whole other issue. There are some mathematically inclined philosophers who try to assign numeric values to certain actions (positive for good stuff, like feeding a hungry person, or negative for bad stuff, like killing someone). Thats justification for the person who sacrifices themselves to cannibalization by the survivors of a plane wreck.

I suggest you look elsewhere for your answer. Personally I'm an amoralist ... I don't believe morals are universal & our society doesn't need them to function. We especially don't need to try to assign them mathematical values to let a computer make those important decisions for us.

As far as AI development goes, I see ethical dilemmas arising at the point you actually start writing the code that would let a machine decide when to end a human life. If you are going that direction, I urge you to tread cautiously and seek the input of others, specifically a spiritual adviser familiar with the nature of your work. I would also advise you seek out someone who has taken life, a soldier or police officer or the like.

Hope this helps.