Consider the following scenarios, each of which at their core refer to a form of 'reward hacking'.
Based on personal observation, each scenario below generally elicits an immediate, visceral feeling of ~'this is just wrong' in the majority of people.
The question is: Which ethical framework would deem 'brain reward hacking' immoral? And why?
The scenarios:
Access to medical monitoring equipment and an unlimited amount of pure morphine
One can imagine a device that monitors a person's blood morphine levels, measures brain activity and vitals, and continuously injects the precise amount of morphine that keeps the attached human in a constant state of 'happiness' (without killing them).
Soma in Brave New World
The book describes a fictional society where a 'happiness drug' called soma is provided to citizens, with the aim of keeping them 'happy' and free of 'negative' emotions. When people become unhappy about something, the government simply administers more soma.
Super-intelligent AI that is a human happiness maximizer
Suppose a super-intelligent AI was developed with the goal of maximizing human happiness. Upon learning a bit of neuroscience and engineering, the AI decides to and surreptitiously implants all humans with electrodes and sensors. Each implant monitors each person's 'happiness' levels and directly stimulates the brain reward centers to keep the humans in a perpetual state of maximum 'happiness'.
Ethical Analysis
Utilitarian: It seems that all three scenarios would be a net-positive increase in happiness, thus not immoral.
Rawlsian: If any of the three scenarios were to become more likely, I am not sure that I would be particularly averse to being born as one of the people described in the scenarios. If I was born as one of them, it seems that I would have a relatively blissful, happy existence.
Kantian: If these were universalized, and everyone was on morphine or soma all the time, I can see that it couldn't go on for too long -- someone would need to operate the morphine/soma factories, get raw materials, machines, etc... However, in the case of AI, the AI would perform those tasks, and all humans could theoretically stay in a perpetual state of 'happiness'. So under Kantian ethics, unlimited soma and morphine scenarios would lead to a contradiction, while the AI scenario would not (?).
Concluding thoughts
It seems that the disgust/'this is appalling' factor primarily arises when the scenarios are considered from a third-person perspective. The person who has the morphine/soma/electrode treatment done to them experiences happiness, while a person who hears or sees this being done to another person feels disgusted.
So then is the moral intuition about brain reward hacking irrational?
