Most everyone would agree that "cold-blooded" torture is morally wrong. We agree so much so that many assume it's objectively wrong.

That being said, imagine this thought experiment:

A criminal has a track record so tremendous that he is sentenced to solitary confinement for life. We don't really care what he does inside his confinement cell, because he is unable to harm any living thing or really interact with the outside world in any way. Now imagine this criminal is somehow able to create a conscious entity out of the atoms in the room and decides to torture it. At what level would this still be considered morally wrong?

On one hand, the criminal is simply playing with a configuration of atoms. But on the other hand, this configuration shares a common property with us: consciousness.

  • 1
    "We don't really care what he does" does not work in ethics, people do not stop being people when they are incarcerated, they retain their human rights and responsibilities. Your sci-fi scenario is hard to take seriously, but it is morally wrong, say, to torture a rat for entertainment (despite the fact that rats are considered pests), and it remains so when a criminal is doing it in a prison cell. And rats, or even humans, are also configurations of atoms.
    – Conifold
    Jul 16 '20 at 20:42
  • @Conifold What I mean by "we don't really care what he does" is exactly what human rights are. What I meant is he is free to do literally anything that he wants, i.e. the epitome of freedom. In the scenario I presented, I was inquiring on the morality of affecting a conscious entity versus a non-conscious one. This is just a thought experiment. No need to take it further than that.
    – Cam White
    Jul 16 '20 at 21:24
  • In ethics, no one is literally "free to do whatever", and it is immoral to torture any suffering being, even an animal, whether they are "conscious" or not.
    – Conifold
    Jul 16 '20 at 21:56
  • @Conifold If he's in an empty room by himself, why isn't he free to do whatever he wants? And in order for a being to suffer it has to be conscious. E.g. a tree is not conscious.
    – Cam White
    Jul 16 '20 at 23:53
  • 1
    @CriglCragl Depends on what "lone individual" and "in relation only to themselves" mean. In one sense, it might be impossible for such an individual to develop reasoning or speech either. But if they carry the spoils of humanization into their loneliness why should morality be excluded? Don't they say of some things that they are between you and your conscience, or between you and God?
    – Conifold
    Jul 24 '20 at 9:08

This is absolutely a philosophical question.

Nick Bostrom identifies the 3 malignant failure modes for Artificial General Intelligence:

perverse instantiation; infrastructure profusion; and mind crime.

The first two are physical, the third is virtual, which is essentially what you are talking about.

In principle, anything can be simulated by even very simple systems given enough time. The limit is quantum systems cannot be simulated by classical systems, so if there are intrinsically quantum features required for consciousness that's an issue.

What is the threshold for consciousness? Integrated Information Theory is one approach to trying to picture one, and it's likely to be a spectrum.

We provide almost no protections for some beings, like cock roaches. In animal experimentation a lot more protections are granted to mammals like mice, which though they can be killed as pests, can't be allowed to face suffering considered unnecessary, eg fed live to a pet snake. Animals we consider ourselves to ve able to form deeper emotional interactions with like cats and dogs, we protect above and beyond a criteria related to their cognitive abilities - many pet owners consider them equal in value to themselves or other humans (although probably a large majority don't). Chimpanzees and dolphins are being granted extended rights, eg through the work of The Nonhuman Rights Project supported by philosopher Peter Singer.

So, being simulated, or synthetic, is definitely not a bar to moral concern. However, exactly what kind of complexity or subjective experience is involved for a being, is key to how we treat that.

A fully simulated human, we would have a clear framework for. A synthetic mind might both not feel psychological suffering when a human mind would, and might suffer in ways that are invisible or unintelligible to us. As the complexity of what we can make/simulate increases this will be an open question. The transition between non-conscious and conscious matter is one of the areas of greatest interest and contention in all of philosophy.


If the criminal is in a cell with another sentient being then he is not by himself. Whether he created it or not is immaterial.

His world now constitutes a microcosm with the same moral issues as anywhere else.

Not the answer you're looking for? Browse other questions tagged or ask your own question.