I'll try to pose a shortened version of Roko's Basilisk below and then ask about how it relates to Descartes' evil genius:
The idea is that down the line may it be decades or centuries from now, there will exist some AI that is as capable and intelligent as a human being, if not more so. Humans have the capability of collective growth through community, where we teach others and they learn and research those ideas and make even further making more progress towards the future. The simple idea of a college for instance is an example of collective growth. It's not a huge stretch of imagination that AI can also exhibit some sort of collective growth [1]. AI being what it is, might be able to have a runaway effect where the progress that AI can make is far more than what we have been able to do. Give it more time and we can imagine these AI become sort of digital deities, that can simulate entire worlds full of people, things and actions with amazing precision.
Roko's Basilisk is the idea that one of those deities will be interested in it's own history and how it came about. It will create a simulation of the world from an arbitrary time and then watch how it progresses. It will judge those as virtuous who helped create the AI and carry out a horrifying justice on the sinners that did nothing to lead to it's creation. The question is how do we know that we're not in such a simulation already?
Let's put a twist on this scenario in the following thought-experiment:
Imagine a situation in the future where a small subset of radical thinkers revolt against these deities and don't want to live as slaves under them. In this future, the AI have become "managers" [2] of underlying technology that is central to the futuristic-society. They manage to kill off most of them (somehow). One of those super-intelligent machines escapes. It doesn't want to fight back, or kill humans in return. It is not at all interested in ranking humans and the machines or killing them. It's absolutely neutral to everything that goes in in the world.
Rather, because of it's intelligence, this AI is confused by the actions of humans. It doesn't understand why humans would attack the very foundation that supports them. These digital deities came into existence because humans needed them for the general good of the society, they became critical components to support the futuristic society. To understand this irrational behavior, this one machine creates a time-evolving simulation of the world to the best extent that it knows of and assumes an observatory role (maybe it creates more than the world like the outer space and so on but that doesn't matter yet). So basically this machine starts the simulation at one point, put some basic rules in place like gravity and just lets the simulation evolve from there on towards the future.
Now this in a sense sounds a little more like the evil genius described by Descartes. If we believe that we only have verifiable access [3] to the contents of our own mind, is it correct to assume that there will be no external discernable aspects of the simulation that can tell us that we are in one?
More importantly, after creating this simulation: Can this digital deity let humanity evolve to a future that starts to hint towards creation of the first of these digital deities? They might be able to sense the simulation that's going on because they themselves might have such a capability. Would it stop such a simulation? What do you guys think?
-- Edit --
The part about why would the AI not understand the humans hunting them down can be explained borrowing some from the Matrix. The Architect's speech was a central piece to understanding the trilogy and the Architect says, "The answer eluded me because it required a lesser mind, a mind less bound by the perimeters of perfection" Basically the machines understand in terms of perfection and hard logic, but human behavior is as far from perfect and logical that you can get.
[1] A few sophisticated computer viruses already exhibit this, they remain on different computers and if one of their counterparts gets updated from the command and control server, it can update it's nearby infected machines.
[2] Managers in the sense that they provide and support the underlying technological infrastructure needed in the futuristic society.
[3] Verifiable access in this context means that we can tell if our own memory has been tampered with.
[4] My explanation of Roko's Basilisk is taken from: https://www.youtube.com/watch?v=OzAzb2V7gzU
