How to recognize a fictional strong AI?

Question

Intro

We (collectively, as humanity) have given quite a lot of thought to recognizing artificial conscious beings. We may not have a consensus, but at least we have a debate.

Now, let's imagine that a company like those that own ChatGPT, OpenAI, etc announced that it developed strong AI. To be more precise, let's say that the claim is "self conscious and self aware artificial being".

Judging by the latest trends, such technology would probably start as closed beta, then would be paid.

Now, such a model allows for a simple yet effective fraud - you put humans on the other side of the cable. It would require work on knowledge sharing (if you say X on one machine, another machine should, at some point, be aware of X too), but that is a matter of good automation and engineering. The strategy of slowly growing the userbase at a controlled pace would be familiar to the public, yet very helpful to the scam.

Question

What is the "reverse Turing test"? How do we prove that the agent claiming to be strong AI is in fact human?

Initially the latter sentence was phrased as "How do we disprove that the other end of the conversation is artificial?", but I realized that it's not the same as the question I really wanted to ask.

Precise setup

• as stated before, the claim is "self conscious and self aware artificial being"
• all the communications happen over an internet page
• there is a phase where the product is totally internal to the company, then another where it is used by a very limited set of users, then a phase when users need to pay to use the product (conduct conversations)
• fact propagation is present (fact stated in one session is at some point "available to the product" in another session)
• fact propagation time grows with the number of parallel sessions (it would make sense that such a distributed entity would delegate agents to conduct conversations and periodically merge them into its core; on another hand that could also indicate a delay in data propagation in a system that guides the humans acting as AI sessions)
• the agent that is claimed to be conscious passed the Turing test
  • this constraint is raised in response to this answer
  • the aforementioned Turing test was conducted in its simplest form (as above, remote communication sessions, one-on-one, not group chats), but on rather large number of people (1000+ with statistically adequate number of artificial agents); constraints that are considered to be standard (e.g. "no politics") were applied
  • long story short, intuitively, you'd say "it passed a serious Turing test" in a casual conversation; this topic would not be purely philosophical, it would have serious real-world / law-forming consequences
  • the experiment we're looking for is not necessarily constrained in the same way as the Turing test; it means that even if during Turing test we said "no politics", we still can do reasoning based on response to "what do you think of Trumps ideas on economy?" to establish if we're talking to a human or not
• ... (TBD in discussion, if needed)

Answer 1

Generally "strong AI" does not mean "human-like" AI. Even "self conscious and self aware artificial being" does not imply human-likeness. Humans have personality and flaws caused by the make of our brain/mind, whereas AIs would have different flaws and possibly different personalities. As a commercial product, we can expect ai "personality" to be very restricted and standardized to fit a purpose, rather than "growing naturally". Most realistic applications of AI would likely not to be human-like, but better than human in some respects for their given intended usage. As an example being available 24/7 without tiring, bring faster, less emotionally frail, less agressive, less demanding, more knowledgeable...

However assume a company claimed that it has created an AI indistinguishable from human Jane Doe living in Ohio. And then by interacting with this, you feel like you're actually interacting with a human Jane Doe in Ohio. Then philosophically there is no way from the interaction to tell whether you were interacting with a human or whether that company was successful. Any hint you would get that this "might be a human instead" could be a clever trick by the company to make their product more human-like.

Answer 2

Amusingly, this "con" actually happened once. From Wiki:

The Mechanical Turk was a fraudulent chess-playing machine constructed in 1770 by a certain Wolfgang von Kempelen to impress Empress Maria Theresa of Austria.

In actual fact, a human chess master was hiding inside to operate the machine. The Turk -- or rather, the successive chess masters hiding in it -- won most of the games played during its demonstrations around Europe and the Americas for nearly 84 years, playing and defeating many challengers including statesmen such as Napoleon Bonaparte and Benjamin Franklin.

Answer 3

That's likely a similar problem to (realistically) faking the moon landing. In that if you want to get all the small details right, that project would be much more costly than attempting the actual thing.

For example there are a lot of philosophical and scientific questions about strong AI and how it works, what their existence would mean for the concept of consciousness, how it relates to us and all that. And before anyone even gets to test your device, you'd be prompted for answers on those, from lots of people, from lots of domains, that team up to bust you (actually maybe not even to bust you, but because they are interested and would bust you in the process). And you would need to be able to produce these answers as your invention moved them from speculations to brute facts.

And it's not sufficient to replicate the literature on those things either, because the people who have written the literature on those things were not able to solve these problems and at some point hit a brick wall. Meaning the questions will likely be very specialized and specific.

And again I'm not talking about one domain here. Computer scientists, the AI and data science people, physicists, biologists, philosophers and everyone else will have questions and we're not talking simple questions here.

Also even just hiring one of these branches ... in it's entirety... is both impossibly expensive and still not going to cut it. And unfortunately these groups of people might actually be the groups of people that are actually interested in finding out how that shit works and are not in it for the money.

Also, if you produced realistic, consistent and convincing answers on all of these questions or at least a whole bunch of them, you might not have needed a scam but could have secured funding for an actual research project, saving you a lot of the legal trouble.

Because otherwise, as others have mentioned, you'd run into a problem of scale and scope. Like if you make this the mechanical Turk2.0 and claim it's a super complicated and unique (idk picture a CERN level experiment that is hard to just copy and paste to another place) so that only a limited amount of people can test it and it takes forever to compile an answer. Then what happens is that people will give the experts the first shot at checking it and approving it, meaning you'll get the hardest tasks right from the beginning and are very likely to be busted before you even get near your targets.

And if you let regular folks beta test that, you'd still need the approval of the expert telling, "It's more than just a carnival trick". But you'd run into a whole different problem namely one of scale. Like ChatGPT basically exploded 1 million users in 5 days and currently 180+ million regular users. If you want to present each of them with a personalized sessions you'd need to hire entire countries worth of people. And depending on the usage you might need to triple those numbers to cover a 24h schedule with 8 hour work days, because there's no reason why the AI should be shut off at night.

And that availability comes with server cost, that the network experts will have an eye one. So either the tech is too expensive to scale (which you'd need to justify) or the tech can be scaled then you need to scale the tech (and the humans involved). Which makes hiding that conspiracy ever more impossible and costly as you'd have to convince people before you can even monetize that.

So as a scam that would borderline be impossible unless you restrict the parameters considerably and only target a very small group of people, who think they are very special, but aren't actual experts. In which case you could have that much simpler.

Also as others have pointed out, humans have countless of flaws. Repeat the same question all over again and they get mad, flood them with even simple requests or for an extended period of time and they get tired, let them solve "simple" calculations like 116545465/454654*41146546+456465 and a computer can do that in fractions of a second while a human would struggle even copy pasting those numbers and if they automatically take them as code, try code injection or codify them three thousand 400 26 + Twelve million 5 hundred thousand 2 hundred sixty 5. Decoding that for a computer is rapid fast, doing it as a human is much more difficult.

Sure the human could use a computer and program but even this interface takes time that it wouldn't take for a computer.

Also heat signals, does you super computer have toilet?

Answer 4

This is an anti-inductive question to some degree - I won't say any method we come up with will also guaranteed have a counter, but a lot of initially-plausible methods will end up being fakeable.

That said, speed is a plausible candidate to start off: human reaction times are relatively slow, and one of the issues that required solving in Turing tests was "getting the chatbot to slow down its responses to a plausibly-human speed". This wasn't hard for the programmers designing the chatbots, but it nonetheless required attention.

That, though, brings up the "fakeable" part: the would-be faker doesn't have to be just a biological human; they could have a capable-but-still-non-sapient chatbot handle the parts requiring inhuman speed, and attempt to take over for the parts requiring human insight. This might break down for, e.g., sufficiently long and original essays, or asking for non-keyboard characters in the responses, but those can similarly be planned for and programmed against.

Answer 5

How to recognize a fictional strong AI?

You ask for the source code. If the company won't give it to you, you may remain skeptical, believing that perhaps it's not an AI, but a Wizard of Oz behind the veil of secrecy. You may remain skeptical for a little longer, until you have read this post.

You spy on the communication channels and see how answers are actually generated. Where are they coming from? Are they generated from datacenters (without the hundreds of thousands of humans needed to maintain the scale of the world-wide 24-hour operation)? If so, then you have strong reason to give up your skepticism.

You ask for numbers about the scale of the operation. How many users are being served? Is it likely or even remotely possible that other humans could actually be generating those answers, and generate them so fast, in all the dialogues going on? (No. The sheer size of the operation makes this impossible -- except in a fictional universe too far removed from ours).

Anyway, a Turing test experiment will not in any way help here. (I mean, that's almost the premisse of your question, that a Turing test cannot help! Unless I misunderstood an important point.)

For another conclusive proof that the AI is really an AI, simply ask it to take its time, but come up with a proof for a math problem that has vexed mathematicians for a long time and is still open (you can tell it, that it can pick any open problem it likes -- the Goldbach conjecture, the ABC conjecture, anything). If it can do that, in a reasonable amount of time, that would definitely convince me. If the company blocks those kind of requests - or if the "AI" simply is not good at math - then we need to fall back on the other more physical methods.

All in all, I don't think the question has much philosophical relevance. I do believe that philosophy should help in unveiling scams - but only if (or in so far) those (imagined or real) scams are based on conceptual confusions, misguided preconceptions, certain strong social-cultural biases, and the like. But I don't see any of that in this problem.

Answer 6

Any strong AI would not only have intelligence on par with humans but also have access to great computing power that humans do not have. Simply ask it to solve some puzzle that you randomly generate with rules that you just made up. For inspiration see Tatham's Puzzles. Give it a time limit of 10 sec, and make sure that it is obvious how to write a program within 1 hr that can solve the puzzle within a few seconds. The AI, being as intelligent as a human, but much faster, should be able to write the program and run it, all within a few seconds. In contrast, a human pretending to be an AI would be unable to write the program within 10 sec. You must make up your own novel puzzle, otherwise a dumb AI (like chatgpt) may find a solution online.