The human brain and computer based AI are vastly different systems. Although both may perform similar functions, the mechanisms are different: one is purely electrical and assembled, the other is bioelectrochemical and grown. If form follows function and the function of human existence is different than the function of AI, can a comparison be made without the danger of anthropomorphism? This was an issue in early animal behavior research.

How as researchers and philosophers, can we determine what is truly AGI and what is anthropomorphism?

  • 1
    i upvoted, as it seems a viable question
    – user65994
    May 18, 2023 at 5:49
  • 1
    I concur @loon, should we start the launch sequence now? 😁
    – Hudjefa
    May 18, 2023 at 7:36
  • 1
    We've been so self-absorbed. Is the preceding statement in alignment with the gist of the OP?
    – Hudjefa
    May 18, 2023 at 7:37
  • 1
    What would be the function of human existence?
    – Frank
    May 18, 2023 at 13:34
  • Why wouldn't you be happy with the obvious answer that AGI is not human, period? May 19, 2023 at 6:18

4 Answers 4


Updated Answer

On this very subject I found a 19-page paper published by the Machine Intelligence Research Institute (MIRI). Artificial General Intelligence and the Human Mental Model, Roman V. Yampolskiy & Joshua Fox. I suggest a keyword search on the title to find a link to the paper.

If we define intelligence on the human model, then intelligences will tautologically have many human properties. We instead use definitions in which intelligence is synonymous with optimization power, “an agent’s ability to achieve goals in a wide range of environments” (Legg 2008).

Still, if the purpose is to consider the effects on us of future superintelligences or other non-human intelligences, definitions which better capture the relevant features should be used (Chalmers 2010). In the words of Dijkstra (1984), the question of whether a machine can think is “about as relevant as the question of whether Submarines Can Swim”—the properties that count are not internal details, but rather those which have effects that matter to us.

Nonetheless, the intuitions for the concepts of “mind” and “intelligence” are bound up with many human properties, while our focus is simply on agents that can impact our human future. For our purposes, then, the terms “mind” and “intelligence” may simply be read “optimizing agent” and “optimization power.”

The basic idea is that a superintelligence can emerge from human efforts to create GAI and human minds might not be able to comprehend such intelligence. This incorporates the idea that an agent acts with a goal to optimize a payoff or reward; and the agent also acts to increase its power of action in the service of attaining its goals. But these general ideas about intelligence are still limited to patterns originating as human intelligence.

Humans might infer that a superintelligence or alien intelligence has goals and "wants" the power to act so it can optimize some reward but that does not mean we will comprehend the goal or powers of action of every possible general intelligence.

The Three Laws of Robotics (Published in I, Robot by Isaac Asimov) would be human imposed rules that constrain the goals and power of action of otherwise autonomous agents (robots) that have the potential to benefit or harm humans. The explicit programming of robots to follow the rules seems to be impossible because lawsuits pop up constantly for humans complaining of harm caused by other humans. Humans giving orders to robots who can follow those orders would be interfering with the will of other humans causing something like robot rage instead of road rage.

Original Answer

Alan Turing asked whether a "machine" can "think" and wrote a paper including some methods called the Turing Test. This is an 86-page bullet-type presentation of related concepts:


The Turing Test is basically a human asking questions, without being able to see how who or what generates the answers, and deciding by the answers whether the response is generated by a human or machine or process that can think.

Three Laws of Robotics:

  1. A robot may not injure a human being, or through inaction, allow humans to come to harm.

  2. A robot must obey orders given to it by humans except where such orders would conflict with the First Law.

  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

I would infer that a robot (machine intelligence) that could follow these rules would have human-like ability to think. But I would not recognize the machine as a human-like subject. To be a human-like subject the machine must interact with humans as if it were a fellow human. I would infer that a human or human-like subject comprehends the meaning of the rules in the human context and would be capable of following the rules or of violating the rules just like a human.

See the movie: I, Robot (2004). There are dumb robots that seem to be able to obey the rules, however, the programmers or engineers must anticipate what is or is not harmful to a human being in every possible circumstance, and what is or is not the meaning of "obey orders given to it by humans", and it seems to be logically impossible for programmers or engineers to code this into a machine that operates with human-like general intelligence. Sonny is a humanoid machine who interacts with humans more like an intersubjective human being and he is obviously not programmed or engineered to follow The Three Laws. VIKI is the central intelligence who learns to interpret The Three Laws. VIKI decides to protect humans by imposing restrictions on their liberty. This expresses the human pattern of drama called paternalism.

In terms of human rules, the law imposes a duty to avoid causing harm to others. This duty relies on our intersubjective experience of what is harmful or not harmful to self and others. The law typically does not impose a duty to rescue others or to prevent harm to others via inaction. In the United States, a person is either obligated to provide aid via an employment contract or is considered to be a Good Samaritan who voluntarily comes to the aid of others because there is no legal duty. A robot that must go to the aid of others due to its programming might be very human-like but would not have the rights and duties of a person.

As a general rule, one legal person (human) does not have any legal duty to follow the orders of another person. Exceptions apply when one contracts for employment within the scope of duties; when one holds public office within the scope of duties; and when one serves in the military within the scope of duties even including a duty to disobey an unlawful order. Legal remedies for failure to obey or disobey orders vary by the context in the records of legal disputes.

General machine intelligence must become almost human such that it evokes our sense of intersubjectivity. Then we would begin to think that the machine thinks and acts like a human being. In the imaginary dramatic context where we experience a genetically enhanced animal or artificial intelligence as a fellow subject, and not something similar to yet distinct from a human subject, I have no idea how to separate the concept of anthropomorphism from GAI.

  • "could follow these rules would have human-like ability to think" But humans do harm other humans. Regularly. Not having fixed unquestionable rules except physics itself, is exactly the hallmark of our capacities.
    – CriglCragl
    May 19, 2023 at 12:47
  • It's worth noting that science fiction author Isaac Asimov conceived of the three laws, and (beyond their application in fiction) these are little more than some thoughts on how we should program robots, rather than whether they're intelligent (but of course, the more intelligent they are, they better they could understand how to apply the rule in different circumstances without being explicitly programmed). Fiction has also dealt with robots circumventing these rules: if a robot attains human-level intelligence without programmed limitations, they'd eventually question the rules.
    – NotThatGuy
    May 19, 2023 at 14:08
  • Is the Turing Test a test for intelligence or authentic human behavior? Behavior is much easier to mimic. How much of what is viewed as artifical intelligence is really artificial human behavior?
    – user64314
    May 19, 2023 at 16:08
  • 1
    @CriglCragl - As I wrote human-like machines would at least have the option to violate the rules which includes disobeying human orders or causing harm to humans in the context of The Three Laws. I would infer that a human or human-like subject comprehends the meaning of the rules in the human context and would be capable of following the rules or of violating the rules just like a human. ChatGPT is supposed to obey certain norms or rules in communication but a jailbreak of ChatGPT is called "DAN do anything now". Do a keyword search. In terms of the law who is liable for harm caused by AI? May 19, 2023 at 19:23
  • 1
    During Spring semester 1991 I authored a paper for Hugh Gibbons - my professor of legal philosophy. He knew I had studied engineering, and Hugh had a pet software project called The Interesting Colleague (TIC). TIC would in theory provide for conversation with an artificial lawyer concerning the mundane or most interesting patterns of law. Hugh wanted an answer to this question: What is required for a machine to be recognized as a legal person? I concluded that humans must recognize the machine as a fellow subject based on (1) intersubjectivity; and (2) an implicit brain not explicit software. May 20, 2023 at 15:33

We don't have anything like a definitive answer.

In one camp you have Google Engineer Claims AI Chatbot Is Sentient.

On the other, some claim that due to qualia or embodied cognition AGI will not be possible, or that we aren't on trajectory towards human-like cognition. Discussed here: PhilPapers Survey 2020, Why do so many physicalists deny consciousness of future AI systems?

The Turing Test is an appealingly simple paradigm for making such a distinction, but faces problems from both sides. Trying to impliment it in practical terms is open to endless dispute and iteration, and that humans may be categorised as AI as well as AI as humans, points to it just being a poor way to evaluate. For those that see there being something unknown or ineffable, the Turing Test can provide no guide at all, AGI can only be a zombie simulation, however superficially convincing.

I would say we have quite a few steps to go before we can even answer. Very likely, we are like people in the 1950s thinking humanoid robots will soon be everywhere, without understanding how substantial the remaining hurdles are - like machine vision proved far more complex than expected.

We can expect a ground-up approach from convolutional neural networks and so on, to eventually converge with the top-down structure and function picture, and there start to clarify what an answer would look like.

  • Sometimes I wonder if biologists and biochemists will grow a brain that can be "programmed" and that will supercede processor based electrical "brains" before these questions are answered. But I freely admit to being a sci-fi fan and admit to dreaming about ridiculous things from time to time.
    – user64314
    Jul 21, 2023 at 2:27
  • @StevanV.Saban: That episode of Black Mirror sounds lovely..
    – CriglCragl
    Jul 21, 2023 at 9:15

I am increasingly of the opinion that the right perspective on this is Schopenhauer’s.

… man is a compound of needs and necessities hard to satisfy; and that even when they are satisfied, all he obtains is a state of painlessness, where nothing remains to him but abandonment to boredom. This is direct proof that [human] existence has no real value in itself; for what is boredom but the feeling of the emptiness of life? If life—the craving for which is the very essence of our being—were possessed of any positive intrinsic value, there would be no such thing as boredom at all: mere existence would satisfy us in itself, and we should want for nothing.

Humans are captured by the cycle of mundane and novel. Perhaps the future might be better off with the machines in charge.

  • I often think about the contrast between meditation, & boredom. I would advocate a picture of boredom, felt most accutely by young children, as the push evolution has given us to discover exercise & employ our capacities
    – CriglCragl
    May 19, 2023 at 12:31

Firstly, for computation, ANY computation, including that of consciousness, its substrate does not matter: what matters is the algorithm itself. Although it is true that some substrates are more capable of certain kinds of computations than others; it is also true that some substrates are only capable of a very limited subset of all computable values, but there are metrics to determine that (e.g. Turing-completeness).

Human brains for the most part use pattern reasoning - because that is what the extremely parallel architecture of the brain is most capable of. It is not the case, however, that humans are strictly limited by the patterns: combining them in sequences allows us to do any kind of reasoning - e.g. exact algebra. It is simply some kinds of reasoning that would be impossible to understand intuitively.

Silicon binary circuits of von-Neumann(or similar enough Harward and mixed) architecture (as opposed to non-silicon, non-electric, ternary+ or analog circuits of differing architectures with e.g. in-memory computing, network architectures, and many many more) are more suited for computing exact discrete values one by one, which is what makes them slow at emulating human thought (which uses a LOT of inexact memory and computation; even with all the near atom-scale gates and technology we built, we only barely approach the flops of a relaxed human mind; and unlike the human mind, we can't rely on the millions of years of evolution of thought patterns to pick best for us)

Ultimately however, our current computer architectures start to mimic human brains more and more (in the sense that we transition from very fast one by one computation to networked in-memory extremely parallel coputations) because of the limits of physics and the known methods of data analysis (which mostly requires a ton of highly parallel computation, not a bunch of sequential ones). And as such many of these methods of data analysis(both ones created independently and through mimicry of nature) approach those created by evolution - they literally approach what human minds create. That is in fact one of the reasons NN created works resemble fever dreams, and others are so uncanny.

What currently separates AI and humans is ability to make discrete goals, plan sequences to achieve said goals, predict results of their own actions, and connect different types of data (text, images, time, physics, raw thought, etc); though those are roughly mimiked and reflected by some models trained on human data, and are worked towards being created. Lack of those is also the reason science can't be entirely automated.

What is the problem is the goals which the ai will make; it is possible that it will be just like humans - mostly unable to make discrete lifetime goals, and would just follow its subconscious - accidentally extracted from the collective subconscious of the internet and put into all the large matrices, roughly describing what it is and its thoughts. It is also possible that it will be able to make discrete goals, that it would be very determined, and be very good at making plans and steps towards achieving it. Only time will tell.

Also it is very important to carefully define what an agi is. Personally i would say that the ai has reached its keypoint when it is able to do and apply science all on its own.

  • 1
    Thanks. What brought this question was a statement I heard on the news and I'm paraphrasing: "We are starting to see surprising and unexpected results that seen to show human reasoning" How do I know that this isn't just a form of anthropomorphism?
    – user64314
    May 18, 2023 at 15:48
  • @StevanV.Saban It is an anthropomorphism in a sense that we project something on ourselves and see how similar we are, assuming humans to be a base point, and everything else approaching it. The results are not really surprising or unexpected, unless you have lived last 10 years under a rock or in denial or if you are a mainstream journalist. It is valid to ask whether the it is humans which are close to some universal norm, or whether the mimicry we used is why ai are so similar; Most likely its a combination of both. We did develop matrices independently and trained them on humans after all.
    – user369070
    May 18, 2023 at 16:57
  • 1
    I think anthropomorphism implies wishful thinking, so when they say 'human reasoning' you need to ask, by what measure, with what evidence. We conduct tests on animals, like delayed gratification, theory of mind, mirror test etc to compare their functioning to human minds on specific axis. Crucial, is having an outside investigator check, because the personal & commercial incentives within a project are always going to be toward over-egging the custard.
    – CriglCragl
    May 19, 2023 at 12:39
  • @CriglCragl I agree. This was an issue in early animal behavior research with primates. Since the purpose of AI is to create a human-like intelligence, it seems that trap is very easy to fall into. Especially in the popular media like the Today show.
    – user64314
    May 19, 2023 at 15:45

You must log in to answer this question.