Unfortunately, a definitive answer to address this question cannot fit into a short text.
First, it very much depends on what a computer is supposed to mean. In computer science, the notion of computing is very broad and I don't see the author's definition in the original question. Physicists could consider the entire universe to be a quantum computer. Some people would think of animals and us humans as biological computers.
The general answer is: software is limited by the methods and formalisms that we use to design it (as long as we remain in complexity domain of the capabilities of life). Particularly by how programming languages are designed.
In principle, software is powerful enough to mechanically generate any computer input that could be physically provided by a human computer operator (mouse actions, keyboard actions, microphone actions etc.). Internally, it's just a spatial-temporal structure of bits and bytes and a computer has far more bytes than are required to encode any possible physical output state at a time.
The holy grail of software would be a computer that can operate itself to achieve goals that it learns by itself from its environment, such that it supports certain emotions or abstract (moral) values (hopefully altruistic ones). Mathematicians dreamt for a long time about a machine that could do mathematics automatically by itself but there is no algorithm (finite by definition) that could do it. On the other hand, Mathematical intuition could be related to generalizing experiences and associations which a person makes in life. A computer without notion of the meaning of life won't be able to replace theoretical scientists or mathematicians.
There are some hard algorithmic limits for problem domains but these apply to all computers in nature, also to our brains. Luckily, otherwise it would allow for even more destructive empowerment of individuals.
If you are thinking of a computer very specifically of a static electric circuit with I/O (displays, basic input devices), there is no form of life, no matter how intelligent it is, even if it can do anything in the virtuality that a human computer operator is able to do. Life requires autonomy and the ability to sustain itself. A conventional computer could not make outside experiences or request inputs with its own will, even if it has that will.
This leads us to actually more interesting questions:
- What are the limits of machine learning?
- What are contingent capabilities of a robot (in contrast to a conventional computer device)?
- How much resources are required to create a simulation of a specific entity or property to an arbitrary fidelity?
The last question is one that I cannot answer.
For the first question. With a robot would could try to simulate artificial life in a real environment in the best case. Maybe not with the currently dominant concept of hardware and software, but theoretically yes. We cannot know, if there is more to life than our physical behaviour. I also think, qualia are beyond just physical behaviour but this is Metaphysics.
Machine Learning models on the other hand are far from what I consider to be actual (artificial) intelligence, marketed as AI. Machine learning models do not work with dynamic goals. They are optimized for static goals (training data plus regularization) and only learn from and work with concrete ostensional definitions which generally have lower clarity, more ambiguity, than extensional or intensional definitions.
A real instance of intelligence would be able to work with dynamic or autonomous goals, ambiguities/abstract concepts (i.e. different points of views), transformations and reasoning (including intensional definitions).
The more general field of work, concerned with intelligent self-acting software, is called Agent Technology. Reinforcement learning comes closest to this field as a machine learning paradigm which tries to learn actions. The problem of the classic AI however is the same, that goals are not dynamic or generated but provided in one or the other way, even if they are worked off dynamically.
One possible argument by believers in machine learning intelligence is, that any course of physical events that we can perceive and reason about are (physical) structures, otherwise we could not perceive them, and structures can be learnt, in the same way as we can directly describe them with code. Machine learning also is able to approximate any continous function between (Euclidean) spaces. Artifical neural networks have been inspired by how the brain has been imagined to work, only as an analogy.
The applicability of machine learning to classify real concepts is based on a biased interpretation of reality, that there are absolute truths or semantics that could be purely learnt from samples (without anything else). Maybe, this applies to special cases but not in general. This idea is problematic in practice. The machine's final precise understanding of a word will be different to ours. A computer vision model will not be able to define its precise understanding to anyone or a text-based model won't understand pronounciation (very different wrong spelling). You can't assume, the machine has the same understanding in a word like we have. We should not treat machine learning output as an interpretive authority, but the danger is real and may be worsened by calling it AI.
Some or most observable structures in nature are dynamic. Staticness emerges only in abstract concepts, by definition. Physical structures are ever-changing and classic machine learning cannot distinguish between changing and static parts inside each training sample. (Regularization can be used to weight features but from an observation point of view, this weight has no further meaning.) So, eventually, a ML algorithm cannot tell at the end, whether an identified structure in training data is coincidence (like the most common colour of horse race winners) or artifically constructed by socialization, and instead will treat it like a general truth. Machine learning doesn't acquire a notion of contingency.
Another contra argument concerning machine learning is that you cannot learn qualia that you cannot experience yourself. We cannot learn what a frog life is exactly like, not with any science. We can only relate to a frog based on our own qualia and biological similarities to us. Qualia are such hard to learn that many people deny animals emotions although I think, emotions are a necessity for self-preservation (even if they were very abstract).
However, it does not prevent us to create artificial machine qualia, encoded qualia, which could be used to drive program behaviour. This reminds me of video games that imagine how the world could look like from the view of a machine.