It depends greatly on your ethical system, your metaethical system, and to some extent, on how you define the terms "good" and "evil" when they are used to describe objects (most ethical systems focus on defining those terms with respect to actions, and not objects or people).
For example, under a consequentialist framework, we analyze goodness or badness in terms of the consequences which an action creates. It is not immediately obvious how we should analyze the "consequences" of an object (or even a person, for that matter), because objects are not actions, and we don't normally think of objects as having consequences. Similar objections attach if we try to use a deontological framework (which focuses on the duties and obligations that the action fulfills or violates), as well as many other common ethical systems.
For artificial objects, we may sidestep this objection by asking about the morality of creating the object. For both natural and artificial objects, we might ask about the morality of destroying them. These are actions, and may be morally analyzed in whatever fashion your preferred ethical system prescribes. You might then decide that an object is "evil" if destroying it is good, or if creating it is evil. This is not a bad definition, although it is not necessarily the best definition we could use. However, I think this line of reasoning may be missing the point of your question, because the act of creation or destruction itself implies an agent who does the creating or destroying, and you're specifically asking about a world which lacks agents altogether.
So let's start over. Instead of focusing on actions, we can focus on the object itself. It is possible that some objects might inherently exemplify certain virtues or vices, in which case you could argue that those objects are (respectively) good or evil under a system of virtue ethics. It's unclear how this would be extended to other ethical systems, or indeed whether an object can truly exemplify a virtue in the first place, but it's better than nothing.
The major problem with virtue ethics is fairly straightforward: A typical modern account of virtue ethics is still rooted in the motivations and preferences of agents, because it needs to offer an account of what "virtue" is and how we should distinguish it from vice. The explanations for this vary, but they nearly always appeal to some sort of relationship between the agent, the action taken, and the agent's motivations or thought processes. If there are no agents, then there might be no virtue for objects to exemplify.
There's a solution to this, which is to characterize virtues as abstract, metaphysical Platonic ideals that may exist independently of agents, just as the number 2 might exist independently of agents (under a Platonist account of mathematics). However, this is likely to be unappealing to the hardcore physicalist, because it more or less requires the existence of abstract, non-physical objects.
So much for ethics. What about metaethics? Well, there are a lot of different positions you might hold, so let's quickly run through a few:
- Mind-independent realism probably has the best shot of working without agents. It posits that moral statements are statements of objective fact which do not depend on the opinions or beliefs of agents, so removing the agents from the equation altogether should not break anything.
- Sentimentalism is complicated. It says that ethical statements are statements of our "sentiment" or emotional reaction towards a situation, and it seems unlikely that you can reconcile that with a lack of agents. However, there are versions of sentimentalism which look more like a theory of epistemology than of metaethics, and under those interpretations, it's less of a problem (but you still need to pair it with a realist theory of metaethics for this to actually work).
- Relativism and non-objectivism are both dependent on agents in an even more fundamental way, since they both want moral statements to be interpreted differently by different agents. I don't think either system is compatible with a world where agents don't exist.
- Non-cognitivism (moral statements do not have truth values) and error theory (moral statements are all false) are extensible to objects in the obvious way, but aren't terribly interesting to analyze once you've done so.