Mary Midgley once wrote an essay, "Duties Concerning Islands", in which she asks us to imagine that Robinson Crusoe blithely desolates the island he was on, as he departs. No humans are harmed, Crusoe presumably feels satisfaction that his efforts worked (he is successful in destroying the surface forest of the island), so "where's the harm"? She argues that it is intuitive for us that the ho-hum ruin of the island represents something perverse or even depraved, though, and goes on to say:
Yet the language of our moral tradition has tended
strongly, ever since the Enlightenment, to make
that objection unstateable. All the terms which
express that a claim is serious or binding—duty,
right, law, morality, obligation, justice—have
been deliberately narrowed in their use so as to
apply only within the framework of contract, to
describe only relations holding between free and
rational agents. Since it has been decided a priori
that rationality has no degrees and that cetaceans
are not rational, it follows that, unless you take
either religion or science fiction seriously, we can
only have duties to humans, and sane, adult, fully
responsible humans at that.
One environmental ethicist with a strong human-independent standard of morality was Aldo Leopold, who conceived of something called a "land ethic," which invokes quasi-teleology to define goodness relative to things like plants, IIRC. Deep ecologists generally try to work out human-transcendent standards; see this SEP article's subsection on deep ecology for a gloss of them. John Rawls focused on the kind of duties Midgley assessed as dubiously framed, but even he says (A Theory of Justice, 1999 ed., pg. 448):
Last of all, we should recall here the limits of a theory of justice... no account is given of right conduct in regard to animals and the rest of nature. ... A correct conception of our relations to animals and to nature would seem to depend upon a theory of the natural order and our place in it.
Nicholas Rescher's Axiogenesis is an example of an attempt at such a "correct conception," with impractical applications to be sure, but still, he forms a theory where value can be intrinsic to not only the reality, but even the very possibility of things in our world. That is, if they had no such value, they would not only be unreal, but impossible.
At the end of the day, though, I would suggest that, "How can something be intrinsically moral?" suffers from a shortsightedness issue, since it is an example of the scheme, "How can something be intrinsically [property x]?" That is, intrinsicness even for nonmoral properties is/can be an amorphous description (see the SEP article on the intrinsic/extrinsic dichotomy). I will leave you with Kant's relevant remarks (in a first-Critique section regarding the "amphiboly" of our a priori judgments) (Meiklejohn translation, B-ed.):
The Internal and External. In an object of the pure understanding, only that is internal which has no relation (as regards its existence) to anything different from itself. On the other hand, the internal determinations of a substantia phaenomenon in space are nothing but relations, and it is itself nothing more than a complex of mere relations. Substance in space we are cognizant of only through forces operative in it, either drawing others towards itself (attraction), or preventing others from forcing into itself (repulsion and impenetrability). We know no other properties that make up the conception of substance phenomenal in space, and which we term matter.