Invoking the diction 'ego' is to many to invoke psychology, or at least Freudian thinking, if you reject the latter is the former. Yet, Max Stirner is recognized in his book The Ego and Its Own as laying the ground work almost a decade before Freud's birth. But regardless of your starting point, inevitably when one talks about the 'reality' of others (for what is it if it is not to challenge the reality of others by equating them to ghosts?) one is destined for the problem of other minds which is an epistemological problem which asks how and what do we really know of others since we have no phenomenological access to their minds?
This of course is par for the course with Continental Philosophy, and German philosophy which maintained its hold over German thinkers despite the fame and repute of both the Berlin Circle and the Vienna Circle one hundred years later.
To answer your question, would Stirner be likely to view others as spooks meant to harass the self? Note this line from Wikipedia:
Stirner's Egoism is not a descriptive psychological egoism, in fact he believes that non-egoism is the most common way of thinking. Stirner also does not advocate a narrow prescriptive ethical egoism of self-interest. Stirner rejects, for example, the actions of an avaricious individual whose only pursuit is material gain. For Stirner such a pursuit enslaves the individual to a single goal and this is incompatible with his idea of autonomy.5
Further on the article states:
He explains the relationship between the egoist and his objects or other persons as one of 'ownership'. For Stirner this means that there are no restrictions, moral or otherwise, on how an egoist can relate to other things and persons. The egoist views others instrumentally, they are "nothing but - my food, even as I am fed upon and turned to use by you".6:263 The consequences of this view is that he does not see murder, incest and infanticide as always unjustified. Stirner admits that this view affords little comfort to others but he states that his audience's concern is of no importance to him.
Therefore, it seems to fair Stirner's views as expressing a range of potentialities between others and the self from collectively altruistic to at a bare minimum metaphorically cannibalistic. Given his theory on the variety of relationships individuals can have with each other, I think it would be fair to speculate that Stirner would certainly reject the notion that others are only spooks meant to bedevil the ego into submission which sounds more Freudian in its pessimism.