Your question - "So why is this 'problem of evil' used as evidence against the idea of a superior Being?"
This problem of evil as evidence against a superior Being only arises in the western monotheistic traditions. It does not arise in the Eastern traditions. In an Eastern context, there is no good or evil, both are illusions. There is only God, the non-dualistic Brahman. Good and evil are illusions that can be seen only from a relative standpoint, not an absolute standpoint.
Good and evil are relative terms, there are no absolute good or evils. What is considered evil in one location at one time is not necessarily evil at another time or place. Example: Several hundred years ago it was a common accepted practice in Europe to put a deformed or crippled newborn outside the dwelling and put them "in the hands of God". Of course what happened was they froze to death, starved to death, or were eaten by wandering animals. This was not considered evil. Today such an action would be considered evil. Even today, what may be considered good in one culture is evil in another.
From the standpoint of the Godhead, all the worlds and their objects are reduced to an illusory appearance. Does the water in a mirage make the earth wet? Likewise, how can illusory evil affect the One Reality? How can illusory good affect the One Reality? (Pancadasi II. 97.)
There is no free will. Krishna says in the Bhagavad Gita (XVIII. 60-61.) "Bound by your own karma, O Son of Kunti, which is born of your very nature, what through delusion you seek not to do, you shall do even against your own nature. The Lord dwells in the hearts of all beings, O Arjuna, and by His maya causes them all to revolve as though mounted on a machine."
David Loy in his book Nonduality: A Study in Comparative Philosophy quotes William Blake's Vision of the Last Judgment:
Men are admitted into Heaven not because they have curbed & govern'd their Passions or have No Passions, but becuase they have Cultivated their Understandings. The Treasures of Heaven are not Negations of Passion, but Realities of Intellect, from which All the Passions Emanate Uncurbed in their Eternal Glory.
The Combats of Good & Evil is Eating of the Tree of Knowledge. The Combats of Truth & Error is Eating of the Tree of Life...Satan thinks that Sin is displeasing to God; he ought to know that Nothing is displeasing to God but Unbelief & Eating of the Tree of Knowledge of Good & Evil.
I do not consider either the Just or the Wicked to be in a Supreme State, but to every one of them States of the Sleep which the Soul may fall into in its deadly dreams of Good & Evil when it leaves paradise following the serpent.