Nietzsche does not seem to rejoice in the reality of atheism in any of his works. He doesn't seem to regret it either (as it just is). As to nihilism, he saw it as a crisis, a crisis that must be overcome.
As to Father Seraphim Rose, what he said about “true ‘existential’ atheism” might apply to Nietzsche as follows: Nietzsche had a burning hated for Christianity – not to be confused with Christ or “God” – and a deep admiration, if not love, for Christ himself. Consider, for example, the following excerpts from The Antichrist (which perhaps would have been more aptly titled The Anti-Christian):
A 33: In the whole psychology of the “evangel” the concept of guilt
and punishment is lacking; also the concept of reward. “Sin" -- any
distance separating God and man -- is abolished: precisely this is
“the glad tidings”. Blessedness is not promised, it is not tied to
conditions: it is the only reality -- the rest is a sign with which to
speak of it.
The consequence of such a state projects itself into a new practice,
the genuine evangelical practice. It is not a “faith” that
distinguishes the Christian: the Christian acts, he is distinguished
by acting differently: by not resisting, either in words or in his
heart, those who treat him ill …
The life of the Redeemer was nothing other than this practice -- nor
was his death anything else. … He knows that it is only in the
practice of life that that one feels “divine,” “blessed,”
“evangelical,” at all times a “child of God”. Not “repentance,” not
“prayer for forgiveness,” are the ways to God: only evangelical
practice leads to God, indeed, it is God! What was disposed of with
the evangel was the Judaism of the concepts of “sin,” “forgiveness of
sin,” “faith,” “redemption through faith”— the whole Jewish
ecclesiastical doctrine was negated in the “glad tidings”.
A new way of life, not a new faith.
A 35: This “bringer of glad tidings” died as he lived, as he had
taught -- not to “redeem men” but to show how one must live. This
practice is his legacy to mankind: his behavior before the judges,
before the catchpoles, before the accusers of all kinds of slander,
and scorn -- his behavior on the cross. He does not resist, he does
not defend his right, he takes no step which might ward off the worst;
on the contrary, he provokes it. And he begs, he suffers, he loves
with those, in those, who do him evil. Not to resist, not to be angry, not to hold responsible -- but to resist not even the evil one
-- to love him.
A 39: I go back, I tell you the genuine history of Christianity.
The very word “Christianity” is a misunderstanding: in truth, there
was only one Christian, and he died on the cross. The “evangel”
died on the cross. What has been called “evangel” from that moment was
actually the opposite of that which he lived: “ill tidings”, a
dysangel. It is false to the point of nonsense to find the mark of the Christian in a “faith”, for instance, in the faith of redemption
through Christ: only Christian practice, a life such as he lived
who died on the cross, is Christian.
A 38: At this point I do not suppress a sigh. There are days when I am afflicted with a feeling blacker than the blackest melancholy --
contempt of man. And to leave no doubt concerning what I despise, whom I despise: it is the man of today, the man with whom I am
fatefully contemporaneous. The man of today -- I suffocate from his
unclean breath. My attitude to the past, like that of all lovers of
knowledge, is one of great tolerance, that is, magnanimous
self-mastery: with gloomy caution I go through the madhouse world of
whole millennia, whether it be called “Christianity”, “Christian
faith”, or “Christian church”—I am careful not to hold mankind
responsible for its mental disorders. But my feeling changes, breaks
out, as soon as I enter modern times, our time. Out time knows
better.
What formerly was just sick is today indecent -- it is
indecent to be a Christian today. And here begins my nausea. I look
around: not one word has remained of what was formerly called
“truth”.
With regard to those who have been "known to end in the blinding vision of Him Whom the real atheist truly seeks," in the months, weeks, and days leading up to his collapse in Turin on January 3, 1889, Nietzsche wrote a number of letters, some of which he signed “Dionysus” and some of which he signed “The Crucified.” (Nietzsche -- A Critical Life, Hayman, 1982) And he ended Ecce Homo with the words, “Am I understood – Dionysus versus the crucified.” (EH Destiny 9). In the latter case, however, he was clearly referring to Christianity, not to Christ himself.