Two different criticisms of the thing-in-itself can be found in Nietzsche's work, a good review is Riccardi, Nietzsche’s critique of Kant’s thing in itself.
Both criticisms argue that the idea is inconsistent. The first, more straightforward one, can be found in Gay Science and is sketched in the OP. It is that Kant applies the notion of causality to infer the existence of thing-in-itself whereas he himself previously restricted the category of causality to appearances only.
"Kant was no longer entitled to his distinction between “appearance” and “thing in itself” – he had denied himself the right to continue to distinguish in this old, traditional way having rejected as invalid the inference from the appearance to a cause of the appearance – in accordance with his understanding of the concept of causality and of its purely intra-phenomenal validity."
This was a common objection already prior to Nietzsche. Riccardi suggests that Nietzsche here copied almost literally a passage from Teichmüller's 1882 metaphysical treatise. The problem with it is that this is not how Kant arrived at his thing-in-itself. It is not the cause of appearances for him, but merely a conceptual plug: if there are appearances then there is something that appears. The relation between them is not that of causation, but of abstract expression. Put more positively, the thing-in-itself is a noumenal completion of phenomena. The above may be a valid criticism of some quasi-Kantian misconceptions, but it has nothing on Kant himself.
Nietzsche's second criticism is more to the point. He argues that positing a relationless propertyless "thing", which Kant's thing-in-itself must be, is even conceptually absurd. This line of reasoning appears in Kritische Studienausgabe c. 1887:
"The “thing in itself” [is] absurd. If I remove all relations, all “properties”[,] all “activities” of a thing, then the thing does not remain left. [...] The “in itself” is even an absurd conception: a “constitution in itself” is non-sense; we always have the concept “being”, “thing” only as the concept of relation."
Anderson unfolds Nietzsche’s second argument as follows in Nietzsche’s Views on Truth and the Kantian Background of His Epistemology:
"The unknowability of things in themselves is part of their very conception: it arises not from some contingent deficiency or incompleteness in our experience or theorizing to date, but from general and inevitable limitations on our cognitive resources, most importantly the lack of intellectual intuitions capable of representing such objects. This means that in attempting to conceive of things in themselves, we outstrip the legitimate realm of our concepts, and therefore stop making sense altogether".
Now to the Will to Power. It is true that many early commentators pointed out Nietzsche's own inconsistency in presenting the Will to Power as a sort of beyondly thing that he rejected in Kant as incoherent, and passages like aphorism 36 of Beyond Good and Evil give quite a bit of fodder to that. It is also true that Nietzsche seems to need a common referent for his "perspectives" in epistemology, and that the Will to Power fits the bill. Moreover, although he generally uses the term ambiguously and confusingly, he does described it as "the essence of the world".
However, we should remember from Gay Science that
"What things are called is unspeakably more important than what they are [...]: what started as appearance
in the end nearly always becomes essence and effectively acts as its essence!"
It is this "essence" that the Will to Power likely refers to at its core, the Relations-Welt essence, not the beyondly essence of metaphysics that Kant relegated to his relationless thing-in-itself. "There is no “essence in
itself”, relations first constitute essences", as Nietzsche says in Kritische Studienausgabe. On this interpretation, the Will to Power only summarizes the world as given to us (in the integrated sense of "us", split up into perspectives), the world of appearances, it does not reintroduce the thing-in-itself or "supplant" it.