What are the criteria that makes metaphysics metaphysics? In my understanding, metaphysics literally means beyond philosophy, but what are the criteria that makes something be metaphysical? Is the only criteria the fact that it has to be beyond what philosophy can uncover?
Metaphysics has a clear definition. It is the foundation of philosophy and all of philosophy depends on it. As Conifold notes, a close synonym is Ontology. It deals with all the things physics does not deal with, which is absolute truths, first principles, fundamental theories and the true nature of Reality.
It is not so much before or after physics as surrounding it. Physics does not make contact with Reality but must operate within the relative world. Thus theoretical physics tails off into metaphysics, leaving space-time, consciousness, matter, ethics, origins and other such things unexamined. Not entirely unexamined, but examined only up to the point metaphysics begins. A fundamental theory is a metaphysical theory. The phrase 'fundamental physics' is a misunderstanding of both physics and metaphysics.
It might be right to say that metaphysics is to philosophy as axioms are to mathematics. If we do not study it with some success then our philosophy is rootless and can never be systematic.
I suspect you've misread something somewhere. Metaphysics is philosophy. What it studies is beyond what the natural sciences study. Biology tails off into chemistry, chemistry tails off into quantum physics, physics tails off into metaphysics and metaphysics tails off into mysticism.
While physics may be different in a different universe, metaphysics is the same in all possible and impossible universes, even in the Matrix or the whacky world of Butler's Erewhon. Metaphysical truths are true in all possible universes.
If you read science fiction you'll see that the physics changes wildly between authors and stories but the metaphysics is always exactly the same. There are no SF stories in which some future hero discovers the truth about metaphysics since to do so the author would have to discover it and those who discover it don't write science fiction, or at least I've never seem an example.
Two good beginner's introductions are The Making of a Philosopher by Colin McGinn and The Mind of God by Paul Davies.