Pigliucci gives an interesting review of the Mathematical Universe based on personal conversations with Tegmark. Apparently, Tegmark does admit plurality of mathematical structures, at least hypothetically, but his plurality is much reduced compared to what even "one-truth" mathematical platonists admit. First,
"Tegmark replied that perhaps only Gödel-complete mathematical structures have physical existence (something referred to as the Computable Universe Hypothesis, CUH). This, apparently, results in serious problems for Max’s theory, since it excludes much of the landscape of mathematical structures, not to mention that pretty much every successful physical theory so far would violate CUH."
Inconsistent mathematics is most definitely out. Second, to Pigliucci's surprise (and mine) "he professed himself to be an infinity-skeptic". It is an odd kind of platonist who is skeptical about infinity, especially considering the kinds of math his mathematical universe requires. But perhaps this means that Tegmark's mathematical universe is somehow made of finitist mathematics, see Finitism in Geometry. Nonetheless, within these strictures, Tegmark is open to testing which of the (few) structures left is realized. That, unfortunately, creates more problems:
"He also has stated in the past that — assuming we live in an average universe (within the multiverse of mathematical structures) — then we “start testing multiverse predictions by assessing how typical our universe is”... But how would we carry out such tests, if we have no access to the other parts of the multiverse?
Max went on to say that his hypothesis has “zero free parameters” and is therefore favored by Occam’s razor. But if you check his paper at arxiv.org he says: “If this theory is correct, then since it has no free parameters, all properties of all parallel universes … could in principle be derived by an infinitely intelligent mathematician. … Finally, the ultimate ensemble of the Level IV multiverse would require 0 bits to specify, since it has no free parameters.” There are a couple of obvious problems here. One is the dearth of infinitely intelligent mathematicians, the second the fact that the above mentioned Level IV multiverse is precisely what gets dramatically (and unrealistically) shrunk as a result of Gödel-imposed limitations."
It seems (to me and many others) that Tegmark could solve some of his problems by splitting apart mathematical and physical existence, which would allow him to admit platonic existence of all coherent structures but to reify only some of them (as most mathematical structuralists do). Unfortunately, this makes reification a contingent matter.
Being a physicist, Tegmark wants more, namely to predict which ones are reified. And this drives him (like Quine before him) towards minimization and Occam's razor, as opposed to the maximization drive of most mathematicians (platonists or not) towards the principle of plenitude. He wants to limit the number of even platonically existing structures to limit the number of hypotheses about the universe to be tested. The result is a confused mix of physical and platonic existence that leads to statements like "the universe is a mathematical structure", see How can the physical world be an abstract mathematical structure?, which puzzle many because they look very much like a good old category mistake:
"When Tegmark said that fundamental particles, like electrons, are, ultimately mathematical in nature, Julia suggested that perhaps what he meant was that their properties are described by mathematical quantities. But Max was adamant... Nevertheless, Julia and I insisted, it is a physical property described by a mathematical quantity, the latter is not the same as the former.
Could it be that theories like MUH are actually based on a category mistake? Obviously, I’m not suggesting that people like Tegmark make the elementary mistake of confusing the normal meaning of words like “objects” and “properties,” or of “physical” and “mathematical.” But perhaps they are making precisely that mistake in a metaphysical sense?"
What "is a mathematical structure" is likely intended to mean is that according to Tegmark what is real are only structural relations, not physical objects that enter them in the theories. This is perhaps motivated by structurally equivalent reformulations of theories that have completely different sets of objects. But this does not erase the difference between physically real and platonic structures, and the former need some causal powers in addition to structural relations to be real.