Plato's notion of anamnesis may not apply to much of the world. But in mathematics, his chosen example in the Theatetus, it seems right. Logic does not require consensus to determine its validity because it is an inborn feature of human beings. Even if we are not trained in any kind of formal reasoning, at a given age and level of intelligence, we have a predictable reaction to contradictions and impossibilities. The most basic kinds of math seem more like remembering than learning.
Mathematics elaborates logic, and seeks out similar things that we will naturally all agree upon (e.g. a straight line as we naturally imagine it, until we think about it too seriously, is infinite and infinitely divisible.)
It can be wrong about what those are, but once it has found one, it can reasonably combine it with all the others it has already collected.
In the history of mathematics, ideas really do become mathematics by consensus. Take infinitesimals. They were basic tools for early physics. But then we found the idea was not thoroughly shared, to the degree we got different results from different people when we combined infinitesimal proofs with other known math. They got thrown away and worked around, with the project of 'arithmetizing' analysis. But once we played with them enough, they got reconstructed in different forms until they took a few forms that can be agreed upon.
Yet once they are there, they are there. L/os theorem, Ultraproducts and 'External' numbers are never going away, they seem to really capture this intuition and to be formally stable enough to combine it with the rest of math. It turns out that the intuition was real and stable, despite prior consensus this was not the case.
On the other hand, mathematical Platonism does fail. There is good evidence that mathematics already contains intuitions that cannot be combined. Russel's theorem suggests that we need to better refine our notions of self-reference, universality or negation. We have silly workarounds, but in the end, the contradiction is really there. When people get close to those boundaries, like the people working in ordinal theory, there is a real necessity for consensus, and a competition between interpretations. It is possible that some of what is accepted now really does lie too close to the boundaries where things break down and within the area ruled out by Russel's paradox and the resulting notion of 'proper classes'. Whether to consider it math sometimes remains a judgement call. We will know only when the building upon it proves useful or falls apart.
Likewise, there are competing relative concepts in math that are in direct competition. Our definition of measuring space and the set-theoretic Axiom of Choice are at odds. Together they give us Tarski's paradox. Whether that is a problem or just a quirk of mathematical reality is not an issue where there is consensus. Folks who think it is a problem have proven the axiom is largly dispensable, and not very much ultimately depends upon it. So they look askance at its continued use. Others have even proposed axioms that seem more useful and rule out this possibility, like the Axiom of Determinacy.
So your overall impression that all of mathematics is free of subjective judgement is not real. But large tracts of it are safe, without seeking consensus, because they are built upon inborn reactions and filtered by rather harsh standards before we start building on them.