The SEP article on inverted qualia discusses this mostly as follows:
One of [Frege's] theses in The Foundations of Arithmetic is that arithmetic is “objective”, which he explains as follows:
What is objective…is what is subject to laws, what can be conceived and judged, what is expressible in words. What is purely intuitable is not communicable. To make this clear, let us suppose two rational beings such that projective properties and relations are all they can intuit—the lying of three points on a line, of four points on a plane, and so on; and let what the one intuits as plane appear to the other as a point, and vice versa, so that what for the one is the line joining two points for the other is the line of intersection of two planes, and so on with the one intuition always dual to the other. In these circumstances they could understand one another quite well and would never realize the difference between their intuitions, since in projective geometry every proposition has its dual counterpart; any disagreements over points of aesthetic appreciation would not be conclusive evidence. Over all geometrical theorems they would be in complete agreement, only interpreting the words in terms of their respective intuitions. With the word ‘point’, for example, one would connect one intuition and the other another. We can therefore still say that this word has for them an objective meaning, provided only that by this meaning we do not understand any of the peculiarities of their respective intuitions. (1884/1953, §26)
This is an inverted spatial qualia scenario. According to Frege, the inversion would not show up in the semantics of words: both Nonvert and Invert use the word ‘point’ with the same meaning, despite associating very different “intuitions” with it.
Take a weakly multiversal standpoint about set theory and go to a local pair of axiomatized universes, one where there is the generalized powerset axiom and the choice axiom, the other where the natural powerset exists but is choiceless (because that axiom is omitted, there) and more acutely is not sufficiently well-ordered and so the choiceless continuum is coamorphous (properly amorphous sets cannot be sets of (all) reals and hence the continuum is not properly amorphous; however, in the choiceless light, it is shown to be thematically similar enough for us to refer to "coamorphism" as a relatively efficient term for the theme).
Imagine two sentient unicorns, we'll call them "Kripke" and "Charlie": Kripke perceives spacetime as a well-orderable continuum (even if primarily or even essentially in abstracto) while Charlie's continuous perception is coamorphous. Particularly, for one unicorn the clearest/intuitive notion of infinitesimals is generalized from an ur-infinitesimal 1/A, when A is more or less abstracted from countable cardinality. The other starts from 1/B, which is an amorphous infinitesimal; helpfully, 1/A/A... to A-infinity comes out to 1/AA = 1/C (for the continuum), so letting CA be the well-ordered version, then CB is a coamorphous (power)set of the naturals. Accordingly, Kripke's viewpoint inherits the relevant topological possibilities of CA, whereas Charlie's has CB-relevant topological options.
If spatial/temporal qualia could be so inverted, would the unicorns be able to tell that they had such inverted qualia, due to some (any?) topological "weirdness"(???) of the discrepancy between aleph-commensurable and aleph-incommensurable continua? (Being aleph-commensurable for a set means that e.g. the natural powerset has a version that is commensurable with the other alephs (and hence there is an aleph-continuum, after all, "somewhere"); or in a much more drastic case, per their transgression upon the choice axiom, then Reinhardt cardinals (an extremely large type of cardinal) are persistently suppressed or else there are sets whose cardinalities might not be properly well-orderable and so they would be rendered incommensurate in some way with the alephs.)
Or worse, suppose that a generalized continuum of vagueness relations, including higher-order ones, i.e. a set of vague ranges whose cardinality is the continuum's, would make the unicorns have problems of vagueness in their interpretation of the physical world, such that each had their own kind of vagueness (on account of the choiceless discrepancy) which, as sets, are incommensurable, and hence we might think that the unicorns would then not be able to properly compare (mutually interpret) each other's perception anyway, and they would never really know the difference after all? (For example, a (co)amorphic multiset might have continuum-many copies of CB in it, so when Charlie's perception is "informed" by multiset "imagery," the set-metatheoretic ordering provided by this "information" is qualia-theoretically inaccessible to Kripke (in the sense that there are prequalia, or conditions that constitute qualia), who can put multisets of the continuum (the multicontinuum) into a "standard" well-order, so his understanding of the concept of copies at that level will essentially differ enough to where either unicorn's very notion of continuous identity will be qualia-theoretically inaccessible to the other.)