The first response to a question of the sort
what is the mathematical world limited to?
is: what do you mean by "the mathematical world"? If you wish to interpret mathematics not just platonically in a colloquial sense — i.e. that there exists an absolute mathematical truth because of [insert reasons] — but take an actually Platonic position that there exists an actual world of forms, which you might call the mathematical world or of which the mathematical world is a subset (together with chairs and horses) of which our world is but a poor shadow, then it cannot be limited by our world unless you make various assumptions.
If you will forgive my injecting some mathematical terminology, you might suppose that the projection from the Platonic realm to ours is a total function: that is, for all objects in the Platonic realm, there exists an object in ours which is its shadow. This would mean that the only things which exist in the Platonic realm are those which can be identified as an abstraction of our own world. (If you take this position, you should wonder how seriously you should take the idea that the Platonic realm isn't just an invention of your imagination to categorize the structures that you already see around you.) This assumption would then allow you to try and discern some limitations of the Platonic realm imposed by representation in the domain of our experiences. But then you are stuck with problems such as: when can you identify what Platonic object is being represented by a real object? Does it matter how crude a representation the real object is, of a Platonic object? If not, then does even assumption even impose any restriction on what sorts of objects might exist in the Platonic realm?
I find that your questions 2–4 are really different phrasings of the same question, and amount to commenting on the Unreasonable Effectiveness of Mathematics in the Natural Sciences. The short answer is that there's no real way to tell, except perhaps the reasons touched on by Bruce's earlier answer: that there exists some structure in the world, certainly enough to allow self-replicating agents to arise and to form complicated social animals; that mathematics is nothing more or less than the collection of formal tools which contain — as a subset — those patterns which we see arising around us.
Basically, the question amounts to what it is that you think mathematics consists of, and where you think it came from. Not where mathematical structures in reality came from, but mathematics itself. If you think that it sort of appeared mysteriously in human culture for no reason, like some sort of divine revelation, it is not really surprising that the relationship of mathematics to the real world would be mysterious. But if mathematics — and generalizing to all subject matter of the Platonic realm, all language and logic — arose as tools made by humans to try to encapsulate not only the patterns they see in the world, but all patterns that they could imagine, then perhaps the apparent presence of mathematical structure in the world is not mysterious; it's because mathematics consists of variations on some basic themes, which we once deliberately abstracted from the physical world.
What we observe is that the physical world is not only structured, but that the complication of that structure does not vary too wildly with scale — at least, not until we consider speeds close to that of light, masses greater than that of the Earth, or spatial scales close to those of atoms. Then we find that the structure of the world is still describable by math, but — quite significantly — that it seems badly counterintuitive; and in the case both of general relativity and quantum mechanics, unweildly and difficult to compute. Is mathematics truly effective there, if we can describe the apparent pattern of things but are unable to very efficiently verify that these patterns hold true by computing their predictions? Does this represent a qualitatively robust understanding, which is different in character than an understanding of Jupiter's cloud layers by a thorough examination of the colours of its atmospheric bands from beyond its orbit?
Mathematics is useful, and is perhaps one of the crowning achievements of humanity, but it is important to keep things in perspective. Developing a psychological theory of mathematics, as with language, is important to be able to arrive at an accurate assessment of why it is as it is, why it is useful, and importantly to be reminded of its limitations, which (in some subcultures of modern society) we are often keen to forget.
