I think if anything this question is an experimental one. There are some studies about how well animals are dealing with numbers. I have read something about pigeons being able to count up to some small number. Here is some link supporting that.
From this one can argue that some concept of numbers seem to come pretty natural. I have also heard of some studies that show some insight of how numbers are perceived by humans or what people with some specific brain damage can still do in mathematics and what not. Also, I am pretty sure I have read that people after some accidents couldn't understand most words, but numerical words as "seven, one, two-billion-and-five-hundred-twenty-four" were understood. The author then suggested, that it seems that we deal totally different with "numerical words" than with other words. Numbers seem to be memorized somewhere else in the brain than say names of animals. I am sorry that I totally forgot where I read that, but anyhow, I am sure you find a lot of similar studies after googeling for a while. The/One related branch is explained here.
So from what I have seen I would say one can confidently claim that naive mathematical understanding (counting small numbers, comparing numbers) evolves rather soon and (all?) somewhat sophisticated animals seem to share that. - Oh and the same goes for children, see also in the Wikipedia link.