I'm going to kind of ad lib here, but as a formalist, hopefully at least something of interest comes out of this.
For a formalist, Semantics is how mathematics gets its practical application. When I say something like "one plus one equals two", I am discussing something logical and syntactic, but when I say "one orange plus one orange equals two oranges", I'm discussing oranges. If I have some theorem about functions over the domain of a three dimensional vector space, I'm talking logic and syntax; if I say that this theorem allows me to show a feature of how I can manipulate the orange in physical space, the applied theorem is telling me about something cool I can do with oranges.
Of course, you can also do mathematics on syntax, which is to say that we want to establish an interpretation of some of the logical/mathematical linguistic symbols as logical/mathematical linguistic objects. There is a particularly interesting domain of mathematics which is just this - Logic.
However, a legitimate criticism of our formalist programme is that it's not entirely clear whether any given theory of logic is a correct account of the cognitive resources people use when processing things using a single language module. It seems accurate to say that human linguistic understanding is messy, activating many different regions of the brain, and the simplification of language to logic doesn't necessarily give us many clues about how best to use our brains to do mathematical reasoning.
Nonetheless, it seems like a much more fruitful theory than something like Platonism, for whom the objects of mathematical reasoning and the epistemological tools we use to grasp them seem decidedly magical. "Use the language bits of our brains" is at least a potential prescription towards improving, even if we might revise our understanding of what we can do with maths more generally once we've shown that this basic bit works.
