Do the expressions of pure mathematics express anything about objects?
i.e. are the bearers of mathematical identity (e.g. the being number one of that number) themselves objects?
Do the expressions of pure mathematics express anything about objects
There is a wide-ranging field of mathematics that does talk 'objects'. This is category theory, it actually arose from what would be usually termed pure mathematics - algebraic topology. Philosophically, one can't understand it without comparing it to set theory; whereas set theory posits sets themselves as the basic ontological entity with relations as a secondary, derived concept; category theory takes relations as the basic ontological concept with sets as derived.
However, as this leads to rather round-about circumlocutions it's generally taken to have a two level ontology, objects and relations (they actually call them arrows or morphisms - but here I wanted to clarify their conceptual status).
One might think this is merely a transposition of nomenclature; I mean, a set is called an object, and a function is called an arrow - so one might ask, has anything new been added here; sure, it has led to conceptual clarifications in mathematics - which points to at least one philosophical point about mathematical logic, and also about set and category theory; this is - before it is about logic or ontology, it is about mathematics.
Given the slant of the question it's worth adding that sets are extensional, this means merely that it is constituted by its elements; objects are not like this - remember, they aren't sets; and to discover what they are one has to 'probe' them; if one is interested in discovering their 'elements' (if they have any) one probes them by a 'point' (a zero object); but they may have further structure which isn't apparent if one merely probes with the point; a good example here, is the real line equipped with its topology; if one probes by the point one discovers all the real numbers - the points of the real number line; but by this method one can't discover the topology; and the topology is important, it's what makes it what Weyl called it - a continuum.
are the bearers of mathematical identity (e.g. the being number one of that number) themselves objects.
This is the crux of your question, and unfortunately nothing I have said so far - be it about set or category theory - sheds real light on this question; what light they shed, is on mathematics per se, and not on ontology - though note I used the word ontology above - this was merely a matter of convenience - and was said to posit what it is I was talking on or about, and to define, roughly, their dependencies and inter-relations. It's not ontology in the authentic ontological sense proper to it.
I suppose one conceptual clarification is that when a someone says such or such a number exists what does he mean? He might mean several things depending on what philosophical school he may adopt - formalism, platonic, intuitionistic or nominalism; perhaps it might mean you ask your question in relation to these schools of thought; for example, it's worth noting that formalism is a species of nominalism; one might object to this, and say, given the lack of philosophical consensus that formalism settles on meta-mathematical principles about what we must/ought to agree on about the mathematical entities per se; and one could argue, from this view, formalism isn't nominalist but merely evades the ontological question and not as an evasionary tactic, but as a way to advance the subject by clarifying the philosophical position.