I am not sure what modern philosophers have written specifically about this, but I can offer a few comments.
One can understand a concept like 'set' as an attempt to formalise, or at least make rigorous, a pre-theoretic or naive concept. We start with the naive idea of a collection of things. Then we wish to make this concept of a collection rigorous by writing down some rules, so we can say with certainty whether a given inference about it is correct or not. At this point our naive understanding informs our choice of rules, and the rules may in turn reveal interesting features of the concept that we hadn't previously considered. Hence the interplay you asked about.
But we can easily run into problems. An early attempt by Frege to write down the rules for sets ran into Russell's paradox. This was a sharp wake-up call for mathematicians, because previously it had been assumed that mathematics was just obviously correct and yet here was an 'obvious' piece of mathematics that entailed a contradiction. This was one of the motivations for the logicist project of attempting to reduce all of mathematics to formal logical systems.
Rigorous definitions and systems also commonly give rise to edge cases, and these can be unintuitive. Most mathematicians consider that the empty set qualifies as a set. It is certainly very convenient to allow it. If we didn't, all of our rules concerning sets would become much more complicated because the empty case would need to be treated as an exception. But we don't ordinarily think of emptiness as being a collection. Come to that, we don't ordinarily think of zero as being a number. Suppose I say, "I have a number of coins in my pocket," and you ask me, "How many?" and I reply, "I don't have any coins, but zero is a number". You would rightly think I was messing with you. But it is highly convenient to treat zero as a number and the vast majority of mathematicians do so, though actually not all.
The resulting rigorous account may only approximate our original naive ideas, and it may include unintuitive edge cases, but as long as it is successful and useful, we may eventually progress to the point where we regard it as having completely replaced our naive conception. Although even then, it may still make sense to ask whether our formalisation is the best one possible, and whether some alternative might have desirable epistemological properties. ZFC was not the end of set theory, though it is the most commonly used. We also have NBG, New Foundations and many others.
At the end of the day, if a theory, mathematical or otherwise, is to be useful then it will have an interpretation that will allow us to say of some propositions that they are true or false. Many theories have an intended interpretation and this constrains the choice of axioms and rules. So the syntax is not arbitrary in that it must ultimately answer to some practical use, while the semantics does not float free of the syntax because of the desirability of having rigorous formalisations that are amenable to formal proofs. Each continues to inform the other.
