The first question a mathematician looking at a new concept is not to ask whether it exists but how it proves itself useful. The differential calculus made it easy to write down the equation of motion of Newtonian Physics and Descartes coordinate geometry made it possible to formulate the same equations in a natural way.
This might be called the pragmatic answer, in the sense we think of it as a tool to predict, describe or solve; or a coherent answer, which rejects the (metaphysical) reduction to a foundation, (philosophically this view is called foundatinalism) to a set of beliefs (axioms) that are secure or justified, and is non-inferential ie axioms.
So how do sets prove themselves useful in mathematics. This is connected with Hilberts programme to reduce mathematics to logic, and one is lead to this thought when one considers that sets are aligned with logic, consider Venn Diagrams and observations that all the integers for a set and so on.
A different approach, called Platonism asserts the existence of any set of axioms that is internally consistent. This draws upon Platos theory of Forms which gives existence to pure ideas. This is essentially a metaphysical supposition, and evidence of such a position can only be given in the most indirect kind of way.
Another position that came out of the early work on mathematical logic is Formalism which drops any need to ground a theory in some Platonic world and instead observes that consistency is all that is important when it comes to manipulating axioms and theorems. This is actually grounded in logic, and observes that a mathematical theory can be seen as simply as a set of strings that are rewritten according to some set of rules.
Finally, the analytic or positivist approach that Rostomyan mentions with regards to Carnap, and which came out of a philosophy of science in Vienna, also dispenses with metaphysics by rendering them 'meaningless' - the metaphysical voice is silenced.
A standard set-theoretic way of defining the natural numbers[1] 1,2,3,... is based on the empty set
Note that we can start from any other set, the set of cups for example: Define 0:=Cups, 1:={0}, 2:={1} etc. Of course the void set was chosen for two reasons, it is a given, and also it is the minimal set that is included in any other set. Another point to notice is that we can say 2 is a member of 3 which doesn't make much sense. Finally note we can just define the natural numbers as a ring with certain arithmetic properties or via the Peano Axioms.
As one notes from this, there are quite a few distinct positions and which position one adopts is most likely a function of ones own inclination as the schools of thought one is aligned with.