This very issue is analyzed in detail in Quine's paper On What There Is, you may want to read it in full, it is short and freely available. Here is the key passage:
"We can very easily involve ourselves in ontological commitments by saying, for example, that there is something (bound variable) which red houses and sunsets have in common; or that there is something which is a prime number larger than a million. But, this is, essentially, the only way we can involve ourselves in ontological commitments: by our use of bound variables. The use of alleged names is no criterion, for we can repudiate their namehood at the drop of a hat unless the assumption of a corresponding entity can be spotted in the things we affirm in terms of bound variables. Names are, in fact, altogether immaterial to the ontological issue, for I have shown, in connection with „Pegasus‟ and „pegasize‟, that names can be converted to descriptions, and Russell has shown that descriptions can be eliminated."
This is argued against the proponents of "intentional objects", who say that talking about Pegasus would be nonsensical if Pegasus were not, therefore our very saying that "Pegasus is not" commits us to claiming that Pegasus is. Quine suggests that "Pegasus is not" can be paraphrased into Russell's definite description "There is nothing that pegasizes", which frees us from any commitments ("pegasize" is an "unanalyzable, irreducible attribute of being Pegasus"). On the other hand, if we wished to commit we could just say "There is something that pegasizes". Later Quine extended his analysis from languages to theories and developed the indispensability criterion: a theory is ontologically committed to those and only those entities that are needed as values of the bound variables for this theory to be true. He then argued along with Putnam that our current science commits us to the existence of abstract entities, like sets and numbers, because they are indispensable.
Quine's position is quite popular in modern analytic philosophy, but as everything in philosophy, it is not without critics. General critiques of descriptivism in establishing reference of proper names and natural kinds were championed by Kripke, and if descriptivism does not, in fact, adequately describe reference then it can be doubted that Russell's paraphrase is innocent with respect to ontological commitments.
An even more radical ontological disinvestment of language is known as fictionalism. As Nolan, Restall and West put it
“The simplest fictionalist approach to a discourse takes certain claims in that discourse to be literally false, but nevertheless worth uttering in certain contexts, since the pretence that such claims are true is worthwhile for various theoretical purposes”.
Berkeley's quip "we ought to think with the learned but speak with the vulgar" is seen as an early expression of this position in response to the criticism that his phenomenalsim runs counter to the "ontology" of common sense. Horwich and Williams interpret late Wittgenstein in this vein, arguing that statements only have "assertibility conditions" rather than "truth conditions", and therefore language can entail no ontological commitments whatsoever.
I am not sure that linguistic arguments concluding that something "must" exist would be considered reasonable even in today's metaphysics. A classical example is the "ontological argument", whose rejection by Kant on the grounds that "existence is not a predicate" is largely shared today.
