I just asked, what I believe is, a similar question recently about what you call "inconvenient formulas" in possibilist modal logic. That is, formulas that attribute properties to an object that doesn't actually exist at the world the formula is being evaluated at, but to avoid truth value gaps, must have an interpretation. Like you, I believe that logic should be metaphysically neutral (or at least that's what I think you are saying). With quantified modal logic this seems especially difficult since there are at least two different approaches to semantics, possibilism and actualism. Possibilism (in modal logic) is roughly the view that there is one domain across all possible worlds, and all objects in this domain exist in some sense in all worlds. The referent of a name is fixed across all worlds, as the possible object which it picks out in this one domain. Quantified statements are statements about all objects in this single domain. Actualism (in modal logic) is roughly the view that there is a domain for each world which can vary, so an object which is in the domain of one world may not be in the domain of another. Quantification is world-relative, only about the objects that exist in the domain for that world. Depending on which approach you take, certain formulas that are valid on one approach (Barcan Formula, Converse Barcan Formula, and the Necessary Existence Formula in possibilism) are not valid on the other. So, it seems the choice of semantics here does make a difference to the truth of some formulas, and to valid deduction.
On the other hand, some philosophers argue that possibilism is the simpler and more general approach to the formal semantics, since actualist reasoning might be adequately captured by possibilist semantics. In "In Defense of the Simplest Quantified Modal Logic", Linsky and Zalta argue that there are readings of "inconvenient" possibilist formulas that should satisfy actualists, as long as they are willing to regard the troublesome objects as non-concrete objects, instead of possible objects ("non-concrete" means something like "abstract", but only contingently so). In this way, "the possible red pen on my desk" is read as "the non-concrete possibly red pen on my desk". It doesn't even have to have the property of being red (though it is possibly red), since it's non-concrete, and non-concrete objects don't possess properties such as being red, having spatial or temporal locations, and mass, just like how abstract objects can't have these properties either. Or, maybe they can (if your metaphysics permits), and so the non-concrete red pen on my desk might actually be red. The choice is yours, and the formal semantics doesn't dictate the answer one way or the other.
I don't really know the rules of this site too well, so I'm not sure if this paragraph is too subjective. But personally, I've sort of just accepted possibilism and the simplest quantified modal logic as a "default" logic for reasoning about metaphysics, and I just think of it as the logic of possibilities and possible objects. As such, we need to be able to refer to possible objects in some way, just like how in probability we need to refer to alternative outcomes of an experiment (ie. elements of a sample space). I think this approach to possibility is kind of like doing mathematics without settling questions about the philosophy of mathematics. Whatever ontology of numbers, outcomes, etc., you believe in, you still talk about numbers, outcomes, etc., "existing" in first-order logic formulas ("there exists a natural number x, such that..."). But whether such a number really, metaphysically, exists is a different question that we can hope to ignore for as long as possible while still doing math. So, even though we might talk like a possibilist regarding the existence of possible objects, metaphysically we might not believe in them. Or, à la Linsky and Zalta, believe in their existence as non-concrete entities. Or as something else.
All this flexibility might be a good thing as it gives us a lot of choice in how to metaphysically interpret modal logic formulas, but doesn't seem to commit us forcefully to any one position.