Let me throw my two cents here. I am not claiming to have answers to your questions. I have just some remarks of methodological nature and some references that you may find useful.
I think that you can only speak about precise definition (or lacks of one) of some entity in the context of some fixed system S. So when discussing notions of a physical/abstract/mental object you should do it only within certain S. Let me explain.
Suppose that you built a rigorous (or even formal) system S of (some part of) knowledge. The utility of having such a system is I think evident - without such system we are mixing various meanings, get involved in circular reasoning and get confused all the time. Science is some sort of approximation of such a system, but rather far from ideal. It is not a critique because science's main concern is to clarify or organize our knowledge, but to expand its limits.
Note that in process of building S you want to construct clear terminology. The basic methodological observation is that you have to start from some notions that are undefined and hence primitive. That is precisely the idea Plato expressed in Theaetetus:
Socrates: Let me give you, then, a dream in return for a dream:—Methought that I too had a dream, and I heard in my dream that the primeval letters or elements out of which you and I and all other things are compounded, have no reason or explanation; you can only name them, but no predicate can be either affirmed or denied of them, for in the one case existence, in the other non-existence is already implied, neither of which must be added, if you mean to speak of this or that thing by itself alone. It should not be called itself, or that, or each, or alone, or this, or the like; for these go about everywhere and are applied to all things, but are distinct from them; whereas, if the first elements could be described, and had a definition of their own, they would be spoken of apart from all else. But none of these primeval elements can be defined; they can only be named, for they have nothing but a name, and the things which are compounded of them, as they are complex, are expressed by a combination of names, for the combination of names is the essence of a definition. Thus, then, the elements or letters are only objects of perception, and cannot be defined or known; but the syllables or combinations of them are known and expressed, and are apprehended by true opinion. When, therefore, any one forms the true opinion of anything without rational explanation, you may say that his mind is truly exercised, but has no knowledge; for he who cannot give and receive a reason for a thing, has no knowledge of that thing; but when he adds rational explanation, then, he is perfected in knowledge and may be all that I have been denying of him. Was that the form in which the dream appeared to you?
By the way, Theaetetus is exactly the dialogue, where Plato discusses the knowledge.
Next I think that there are some reasonable criteria for constructing S.
Consistency - S should be consistent.
Expressibility - S should be designed to explain fixed set of input data. Ideally it should be complete i.e. explain the totality of intersubjective data available.
Economy - the collection of primitive notions of S should be as small as it is possible with regard to expressibility. That is just an invocation of Occam's razor.
These three conditions impose certain limitations on the choice of primitive notions of S.
Now there are two famous classics in modern philosophy which discuss and rigorously construct certain systems of knowledge (or at least attempt to do so). They are chiefly focused on defining notions (so called constitutional systems) and choosing appropriate collection of primitives. These are:
- The Logical Structure of the World by Rudolf Carnap
- Structure of Appearance by Nelson Goodman
Moreover, they both attempt to do it from phenomenalist/nominalist perspective. There is also less known but great idea of reism coming from Warsaw-Lvov school of philosophy.
Edit.
I was thinking about your question for a while and aside from previous metaphilosophical remarks, there are some other (rather vague) observations that are better suited to address your original question.
There was (and still is) an intellectual temptation among philosophers (Descartes is a prominent example, Berkeley is another, Chalmers is one of the most prominent examples nowadays) to consider mental objects as primitive ones. It comes from the strong intuition that we have access to the outer world mediated through our impressions, thoughts, whereas on the other hand we have direct access to our inner-self. These entities are precisely what we call mental objects in our colloquial language. I can't say that there is a similar intuition with respect to physical objects (consider electrons, curvature of spacetime, wave functions), but that is just my opinion. Nevertheless it is not unjustified to assume that mental objects are more primitive cognitively than physical ones.
Among abstract objects there are mathematical ones. Since all physical models constructed so far are expressed in terms of mathematical objects, it seems that in order to give a satisfying definition of physical object one needs first to define mathematical objects or accept them as primitive. This is another serious objection to physicalism c.f. Susan's Schneider article. It also supports a view that abstract objects are more primitive than physical.
I think that those are some reasons why defining physical objects is a more serious issue, than giving a precise definition of mental or abstract.