Nonphysical entities cannot be observed. Therefore such entities cannot be verified by observation. How could statements like "God exists" be even considered true? Why would anyone appeal to the metaphysical realm at all?
What you are describing here is the position known as positivism, which later evolved into logical positivism (also know as logical empiricism). One of the earliest and most famous formulations such a position was given by David Hume, in what became known as Hume's fork:
If we take in our hand any volume; of divinity or school metaphysics, for instance; let us ask, Does it contain any abstract reasoning concerning quantity or number? No. Does it contain any experimental reasoning concerning matter of fact and existence? No. Commit it then to the flames: For it can contain nothing but sophistry and illusion. -- David Hume, An Enquiry Concerning Human Understanding (1748) Section 12 : Of the Academical or Sceptical Philosophy Pt. 3
The Logical Positivists continued Hume's position, and were famously anti-metaphysics, which they considered to be a derogatory term. For them, the only statements that had any meaning were those that could be verified empirically, or analyzed using logic See the SEP article on Logical Empiricism - Section:4.1 Empiricism, Verificationism, and Anti-metaphysics, and the following quotes by notable Logical Positivists A.J Ayer and R. Carnap:
The criterion which we use to test the genuineness of apparent statements of fact is the criterion of verifiability. We say that a sentence is factually significant to any given person, if, and only if, he knows how to verify the proposition which it purports to express — that is, if he knows what observations would lead him, under certain conditions, to accept the proposition as being true, or reject it as being false.
-- A.J. Ayer , Language, Truth, and Logic, p. 16.
According to this view, the sentences of metaphysics are pseudo-sentences which on logical analysis are proved to be either empty phrases or phrases which violate the rules of syntax. Of the so-called philosophical problems, the only questions which have any meaning are those of the logic of science. To share this view is to substitute logical syntax for philosophy.
-- R. Carnap, Logical Syntax of Language, p. 8
For the logical positivists, the end result such reasoning is that Philosophy should be just a linguistic tool to help analyze the statements coming from empirical sciences. Questions of metaphysics, philosophy of religion, etc,..but also of ethics (per your original question, ethical principles are also unobservables), aesthetics, etc...are strictly speaking, non-sense. See also A.J Ayer's emotivism.
A major problem with such a position is that it is self-defeating. Consider Hume's statement about throwing away anything that isn't either logic or empirical science: By his own reasoning, his own book "An Enquiry Concerning Human Understanding" should be itself committed to the flames.
Alternatively, we could state this problem in Logical Positivist terms:
The above mentioned statement by A.J Ayer
"The criterion which we use to test the genuineness of apparent statements of fact is the criterion of verifiability."
itself fails the criterion of verifiability -- there is no way to verify such a statement.
This points to a larger problem that Logical Positivists faced, and that is the fact no matter how hard one tries, it is impossible to completely separate empirical statements from the theoretical presuppositions they are based on.
As Mozibur Ullah mentions in his answer, everything is theory laden (see this post and answers within). W.V.O Quine, who considered himself a Logical Positivist who had worked with Carnap and Ayer, pointed out this problem of the Logical Positivist stance, and also offered a pragmatic solution out of the problem with his confirmation holism (a pragmatic view of science: science is a useful tool that works, religion and other forms of metaphysics aren't) in his 1951 paper "Two Dogmas of Empiricism", which he concludes with the following statement:
As an empiricist I continue to think of the conceptual scheme of science as a tool, ultimately, for predicting future experience in the light of past experience. Physical objects are conceptually imported into the situation as convenient intermediaries not by definition in terms of experience, but simply as irreducible posits comparable, epistemologically, to the gods of Homer . . . For my part I do, qua lay physicist, believe in physical objects and not in Homer's gods; and I consider it a scientific error to believe otherwise. But in point of epistemological footing, the physical objects and the gods differ only in degree and not in kind. Both sorts of entities enter our conceptions only as cultural posits. (emphasis mine)
Finally, I don't have references or quotes to back this up, but another argument in defense of studying metaphysical unobservables is the following:
- Democritus' atoms were definitely unobservable metaphysical objects back in his day. Yet the atomic theory of matter is firmly grounded in observable physics now. If we were to subscribe to a logical positivist stance, we would be depriving ourselves of useful metaphysical exploration that might lead to future developments in science.