Preamble -- "Naturalism" is not a clear concept
As a start to this discussion, Oppy's definition of naturalism is seriously flawed:
Naturalism is the claim that there are none but natural causes, beings and forces. Naturalism entails that all causally efficacious beings and forces are located within the natural world. As noted above, naturalism is inconsistent with theism
Methodological naturalism, which is just to apply reasoning and empiricism to understand our world, is NOT inconsistent with theism. Theism is a possible consequence of applying methodological naturalism to our world, and that is how the world's major religions were originally developed.
Oppy seems to be trying a bait/switch. MATERIALISM, which is an ontological claim, is incompatible with theism. However, naturalist tests of materialism showed it to be untrue, in the physics breakthroughs of the early part of the 20th century. Matter is not fundamental.
Wish-it-were-so materialists have mostly rebranded themselves as physicalists. But physics is not an ontology, and the process of DOING physics requires presuming the reality of and causation by world 3 (referencing Popper's three worlds) relationships, processes and phenomena. Energy and entropy and curved space-time are all abstractions, rather than world 1 objects, so physics cannot be a monistic ontology. And physics cannot justify itself, requiring the prior validity of epistemology in philosophy. And physics is not and cannot be closed, as noted in Hempel's Dilemma. Therefore, physicalism cannot exclude causal deities. Daniel Stoljar made this point in his book-length justification of his walking away from physicalism: https://www.amazon.com/gp/customer-reviews/R13R2OUNXMIN6H/ref=cm_cr_dp_d_rvw_ttl?ie=UTF8&ASIN=0415452627
If Oppy means physicalism by his claim that "naturalism" is contrary to theism, this claim is wrong (per Hempel, etc. above). If he means some other ontological claim, then he has to do a lot more to articulate what "naturalism" means, and show that it is actually a valid and coherent claim. Given the problems that "physicalism" has as an ontology, the possibility that a more general epistemological methodology could somehow transmogrify into an ontology does not look promising.
Without Oppy spelling out his views in a useful way, I will try a substitute. I will take Oppy as debating Spiritualist pluralism (that of the ontology of our universe, there is a substance or plane which supports non-physical conscious agents such as souls or Gods) vs. anti-spiritualist pluralism (that conscious agents may exist, but if they do so, they are always dependent upon a world 1 or world 3 substrate, such that causation is always ultimately traceable to a non-world 2 aspect of reality and there are no independent spiritual agents).
Evaluating spiritualism vs anti-spiritualism
Oppy offers an interesting and potentially useful metric for evaluating spiritualism vs anti-spiritualism, with his 'theoretically virtuous' standard. Lets try to walk thru his criteria with the redefinition above about what we are evaluating.
One issue I will repeatedly encounter with Oppy's paper, though, is that he prioritizes "simplicity" over predictive power in his approach to a virtues metric. But the "brute fact" leg of Munchausen will always be "simpler" than walking part way down the infinite series of explanations. Methodological naturalism/science ALWAYS prefers to walk down the infinite series leg as far as one can, despite the increased complexity this leads to (our scientific explanation of our world is astonishingly complex, such that no individual can fully comprehend even one sub-field of it). Science and methodological naturalism vastly prioritize explanatory power over simplicity, contra Oppy. Rather than simplicity, methodological naturalism prioritizes the predictive power of an explanation, based on its ability to be falsified (this is Popper's rewrite of Occam). I will reevaluate his criteria based on Popper's explanatory power/falsifiability criteria, rather than Oppy's simplicity.
I will also note that Oppy uses a very broad definition of "naturalism" (the claim being evaluated is equally broad with my recast to pluralist anti-spiritualism), and this collective category of claims includes moral realism, math realism, emergent phenomena, causal consciousness, and multiverse explanations for the origin of our universe. This collective of views is NOT any simpler than spiritual dualism, as my review below of a multiverse thesis vs a Godly Creation thesis shows for the origin of the universe test case, and as an examination of what is needed for emergent consciousness to be morally relevant efficacious causal agents does for the causal consciousness test case. Between the vast prioritization of explanatory power in actual science and methodological naturalism, plus an at least close wash on complexity (I think a case can be made that interactive spiritual dualism is logically simpler than the Popperian emergent dualism that I credit anti-spiritualism with -- interaction laws are needed in both cases, but emergence laws are needed too for the anti-spiritualist proposal), I ignore "simplicity" and just focus on explanatory power/testability.
3.1 Ultimate Explanation
While this is a useful question to ask, Oppy's treatment of it is seriously flawed. Per the Munchausen Trilemma, we CANNOT arrive at an "ultimate explanation", so scoring competing worldviews based on their inability to do so, is an impossible standard fallacy.
Oppy then resorts to his "simplicity" argument that we have "a global (efficient) causal order" that applies as a minimum to matter, consciousness, and abstractions "just because" is simpler than because a spiritual agent structured the universe that way. However, the real question is whether it is more or less explanatorily useful than that we have such an order because a conscious spiritual agent WANTED our universe to be ordered, or "just because"? The teleological answer is potentially predictively testable, so is superior per Popper's predictive rewrite of Occam. Whether it is actually usefully testable is an open question but in principle it is a more virtuous answer and we have one score for spiritual agency.
3.2 Order
Here Oppy is evaluating the Fine Tuning of our universe. He leads in with a preamble that Fine Tuning is in dispute, but this is not really the case among cosmologists. Oppy then peculiarly switches to the presumption that the universe may have become fine tuned at some point, vs. always being so tuned. But the cosmological alternatives to Fine Tuning (multiverse plus anthropic principle) all agree that our constants were set at the start of our universe, so the relevance of the question he then focused on is completely unclear. Both competing models hold our universe was fine tuned from the outset -- and both presume a pre-universe cause to our universe. His criteria does not sort.
One again, he invalidly argues based on simplicity, but he seems to not be aware of what the complexity is of the multiverse models. This review of Susskind's the Cosmic Landscape will highlight the complexity, and similarity to theist assumptions, behind multiverse thinking: https://www.amazon.com/gp/customer-reviews/R3JVQDAK1408BR?ref=pf_ov_at_pdctrvw_srp
When one looks at predictive power, these two approaches may be a wash with a small possible bias toward theism. Much of both models are not testable, but per interactive spiritualism, more of God claims are testable in principle than multiverses, which are so logically isolated from us that most aspects of them are intrinsically untestable.
However, the testing that HAS been done, leans toward the multiverse which predicts a tuned but not optimized universe, while theism predicts optimization for life. And with life possible in a vanishingly small fraction, less than 0.000001% of our universe, this universe is FAR from optimized for life.
In my judgement, this criteria is a wash, between the higher testability of spiritualism, vs the few tests leaning toward anti-spiritualism.
3.3 The Necessary and Knowable A Priori
Here Oppy discusses math, logic, modal concepts, morality, etc. -- all of world 3. He is presuming world 3 is knowable a priori, which I at least consider clearly false, as logical pluralism demonstrates. Whether these abstractions apply to our universe is an empirical question, not one knowable a priori.
Recasting this question from a priori to empirical arguments for logical, math, moral, etc. reality does not change the scoring of this issue. The simplest versions of materialism/physicalism are not compatible with the reality of these abstractions, but I have already recast Oppy's argument as being for a more complex ontological pluralism that allows for the reality of each of the above, but just denies independent spiritual agency. The reality of each of the above is an open question in both models, and neither is more predictive than the other. This subject is a wash.
3.4 (Objective) Value
Here Oppy takes a peculiar detour into necessitarianism, which he ascribes to both views, but which methodological naturalism intrinsically rejects. He cites the problems of Euthyphro for theistic moral necessitarianism, which is valid as far as it goes, but as we saw with Munchausen, inescapable contradictions are sometimes something we pragmatically must accept.
The main issue here is that spiritualism lends itself more to accepting that morality is real and matters than anti-spiritualism does. This leads to the disquiet among those who have discarded religion cited by Charles Taylor in A Secular Age. Is disquiet over a hole in one's worldview a useful test of competing worldviews? For a pragmatist, who thinks a major purpose of philosophy is to figure out how one should live -- my answer is a definite "yes". This subject is therefore a second win for Spiritualism.
3.5 Meaning
This subject is often mistakenly coupled with the moral question above, but it has a different outcome. My treating his anti-spirituality as accepting conscious agents is key here, and once one accepts conscious agency, Oppy legitimately cites the creation of meaning by existentialists as a legitimate source of meaning.
In symmetry to anti-spirituality, where most anti-spirituality is incompatible with meaning but a few versions are compatible with it, meaning is also an issue for most monotheism. In absolutist monotheisms there is little space for we humans to create valid meaning as our agency is irrelevant in an environment of an infinite and active agent. But there are also a few spiritual models that allow for human mattering, by limiting God and putting God inside time, so this issue is a wash.
3.6 Consciousness
Oppy very over-optimistically assumes that physicalism can (and actually has) answered the hard problem of consciousness -- contrary to the consensus of pretty much all other philosophers. But his anti-spirituality includes emergent dualism, which allows for causal consciousness, and because of that our observations of causal consciousness are a wash.
3.7 Reason
The existence and utility of higher reasoning is possible due to an evolutionary process due to the causal order of 3.1, and this is true for both spiritual and anti-spiritual worldviews. Oppy once more invalidly argues for a "simplicity" advantage for anti-spirituality, but under explanatory and predictive power, this is a wash.
3.8 Supernatural Experience
Relative to direct experiences of ghosts, the voice of God, guidance by spiritual guides, near death experiences etc, Oppy basically denies that these things happen. With approximately 30% of Americans reporting such direct experience, Oppy's denial comes across as just a regurgitation of dogmatism. The successful history of parapsych experiments of telepathy, remote viewing, animal communication, etc. demonstrate that spiritual interactionism makes useful testable predictions, which actually are confirmed. Denial of evidence is always anti-empirical and anti-methodological-naturalist, and is explanatorily refuted, so this criteria is a third win for spirituality.
3.9 (Supernatural) History
Here Oppy once more denies the evidence, where historical reports of miracles are no less supported than historical reports of non-miracles, but he dismisses the miracles in a special pleading fallacy. Oppy correctly points out that the historical evidences for spiritual entities support contradictory worldview narratives, hence it is difficult for all of them to be true simultaneously, but this is not a legitimate rationale to dismiss all of these evidences.
An interesting test of the history evidence is the convergence test. Why has religious evidence not converged on the One True Religion? A possible answer is that religious experiences are a combo of expectations and real spiritual events, which we have to interpret. And we humans evolved to be an anthropological species (small groups of hunter-gatherers) not a sociological one. But with the development of agriculture, we also developed sociology. And the growth of additional technologies has not left sociology stable long enough for the spiritual experiences to converge on the One True Religion. This speculation would propose that we DID have such a convergence when we were anthropological and the extreme similarities of Shamanism in anthropological settings around the world support this speculation. This example of testability of the spiritual hypothesis around the historic record shows a fourth dramatic win under the theoretical virtue standard for the spiritual model over the anti-spiritual one.
Wrap-up
I rephrased the comparison of worldviews that Oppy was evaluating into spiritual pluralism vs emergent anti-spiritual pluralism -- to be as generous to Oppy as possible. I also reevaluated theoretical virtue using Popper's testability/predictive power criteria, not Oppy's "simplicity". Once I did that, Oppy's 9 criteria he scored for theoretical virtue show 4 wins for spiritual pluralism, and zero for anti-spiritual pluralism.
Oppy's metric is worth looking at, and once the flaws that Oppy was applying to it are corrected, it shows the opposite of what Oppy claimed.
Addendum and caveat -- Oppy used his metric to evaluate justification for theism that others have argued lead to theism being a valid conclusion. He was running a conservative test, against the preferred data cited by his opponents. The point of a conservative test is that if THAT fails, then an unbiased test is not necessary. Oppy ran the test using the wrong criteria, and therefore incorrectly thought he has showed that Spiritualism was not credible.
My positive result doing the test correctly still faces a caveat. This list of issues was biased in favor of Spiritualism and theism, and the conclusion I reached, that this method supports spiritualism and theism, may be much less strongly supported by a more balanced list of evidence to examine. To do this right, a collaboration of theists and atheists should put a more comprehensive list of evidences to examine together, and then do this evaluation collaboratively too.