# Is probabilistic modus tollens a fallacy?

Modus tollens takes the form of "If P, then Q. Not Q. Therefore, not P."

A probabilistic version of Modus Tollens says "If P, then Q is very improbable. Q. Therefore, P is very improbable". Elliott Sober, in his paper against the design argument, describes this as a fallacy. See the paper here: http://philonantes.free.fr/ElliottSober_IntelligentDesignAndProbabilityReasoning.PDF. For example, there is a 1/52 chance to get a seven of hearts from a deck. Suppose you do get that card. This does not imply it is improbable that this occurred by chance.

The problem is that significance testing which is at the basis of modern science ultimately relies on this. A certain probabilistic threshold is chosen below which it is determined that chance did not play a role in this outcome.

He further argues that evidence is comparative. No matter how improbable a certain observation O is under a hypothesis H1, we cannot say that H1 is improbable if O happens unless we can show that some other hypothesis H2 confers a higher probability on this outcome.

Intuitively though, I can imagine a scenario that is extremely unlikely under H1 (say chance) and it would arguably still make sense to dismiss H1 even if we don't know what the probability of that observation under a different hypothesis H2 would be.

For example, if it was a rainy day and I started seeing water droplets on my window spell out the words “Hello, you are not alone” I would understandably and very obviously think that this was not done by chance or any blind naturalistic process. This would be true despite not knowing what the probability of this occurring is under another hypothesis, say God. I would not know what God would do and what the probability of this happening would be under God. Yet it doesn’t seem to be a fallacy to state this did not happen by chance by simply realizing how absurdly improbable it is for water droplets to form that sentence by themselves.

So is this a fallacy or not?

• The fallacious probabilistic modus tollens that Sober refers to has the form: If P then probably Q (if a lottery is honest you probably won't win), not Q (you won); therefore probably not P (the lottery is probably dishonest). The valid form used in significance testing is: If P then Q (if the null hypothesis then no significant observations); probably not Q (observations are probably significant), therefore probably not P (the null hypothesis is probably false). The implication premise must be non-probabilistic, see Widaman. Feb 27, 2023 at 7:15
• @Conifold That is just incorrect. If P then Q would imply that if the null hypothesis is true, there would be no significant observations. Q would have to deductively follow. But the null hypothesis does not necessarily imply no significant results. Fisher’s significance testing IS ultimately a probabilistic version of modus tollens. In your example, it would be “If P, then probably Q” if Q = no significant observations
– user62907
Feb 27, 2023 at 9:36
• Read the link:"To convey statistical testing properly, the first premise must remain categorical (i.e., unqualified) and the second premise is the one that is probabilified. Researchers never argue that, if their theory is correct, they will probably observe a predicted pattern in data. Instead, they argue that a predicted outcome will occur if their theory is correct. Stated conversely, if the null hypothesis is true, then the theory-predicted outcome will not occur." This "problem" has been discussed to death for decades. Probabilified implications occur in Bayesian inference, not here. Feb 27, 2023 at 9:56
• @Conifold and I’m saying that’s simply incorrect. If the null hypothesis is true, the theory-predicted outcome can and does occur. It may be very improbable but that’s literally what the probabilistic version of modus Tollens is.
– user62907
Feb 27, 2023 at 15:21
• Are you even familiar with significance testing? The null hypothesis is that the theory predicted relationship does not exist and the outcome is purely random. Feb 27, 2023 at 21:18

"If P, then Q is very improbable. Q. Therefore, P is very improbable"

Yes, that is invalid, as you can see by substituting specific propositions for P and Q. For instance: "If it's not raining, then I'm unlikely to pick a seven of hearts at random from a deck. I picked a seven of hearts from a deck. Therefore, it's probably raining."

The problem with that argument is that you're equally unlikely to pick the card if it is raining. The data doesn't provide any support for the hypothesis that it's raining over the hypothesis that it isn't. That's the point of the paper.

For example, if it was a rainy day and I started seeing water droplets on my window spell out the words “Hello, you are not alone” I would understandably and very obviously think that this was not done by chance or any blind naturalistic process. This would be true despite not knowing what the probability of this occurring is under another hypothesis, say God.

If you think that this supports hypothesis G over hypothesis C, then you think that the event is more likely given G than given C, even if you can't supply specific numbers. Whenever someone rejects the null hypothesis in a paper, they have an alternate hypothesis in mind under which the outcome is more likely.

Why did you choose for your example water droplets that seem to spell out words, rather than water droplets in this precise arrangement? That arrangement is much less likely to appear, per the null hypothesis, than some arrangement that can be interpreted as spelling out those words. What makes it a bad example is that you can't think of an alternate hypothesis that singles out that pattern. Without the second hypothesis it becomes meaningless to say that an event is improbable, because wildly improbable events happen constantly.

• I don’t see how that arrangement would be less likely than words being spelled out under the null hypothesis. If anything, they might be equally unlikely. The point of the example is that even though we don’t know how a designer would work if He exists, we would be confidently able to say that words being spelled out on a window were not formed by chance. As such, this is an example of ruling out chance even when we don’t know how an alternative hypothesis would look like
– user62907
Feb 27, 2023 at 18:35
• @thinkingman I mean that there is only one arrangement that looks precisely like that photo, but there are n>1 arrangements that look like they spell your words (different fonts, sizes, etc.) so the latter is n times more likely. It's still very unlikely, but that's beside the point, which is that my example is not n times better at rejecting the null hypothesis than yours. You claim you don't know what alternative hypotheses look like, but you do know – you stated one, and you could come up with more. Feb 27, 2023 at 19:01
• Yeah fair enough, although the specific arrangement/way in which the words are formed would still have the same likelihood. Anyways, the point is that the fact that it is meaningful and unlikely is enough to dismiss chance, even though we cannot predict how God would behave. Postulating alternative hypotheses does not amount to knowing what they would predict. One could postulate an alternative hypothesis to explain anything, and there's an infinite amount of them! A god can explain the arrangement that you linked as well. Why is it irrational to believe that a God made your pattern?
– user62907
Feb 27, 2023 at 19:06

I think the note you reference by Elliott Sober is not entirely correct. To understand these examples it is helpful to make use of a common distinction between the synchronic and diachronic use of conditional probabilities and of Bayes' theorem.

Bayes' theorem itself is neutral as to the interpretation of probability, i.e. it holds under the Bayesian or frequentist interpretation, or any other. Understood synchronically, i.e. at a particular time or information state, it simply states a relation between two conditional probabilities. The use of the theorem only becomes distinctively Bayesian when it is used diachronically as an updating rule, i.e. when the conditional probability P(B|A) is understood to mean that if I learn that A is true, I should update my degree of belief in B from P(B) to P(B|A).

This distinction is important. Understood synchronically, modus ponens and modus tollens are both probabilistically valid. Ernest Adams in his books, "The Logic of Conditionals", and "A Primer on Probability Logic" showed many years ago that the criterion for the probabilistic validity of an argument is that the improbability of the conclusion cannot exceed the sum of the improbabilities of the premises (where the improbability of A is P(¬A)). In simple cases where we have a few premises and all the probabilities are either close to one or close to zero, this has the consequence that if the premises are highly probable then the conclusion is highly probable.

So, for a given conditional "if A then B", modus ponens is valid, since if P(B|A) is high, and P(A) is high, this entails that P(B) is high. Modus tollens is also valid, since if P(B|A) is high, and P(B) is low, this entails that P(A) is low. Note the synchronic use here: I am not speaking of what happens when we learn something to be true, but what probabilities hold at a particular time or under a particular state of information.

As a slight aside, several other familiar rules from classical logic do not hold in probability logic. In particular, contraposition, hypothetical syllogism, or-to-if, and strengthening of the antecedent do not hold. These are not guaranteed to preserve high probability from premises to conclusion.

When we come to the diachronic use of probabilities, things are different. Strictly speaking, we are no longer doing simple probability theory but adopting a kind of Bayesian epistemological theory about belief revision. Bayesian epistemology is a contentious theory and works only approximately at best. Diachronically, modus tollens may fail. As Sober shows in his example, if P(B|A) is high and I learn that B is false, this does not entail that A is false. I may have just been unlucky. But also with modus ponens, if P(B|A) is high and I learn that A is true, I may be unlucky and B may turn out false.

It should perhaps not come as a surprise that modus ponens can fail in cases of belief revision, since this can happen even in non-probabilistic examples. An example much discussed several years ago runs as follows: "If President Reagan is a Soviet spy, we'll never find out. President Reagan is a Soviet spy. Therefore, we'll never find out." Diachronically, this doesn't work. As soon as we learn the second premise, the conclusion is ruled out.

The moral is that modelling belief revision is much more tricky than working with static probabilities. Learning some proposition to be true requires that we accommodate it among our existing beliefs, and in general there is no simple rule for doing this. This is also one of the reasons why the logic of conditionals is much more complex than that of other logical connectives.

Significance testing is somewhat different, because that is a frequentist method. A frequentist may form a null hypothesis, acquire data that is highly improbable on the basis of that hypothesis, i.e. a low p value, and then at some chosen threshold reject the hypothesis. A Bayesian prefers to take a prior probability for the hypothesis and update on the evidence. The result is not always the same, though in both cases a hypothesis can be disconfirmed by highly improbable evidence.

Using Bayes' theorem in a comparative form to compare two hypotheses is very common. It allows us to eliminate the term P(E) which is often unknown. In the case of comparing naturalistic and theistic theories of some event, it is of little use. The problem with theistic theories is that the probabilities are impossible to assign because they are imponderable. Also, the probabilities are not the only issue. Theistic theories lack the desirable properties of a good theory, such as simplicity, parsimony, explanatory value, and lack of adhocness.