# How can probability statements be falsified?

Have studied recently some about philosophical views of probability and ran into an interesting problem put forward by Popper:

According to Popper, probability statements are not strictly falsifiable. [For example, the statement "the probability that it would rain tomorrow is equal to 0.85" would not be falsified even if it would not rain tomorrow, since the statement also says indirectly that the probability that it would not rain tomorrow is equal to 0.15; so probability statements in fact resemble statements that cover all cases: namely, statements of the form "A or not A"].

Yet Popper adds, that probability statements are nevertheless treated by scientists as falsifiable. In turn he proposes to treat them as such but he seems to leave it for statisticians to spell out the details of how to falsify probability statements.

My question has two parts:

1. Concerning Popper's methodological decision to refer to probability statements as falsifiable: would it not weaken his demarcation criterion? Would it not render scientific every social science which resides on statistics?
2. Concerning statistics (of which I know very little): what kind of tests statisticians perform in order to refute hypotheses?
• 2. Suppose the probability of A occuring is 0.2 and the probability of B occuring is 0.3. Assuming that A and B are independent, the statement: the probability that A and B both occur is 0.5 is falsifiable. (basic) Probability theory tells us that the probability of A and B both occuring is in fact 0.06. – M. le Fou Feb 15 '16 at 8:30
• In some cases, the Law of Large Numbers can be used to falsify probabilities. – Era Feb 15 '16 at 18:24
• I had a series of questions regarding this some time ago. The answer seemed to be that the philosophy of science is murkier than the nice easy clearcut falsification of logical statements might suggest. From the answers I got, the acutall process of falsifying involved abduction to reject hypotheses which are sufficiently "unlikely." – Cort Ammon Feb 15 '16 at 20:18
• With respect to the 2nd part, one of the methods that can be used, is "correlation" with a double blind test. – Guill Feb 16 '16 at 5:47
• The statistical part is answered in Hypothesis Testing: Fisher vs. Popper vs. Bayes hsm.stackexchange.com/questions/3176/…, and Popper's demarcation as it applies to social sciences is addressed in Semantics of Popper's Demarcation criterion philosophy.stackexchange.com/questions/24530/… – Conifold Feb 16 '16 at 18:25

Popper expresses his position on the testability of statements about probability most clearly at the end of Section 68 of LScD, see also Section 66. His position is that we have to make a methodological rule about what relative frequencies should be deemed consistent with a probability estimate. That rule, he maintains, should not be arbitrary but should be a result of the accuracy with which the rule can be tested with available technology. In LScD, Popper advocated the frequency theory of probability. He later changed his mind and adopted a propensity interpretation of probability but this did not change the substance of his position on the testability of probability statements. The propensity interpretation postulates some sort of measure over the set of possible states, but Popper didn't provide any explanation of why this measure was supposed to be relevant as far as I can tell.

A more satisfactory explanation of the testability of probabilistic statements has been supplied by David Deutsch. Statements of the sort commonly described as probabilisitic can be tested when the laws of physics provide a measure over the set of possibilities that respects the probability calculus. Such a measure has been derived in the context of quantum theory. See also a lecture he has given on this issue.

• Thanks again, @alanf. Have just listened to the lecture you referred to and took a glance, albeit quick, at Deutsch's paper 'The Logic of Experimental Tests' - though need to process it. Wish to verify - does he dispense with interpretation of probability via appealing to multi-world metaphysics?...Could you kindly please explain his reasoning in simple words? (He says he is Popperian but...he does not embrace Popper's approach to probability by talking on objective probabilities - or maybe I misread him). – L.M. Student Feb 17 '16 at 22:05
• Most discussions of probability suppose that some state is picked at random out of a hat. Such discussions don't explain where these probabilities come from, which is no good. That's not true in quantum theory. There is an objective physical quantity that respects the rules of probability and can act as a guide to some kinds of decisions. The fact that the quantity in question can act as a guide to decisions doesn't make it subjective. There may or may not be other such quantities related to other theories. – alanf Feb 17 '16 at 22:59
• The relevant similarities to Popper are: (1) Rejection of subjective probability theories. (2) Rejection of inductivism in general, including probabilistic induction. – alanf Feb 17 '16 at 23:00

This answer should be read as a kind of extended comment on alanf's answer, which I broadly agree with, but would like to qualify. Deutsch argues that probabilities can be eliminated from physical theories, in other words, that we have no need of stochastic processes in a physical theory. In particular, he is concerned to maintain that quantum theory, which has often been interpreted to involve fundamental indeterminacies, can be understood instead as a deterministic account of how particles and fields behave across a multitude of worlds, according to the many-worlds interpretation.

Deutsch then proceeds to dismiss the epistemic notion of credences, but this need not follow from the rejection of physical probabilities. We, as cognitive agents with limited and imperfect capabilities, never possess perfect information about anything. Whenever we make decisions, which is all the time, we are forced to make those decisions under uncertainty, and unless we have some way of quantifying that uncertainty, we will be prone to making bad decisions. This is why probabilities show up in decision theory: it does not mean we are making decisions about stochastic events, merely that we are making decisions with incomplete or imperfect information. The probabilities are simply there to quantify the uncertainty. Bruno de Finetti showed how, using Dutch book arguments, we can start from a very innocuous and plausible notion of what consititutes a bad or irrational decision, and proceed to derive probability theory from it. Others, including Richard Cox and Edwin Jaynes, have shown how a primitive notion of inference can be used to derive probability.

The upshot of this is that inductive reasoning using epistemic probabilities is very much alive and well and flourishing in statistical practice, and in particular in the machine learning/articifial intelligence domains. As to your specific question of what kinds of tests statisticians perform, there is no general agreement about methodology. The three main camps are classical (frequentist), Bayesian, and likelihoodist. The classical approach broadly involves forming null hypotheses, designing experiments to test them and rejecting the hypothesis if the results are significant (Fisher), or testing hypotheses according to their false positive and false negative error rates (Neyman and Pearson). Bayesians specify prior probability distributions and use data to update those distributions. Likelihoodists take two rival hypotheses and calculate a likelihood ratio that allows one to say which hypothesis is confirmed relative to the other.

• This answer makes no sense. Probabilities are precise numerical predictions. You can't get precise numerical predictions from ignorance. Rather, you have to assign a measure over some space of states. So you have to know the laws of physics that give the relevant measure and space of states. The Jaynes type approach consists of implicitly assuming such a state space and measure. This obscures potential problems with the chosen measure. – alanf Feb 17 '16 at 10:23
• Inductive reasoning is not used at all since it is impossible (Deutsch, "Fabric of Reality" ch 3 and 7 and Popper "Objective Knowledge" Chapter 1). For criticisms of inductive probability see Part II of "Realism and the Aim of Science" by Popper. Machine learning etc involve setting up a computer with a state space and an appropriate measure over that state space to get the computer to perform some task. How is this done? You guess a bunch of measures and run programs that respect those measures. You then pick some of the programs, tweak them and try again until you get a good enough program. – alanf Feb 17 '16 at 10:32
• Probabilities do not have to be precise, they can take the form of ranges or distributions. Also, I am not claiming that predictions arise from ignorance without a measure or a state space, since in practice we are never without those, even though our initial guesses at them may be highly inaccurate and in need of revision. What probabilities do is quantify uncertainty and help us to make inferences and decisions with uncertain information. If they don't do this, why do you suppose probability has so many useful applications? – Bumble Feb 17 '16 at 22:36
• As to inductive reasoning being impossible, I would say rather that it is ubiquitous and inescapable: so maybe we mean something different by the term. I am using it in a wide sense to cover any ampliative kind of reasoning. Saying that induction is impossible seems to mean only that induction is not reducible to deduction, but that would be missing the point of it. – Bumble Feb 17 '16 at 22:36
• when you think the impossible is "ubiquitous and inescapable", check your premises. i guess you don't really believe it's impossible, but nor have you addressed the arguments that induction is impossible and that anyone ever doing it is a myth. you don't seem familiar with the Popperian ideas you're hostile to. – curi Jun 11 '16 at 11:15

No empirical falsification is ever decisive whether the theory is probabilistic or not. Popper discussed this, e.g. section 29 of LScD "THE RELATIVITY OF BASIC STATEMENTS". the basic issue, also known as the duhem-quine problem, is whenever you have a refutation either the refuted theory is wrong OR the refutation itself could be wrong, so it's never totally clear, decisive and settled.

So what do you do instead? You criticize and stop seeking certainty. You try to find bad ideas and reject them. Does it makes sense? Any logical issues? Does it solve the problem it's supposed to solve? Does it cause any problems? You look for good qualities for ideas and things wrong with them. Do your best to think and improve your ideas. Grow your knowledge instead of yearning for final unquestionable answers. This works the same with probabilistic and non-probabilistic theories.

I don't think this is complicated.

If we state that the probability of some event is very very small and then it occurs, then that probability statement is falsified (with a very very small chance of being in error). This gives a subset of probability statements that can be falsified. This would be the classical statistical approach taken by Fisher, and Neyman and Pearson.

Note that if an event has already occurred it has probability 1, and it is meaningless to assign any other probability to it. The order of is important. For example, we would not role a dice 10 times, obtaining 4 5 2 4 5 4 2 3 1 4, and then go on to say that the chance of obtaining this sequence with a fair dice is 1.6e-08 and thus reject that this is a fair dice.

Concerning your "chance of rain example": The probability given is large, hence not falsifiable. However from a theory that can be used to calculate such probabilities you might come up with a probability statement with small probability, eg "the probability that it will rain every day for 100 days is 1e-6". If it did then rain every day for 100 days the theory would be falsified.

I'm not sure how Popper resolved the problem, but in general, we treat statistical statements as concerning a set of events, not a single event. The statement "a uranium atom has an X/Y probability of decaying in time T," it means that we'll expect the observed ratio to approach X/Y as we observe more and more atoms.

In a sense, this is still not strictly falsifiable, because there's nothing preventing statistical anomalies. However, standards can be set for acceptable expected deviations.