Is a prediction better than an ad/post hoc hypothesis apriori?

Question

Suppose John wins the lottery. Eric, after he wins, claims that he rigged the lottery for John to win. Now imagine an alternative scenario where Eric predicts that John will win and that he claims to have rigged it and then John wins.

Intuitively, the second scenario seems to be better and very good evidence for Eric to have rigged the lottery compared to the first. But why exactly is this the case?

Clearly, it can’t just be the fact that the probability of John winning is low. This would remain the same in both scenarios. Why then exactly is a prediction better evidence?

It seems, atleast to me, that this would only be justified if, in general, riggers who do rig lotteries are more likely to rig it if predicting in advance vs. just claiming to rig it after the fact.

The question then is: without this assumption, would a prediction be better evidence? Is there anything apriori about a prediction that serves as good evidence for a particular hypothesis or is it purely empirical based on past and prior evidence?

Answer 1

The example you gave is complicated because it introduces the possibility of Eric intentionally lying. That obfuscates the core issue, which is whether a prediction is better than a post-hoc explanation even when people are being honest.

Here's a more pure example concerning that core issue.

Scenario A. A scientist, David, is about to run an experiment. He predicts that the data will fit the curve y = 3x^2 + 1. He runs the experiment and in fact it does fit the curve.

Scenario B. David makes no prediction before running the experiment, but after he runs it, he fits a curve to the data and realizes y = 3x^2 + 1 is a good fit.

Scenario A is obviously much better evidence for the hypothesis H, that if the experiment was run again, the data would again closely fit the curve y = 3x^2 + 1. But why?

In idealized Bayesian inference, the probability of H given the experimental data O doesn't depend on when H was proposed. Bayes' law says:

P(H|O) = P(O|H) P(H)/P(O)

In scenario A, the prior probability P(H) shouldn't be any different from scenario B. Nor should the prior probability of the observation, P(O), and nor should P(O|H), and so P(H|O) should be the same in either scenario.

But! In practice, the scientist does not exactly know P(H). But P(H) can be partly estimated by the following heuristic: a hypothesis that occurs to the scientist, before the experiment, is more likely (higher prior probability P(H)) than a hypotheses that does not occur to the scientist before the experiment.

So, the fact that in scenario A the scientist has made a prediction, points towards P(H) being legitimately high in scenario A. In scenario B, the fact that the scientist failed to make the prediction points towards P(H) being relatively low, not distinguished from any of the other hypothesis that the scientist didn't predict would happen.

This can be a very large effect. If there are a thousand possible hypotheses initially, the fact that the scientist correctly picked one out of a thousand during the prediction phase is good evidence that there's something special about that particular hypothesis. And that explains why we would expect P(H|O) to be much higher in scenario A than in scenario B.

Answer 2

People don't judge hypotheses in isolation; they weigh them against other hypotheses.

In this example you're weighing at the very least two hypotheses: that Eric rigged the lottery, and that Eric heard who won it and echoed what he heard (lying about the origin of the information).

The second hypothesis goes from highly implausible to highly plausible in a short time span starting at the point where the future light cone of the lottery drawing intersects Eric's worldline. The absolute plausibility of the first hypothesis may also change, but it doesn't change by nearly enough to prevent a flip in the relative plausibility of the two.