Can a zero prior probability for some theories be justified?

Question

Let us assume the case of psychics and call the hypothesis of a “psychic explanation” H. Bayesian theory tells you to never assign a prior of zero. This is because if P(H) = 0, then no amount of observations can make that hypothesis likely as an explanation.

But it is easy to see that certain observations never make this hypothesis likely as an explanation. For example, suppose a psychic claims to be able to guess a number in my head from 1-10. Suppose he does this five times. Heck, suppose he does this 100,000 times. This may seem, intuitively, to be irrefutable evidence that psychic powers are at work. Or even if you don’t think it is, it would seem to atleast make the psychic explanation more likely.

But does it actually make it more likely? Sure, P (Evidence|No Psychics) may be extremely low here and P(Evidence|Psychics predicting numbers) may be 1, thus resulting in a likelihood ratio favoring a psychic significantly. It seems absurdly improbable to occur by chance. However, this is all useless without a non zero prior for psychics. In other words, without the prior assumption that it is possible for psychics to do their work, this evidence is irrelevant by itself. A zero prior would make this evidence irrelevant.

As per Bayes’ rule, the P (H|E) / P(~H|E) = (P (H) / P (~H)) * ( P(E|H) / P (E| ~H)). Thus, if one does not assign a non zero probability to H, even a million successful predictions become irrelevant. Note that the first term is independent of the second. In other words, a million successful psychic predictions do not increase the prior probability of psychics existing, which is determined independently. It only increases the likelihood of psychics performing something if one already assumes that psychics exist.

Now on what basis can we ever put a non zero prior? As mentioned, we cannot use successful predictions to justify one, since the prior by definition is independent of new evidence. So how else?

It seems that the only way to do this would be to have direct evidence of a psychic. By this, I mean having an actual proposed mechanism for how it is actually being done and being able to empirically observe it. This would justify a non zero prior since one can explain from start to finish how a psychic is actually doing what he does. Only then can one justifiably start looking at psychics as explanations and make it more likely than chance or another explanation.

It seems that without this crucial discovery, psychic explanations shouldn’t even be considered possible, and allow us to justifiably give them a prior of 0 in the meantime. In this case, the zero prior wouldn’t signify that psychics are impossible. Rather, it would from a practical standpoint, signify that it should only be considered possible once direct evidence for psychics existing is observed. This may also reflect a weak point in Bayesian epistemology since the act of observing a mechanism empirically and then updating the prior isn’t part of the epistemology itself. Priors are only updated by what a hypotheses predicts, not by its inherent mechanism of action.

Nevertheless, does this give us a practical case for legitimately giving a theory a prior of 0 in certain contexts?

Answer 1

I argue that, for any falsifiable hypothesis, a prior probability of exactly 0 or 1 is irrational; to assign such a prior is to explicitly abandon one's capacity for reasoning in the face of evidence.

When faced with a new hypothesis, a rational agent should never assign a zero prior. Among other things, it leads to contradictions where two new hypotheses are encountered, both assigned a zero prior, but then it is discovered that, logically, at least one must be true, but the agent is prevented from ever getting away from their zero starting point credence. An agent that assigns a prior of exactly 0 or 1 is not exhibiting sound reasoning and will consequently make poor decisions.

Commonly, rational agents apply the principle of indifference to new hypotheses, assigning a prior of 0.5 to any new hypothesis.

However, as the agent first contemplates the new hypothesis, they may discover that existing, known evidence bears on the hypothesis, and immediately begin updating their credence accordingly. For a human, this can happen so fast as to be unconscious; if someone suggests to me an absurd but novel hypothesis, for example that I am currently on fire or dead, I don't consciously assign a prior of 0.5 and then begin testing the hypothesis to refine my credence. Instead, I'm immediately aware that the hypothesis contradicts a wide body of available evidence, and hence my first conscious impression of the probability of the hypothesis is that it is very unlikely--but not zero!

Let me emphasize: for the hypothesis "I am currently dead", my current credence is not exactly zero, although of course it is low (maybe 1e-15?). And again, I contend that any agent that assigns a zero credence to that (or any other) hypothesis is not reasoning soundly. Among other sources of uncertainty, in this case, even defining the word "dead" is challenging.

Now, for a novel hypothesis for which one currently lacks evidence, it is not necessary to use 0.5 in particular as the prior. In fact, any prior in the open interval between 0 and 1 will suffice, so long as one then diligently and accurately collects relevant evidence and applies some procedure like Bayesian inference to update their credence. Given sufficient evidence, any soundly rational agent should converge at the same credence, regardless of the very first assigned prior probability. The only advantage of 0.5 in particular is that it, arguably, leads to faster convergence across a broad range of hypotheses.

For the specific hypotheses of capacity for psychic mind reading or the existence of extra-terrestrial technological civilizations (ETTCs), these only seem unlikely due to our current cultural context, but they are not a priori absurd or unlikely at all. I can't read your mind, but with an EKG I can measure your heart in a way someone 1000 years ago could never have imagined. I don't know if ETTCs exist, but a priori, the idea that they do should not be any more absurd than the notion to Europeans 1000 years ago that America existed and people lived there. If someone in the middle ages truly assigned a zero prior to the possibility of EKGs or people on another continent, they would be forever unable to add those facts to their knowledge of the world, even after coming into direct physical contact with those objective realities.

Side note: A mathematical statement like "2+2=5" is not a falsifiable hypothesis because the latter is (paraphrasing Wikipedia's definition) something that can be contradicted by an empirical test. Confidence in mathematical truth comes from proof rather than experimentation and inference, so I see its statements as outside the scope of this question. (That said, there is a sense in which the acceptance of mathematical definitions and proofs is a social process, with inference potentially playing a role, but I don't know how to sensibly formalize that.)

Answer 2

Yes, if you have reason to suppose a phenomenon is impossible, you can allocate a zero probability to it. If later you find it is possible after all, then you were wrong in your initial assessment. Bayes' theorem assumes that it is possible to assign valid probabilities to events. In the case of your hypothetical psychic, there is no clear recipe for assigning a prior. More importantly, perhaps, without a proposed mechanism for mind-reading, your hypothesis is pretty meaningless. Suppose you conducted an experiment in which someone correctly guessed a thousand times in row which number you were thinking of between 1 and 10. Bayes' theorem would tell you that the odds of that happening randomly were astonishingly small, so it is much more likely that there was some other reason for it. But what is the other reason? To simply conclude that the person is 'physic' is just a shorthand way of saying that you don't know how they did it.

Answer 3

It's not impossible to assign priors of 0 or 1 in the Bayesian framework it's just pointless.

Like assigning a probability of 0 or 1 expresses your firmly held believe that something does or doesn't exist, is or isn't possible. So throwing other possibilities at that isn't going to change that because you treated it as a certainty, as axiomatic to your worldview, so nothing is ever going to change that. The only change that can occur is if something convinced you to believe in the impossible or more precisely to believe in something that you previously thought was impossible. Which can happen due to being gullible or having been wrong in the past. Now it would be quite a feat for a modest mathematical framework to guess where you went wrong in your reasoning and why and what is psychologically able to rectify that problem.

So of course you can presuppose the hypothesis that psychs don't exist, until you're explicitly provided with a proof to the contrary, but that's not really the kind of hypothesis that lends itself to the Bayesian framework.

So instead of blindly guessing the probability of an explanation that you don't even know or actively think doesn't exist in the first place. The more useful approach is likely to assume the hypothesis of a fair game of chance. So if that were true you'd expect a 1/10 chance of hitting the correct number by chance. So if you end up with a strong deviation from that, Bayes could help you put a label on the likelihood of that deviation.

The other problem is your definition of what a "psych" is. Like if you called any distribution that is better than 1/10 "psychic" than that is more likely, then it is equal to or more likely than actual mind reading, because that set would include mind reading but would not be limited to that. Idk there could be manipulation tactics where you prime your target with certain numbers so that the brain is more likely to think in that direction when prompted on the spot of thinking of a number or you could scan some output signal, idk from facial expressions to sounds, heat signals whatever or you could hook them up to a MRI and literally read their minds. There's a plurality of options so just because it's not chance doesn't mean it's one of these in particular.

That being said if you assign a 0 probability of that outcome to begin with and claim anything other than chance is "psychic", then it would actually include all these methods and it's quite possible that at least some of them have a non-zero probability.

Answer 4

The experiment proposed tests that the person (called "psychic") can guess the number in your head. If they do it 100,000 times, it is very reasonable to conclude that they can do it. However, this experiment does not test the mechanism by which they guess the number - elucidating the mechanism requires designing special statistical tests, which would allow to single-out or exclude specific mechanisms.

E.g., they may guess the number, because they ask you first to write it on a piece of paper (so that you do not cheat) and see its reflection in a mirror behind you or get a signal from the spy place behind you (or any other classical poker tricks.) One may try to exclude such possibilities, by ensuring the absence of mirrors, doing it in one-on-one setting or even via telephone. The real scientific ingenuity goes into designing experiments: figuring out the possible explanations and the ways to test them. The testing itself is not science - it is a technical mathematical procedure, which requires basic qualifications or simply good statistical software.

See also Design of Experiments:

The experimental design may also identify control variables that must be held constant to prevent external factors from affecting the results. Experimental design involves not only the selection of suitable independent, dependent, and control variables, but planning the delivery of the experiment under statistically optimal conditions given the constraints of available resources. There are multiple approaches for determining the set of design points (unique combinations of the settings of the independent variables) to be used in the experiment.