# Should X, if there’s no evidence for X, be given a non zero probability?

There may be no evidence that a fairy is sitting on a table. Many argue that one cannot prove a fairy doesn’t exist. Thus, many decide to attach an (infinitesimal) probability to it existing, as many Bayesians regularly do.

However, attaching a non zero probability to fairies implies that it is possible for fairies to exist. But there is no evidence that they exist. So on what basis can one justify (even an infinitesimal) nonzero probability?

Is the better solution to refuse to attach a probability to this altogether?

• Exponentially small probabilities are effectively impossible. This is how physicists deal with the issue; yes, a fairy could appear, but it's so unlikely that it can be dismissed. Jul 5, 2023 at 14:09
• Use the traditional method (occurrences/universe = 0 = fairies don't exist). Probability does not imply actuality. Fairies might exist, but the more you test, the lesser the possibility, so it is natural for probability to be zero) Jul 5, 2023 at 17:18
• Your question seems to be about the philosophical difference between Bayesian and Frequentist approaches. But "Is the better solution...?" has experimental/practical overtones. That exact question has been debated in my (former) community, experimental high energy physics, so I'll offer a link to an interesting commentary: "Why isn't every physicist a Bayesian?", astro.princeton.edu/~strauss/AST303/bayesian_paper.pdf Jul 5, 2023 at 21:13
• That's not quite how science works. Since 2012 physicists have assigned a reasonably high probability to the Higgs boson existing, say 0.999999. The Higgs was postulated back in 1964 on theoretical grounds. It wasn't like a fairy: there were good theoretical grounds for supposing it existed, so it would have been reasonable to assign it a small probability; small because there was no evidence. Then they did the experiments. Of course if there is something weird happening around your house, and a fairy might be an explanation, then you might go full Bayesian and assign a small probability. Jul 7, 2023 at 2:57
• I've just flipped a coin. I haven't looked at it yet. There is no evidence that the coin came up heads. Can a nonzero probability be justified for the event that the coin came up heads? Would it be better to assign a zero probability instead? Jul 7, 2023 at 10:03

You mention Bayesians, so I'll reply in that context.

The Bayesian framework is (roughly) a way to take a "prior probability" of a statement, multiply it by a factor accounting for new evidence (the "support", or the ratio of the probability of the new evidence if the statement was true to the probability of the new evidence independent of the statement), to get the probability of the statement with the evidence taken into account (the "posterior"). (Symbolically, `P(H|E) = P(H) * P(E|H)/P(E)`.)

The general approach to "Bayesian" reasoning is using this with continuous updating. You can use the posterior probability after one observation/experiment as the prior for the next. With enough observations, the influence of the original prior you started with becomes negligible, and your statement of probability is (almost) entirely based on the evidence.

Well, mostly. You run into issues at the extremes of the probability distributions. If your prior is 0.0 (exactly), then it doesn't matter what happens with the support term. Anything times zero is zero, and using Bayes rule you'll never get a non-zero posterior for any statement with an exactly zero prior. (If `P(H) = 0.0`, then `P(H|E) = 0.0`)

Which means that if you have an exactly zero prior for a fairy sitting on a table, it doesn't matter if later on you get new evidence where you see a fairy sitting on the table, with the President and the Prime Minister awarding her medals, being covered by hundreds of reputable international news organizations. Anything times zero is zero, and all the Bayesian updates in the world won't budge "there's a fairy sitting on this table" above zero, regardless of how incontrovertible the evidence is.

Now certainly you could say "well, a fairy having a news conference with the President is a completely fluke event, and if that ever was to happen I'd just simply discard my prior and make a new one.". But that just shifts things. What do you think the probability of needing to remake your prior is? You've just admitted it's possibly non-zero. Why don't you then just include the probability of remaking your prior into the prior itself? (The Bayesian approach allows you to incorporate probabilistic evidence.)

This is why people who are following a Bayesian approach avoid assigning exactly zero probabilities to things. From a practical perspective there's very little difference between a probability of exactly 0.0 versus a one-in-a-centillion probability. (They both in practice mean "neither I nor anyone I'll ever hear about will see it happen".) But when used with Bayes rule a one-in-a-centillion probability has the possibility of being updated with enough evidence in a way an exactly zero prior does not.

So why not just refuse to assign a probability at all? Well, to incorporate evidence in a Bayesian fashion, you need to have a prior to update, so at some point you'll need to create a pre-evidence probability. Whether you do that now versus right before you incorporate the evidence is immaterial.

You likely have some evidence from prior experience to use for the prior. You have knowledge of tables and of fairies, so you're not completely at a loss for assigning probabilities. You have "evidence" in the form of prior experience seeing (or not seeing) other fairies on other tables, so you can form a prior on that basis. (Hopefully the specific evidence regarding this fairy on this table will be such that it doesn't really matter much what the prior was.)

And you can't really "refuse" to assign a probability to something if your choices and actions were to change depending on whether it was true or not. Every time you set a book down on a table, you're implicitly making a call on the probability that there was a fairy there that you would squish. The fact that I ignore the possibility doesn't mean that I refused to assign a probability to it, rather, it means I implicitly thought the probability was negligible. Contrast a situation where I have no evidence there is a child with a ball on the sidewalk in front of my driveway. I might "refuse to assign a probability", but that doesn't mean I shouldn't look as I back my car into the street. I may have absolutely no evidence for the specific statement, but there's a non-negligible probability that it is true, so it behooves me to check.

It should also be mentioned that often the Bayesian approach isn't used with precise probabilities, but rather probability distributions. That is, you can say things like "for my prior, I think the probability is somewhere between 1% and 2%, leaning toward the lower end of that range". It can get a bit complicated because you start talking about probabilities of probabilities, and often the form of the priors are chosen due to mathematical tractability rather than other considerations, but there are ways of dealing with uncertainty in the specification of the prior, and examining the way those predictions become more certain as evidence is incorporated. So just because you assign a probability doesn't mean you assign an exact probability.

• To me, Bayesian reasoning has to do with the probability that a hypothesis, H, is true (let's say the hypothesis is that "Fairies exist") given some set of evidence E (books say that fairies exist or some powder called "pixie dust" exists that has some properties that are purported to be supernatural). --------------------- It's --- P(H|E) = ( P(E|H) P(H) )/P(E) --- Jul 5, 2023 at 22:12
• @robertbristow-johnson Yes, that's the second paragraph. Your prior is `P(H)`, the support for your evidence is `P(E|H)/P(E)`, which you multiply together to get your posterior `P(H|E)`. If `P(H) = 0.0`, then `P(H|E) = 0.0`, regardless of what `P(E|H)/P(E)` is.
– R.M.
Jul 5, 2023 at 22:22
• Isn't the whole lesson of Pascal's wager that nobody treats religious belief in a Bayesian way? Jul 6, 2023 at 22:17
• This answer implicitly assumes that the alternative option is to assign a zero probability. Why not simply refuse to assign a probability to a claim without evidence? It seems more appropriate and conducive to reality Since assigning even a very small prior seems to assume some knowledge even when there isn’t. Doing something for “practical” reasons without evidence seems to go against the pursuit of truth Jul 6, 2023 at 23:41
• Hmmm. Right now the big problem is the Axis of Evil (cosmology) needs an explanation, and inadmissible hypotheses are being looked at because of it. Seems like a prior of 0 can be overcome when the alternative is the theory of everything is broken. Jul 7, 2023 at 1:51

In training ML models (as an example), it is important to account for previously unseen data, otherwise you will get nonsensical results when you do see something rare for the first time. In this case (and if using a probabilistic model), it makes sense to attach a small probability to events that were never observed in training.

It's actually (probably) better not to include something in a model than to include it but say it is impossible.

Out of the options:

1. include in your model as impossible
2. include in your model as possible with a very small, arbitrary probability
3. don't include in the model

Option 1. is the worst, it is basically guaranteed to come back and bite you. Rare events will happen eventually, and your choice is between your model being inaccurate at that point (options 2., 3.), or the model simply not giving sensible answers at all (option 1.).

• I don’t design ML models but I’m guessing those events incorporated are still possible events in the sense that if they don’t occur in the training data, they occur somewhere in the world. I doubt an ML model would incorporate something that has truly never occurred before in the world (I.e. a true case of no evidence). For otherwise, there can be an infinite number of events you can conjure up Jul 5, 2023 at 13:31
• @thinkingman The problem is, you don't know if a given unseen event is possible in the real world, and if you assign it a 0 probability and it does happen, you typically end up dividing by 0 or something and the math falls apart. The consequences for assigning a small but nonzero probability to an event that's actually impossible aren't so bad (as a practical matter). There are various hacks for dealing with infinite sets, but they're uninteresting philosophically; they're understood to be engineering compromises for which the math will slightly off, but won't fail catastrophically.
– Ray
Jul 6, 2023 at 4:05
• @thinkingman Right, models have costs, so we model things that matter to us and leave out things that don't, like tornadoes in Antarctica. So there is always the context of "What am I doing this for?" Also, searching for extremely rare or unlikely things is much more subject to error, like medical tests for uncommon cancers or whatever. Jul 6, 2023 at 11:02
• @Ray I think you're begging the question. "if you assign it a 0 probability ... the math falls apart" doesn't mean assigning 0 probability is wrong, it just means assigning 0 probability is wrong or the math is wrong. But whether the math is right (as in "appropriate to model belief") is exactly the OP's question. Jul 6, 2023 at 22:14
• @Ray Not assigning a non zero probability does not imply one should assign a zero probability to an unseen event. One could just simply refuse to assign a probability in the first place. Jul 6, 2023 at 23:39

This is known as Cromwell's rule, which states that

the use of prior probabilities of 1 or 0 should be avoided, except when applied to statements that are logically true or false, such as 2+2 equaling 4.

If you and I start with different priors that satisfy Cromwell's rule, our posterior probabilities will eventually converge as we accumulate evidence. If one or the other of us starts with exactly 0 or 1, we will never agree, regardless of the evidence.

BTW, to a Bayesian, probability is belief that is justified by evidence. If I get more evidence that either supports or rebuts my belief, the Bayesian probability changes.

• "He who doubts from what he sees, will never believe, do what you please." - William Blake Jul 6, 2023 at 14:06
• "I beseech you, in the bowels of Christ, think it possible that you may be mistaken." - Oliver Cromwell
– ojdo
Jul 7, 2023 at 10:48
• @ojdo And they didn't, so Cromwell massacred them. He wasn't a very nice man. Jul 7, 2023 at 21:01
• But he didn't clamp his priors to 0 or 1! ¯\_(ツ)_/¯
– ojdo
Jul 10, 2023 at 12:52

Probabilities are empirical predictions about future measurements given past measurements: how many times should I expect to win if I play the game a certain large number of times?

I think you mean extremely small finite, not infinitesimal.

Literally infinitesimal probability and zero probability are identical over any finite number of trials because that's what infinitesimal means. 1-(1-dx)^n = 0 for any finite n, infinitesimal dx: you can play the game any finite number of times and still expect to win zero times.

Extremely small finite probability and zero probability are identical over any sufficiently small number of trials. Sufficiently small can still be impossibly huge. For example, there's about a hand grenade worth of available Potassium-40 alpha decay energy in your body right now. The probability of you exploding because a significant fraction of the Potassium-40 in your body spontaneously alpha decays all at once is finite. (I calculate order of 10^(-10^19) for a significant fraction decaying in one second. How's that for an exponent?) But that finite number is so low that if you populated every star in the observable universe with a trillion humans and ran the experiment for a trillion times longer than the time between now and the Big Bang, you'd expect to get exactly zero Potassium-40 explosions.

There is no difference between winning the game zero times if you play the game as often as it is possible to play the game (e.g. spontaneous explosion) and winning the game zero times if you play the game a bazillion times more often than it is possible to play the game (e.g. a macroscopic object quantum teleporting across the room), because you can't play the game more times than it is possible, and zero wins is zero wins.

Okay, I am going to try to plumb this Bayesian inference thing from the POV of an electrical engineer. In communications systems we use Bayesian inference regularly to postulate what symbol was originally transmitted when some particular signal was received. I think, when Bayesian inference is used in science, it is to consider the likelihood that some hypothesis is true when some other evidence is observed. The communications problem of the electrical engineer is simply a specific example of the more general inference of hypothesis, given evidence, used in science.

Now, if we call a specific hypothesis, H, and some set of evidence (that may or may not support the hypothesis), E, then Bayes rule (along with some other quantitative axioms of probability) says this:

Now, I would like you to focus on the third line of the conditional probability, P(H|E). The quantitative value depends completely on the second term of the denominator:

Now, if that term is equal to 1, then the entire dependent probability is P(H|E) = ½ and the odds (given evidence E) are equal and it's just as likely that fairies exist as not. If that term gets "astronomically large" in value, then it's astronomically unlikely that the hypothesis, H is true. But if that term gets extremely small, very close to zero, then the likelihood that H is true, given this evidence E.

So this is how a Bayesian reasoner would frame the debate. Now, of course, it depends on what the evidence E is.

The OP posits that there is no evidence that fairies exist. Well, I would like to, just for the sake of argument, say that perhaps that there are books that say something about fairies, that this be treated as "evidence". Now, it seems to me that the fact that these books that say that fairies exist, that this does not help the case for the existence of fairies.

Now, with the fairy hypothesis, let's say that the only evidence to support fairies is that there are books that talk about fairies. Now we have books that talk about George Washington. I have never seen George Washington nor have seen direct evidence of the existence of George Washington other than what I see in history books and in modern political discussion of American history. It is my judgement that P(EH) is much much smaller than P(E|H) so, even if I start out with the initial belief that P(H)>0, but is not ≈0 (that the a priori likelihood that George Washington existed is not astronomically small) then all of that contributes to the value of that term above being very close to zero, so I judge that, given the evidence of George Washington that I read in the literature (and in all other references to the first president), the hypothesis that George Washington is an actual historical person is quite likely true. P(H|E)≈1 . It's because if Washington did not exist, then the likelihood of seeing this evidence is much smaller than the likelihood of seeing it if Washington did actually exist.

But with fairies, the evidence referring to and describing fairies in books would exist almost as likely whether fairies actually existed or not. In that case P(EH) is about the same as P(E|H), so that ratio is about 1. So for me to conclude that fairies exist given the evidence in the literature, I would have to start with a reasonably strong a priori likelihood that fairies exist P(H)≈1. But if I start out with P(H)≈0, then the "evidence" in the literature does not help.

Now, however, suppose there existed some magical pixie dust, that had supernatural properties, as evidence E? That would change things. That is because P(EH) << P(E|H) and that ratio becomes very close to zero. So then, for me to conclude that fairies don't exist, I would have to start with the a priori likelihood of fairies existing P(H) being astronomically small to begin with. So, in other words, if you are staring at evidence of pixie dust with supernatural properties, for you to conclude that fairies don't exist, in the face of this evidence, would require a strong a priori belief that fairies don't exist to start with. I would consider that to be a prejudice.

Now, in my opinion, a better hypothetical scenario to consider is more like this: Suppose you are seated at a poker table for the very first time in your life and you are dealt, for your very first hand, a Royal Flush in hearts. What are you gonna think? That you're a superb poker player? That you're extremely lucky? Or, simply based on the probabilities, that maybe someone is stacking the deck (and maybe whoever did that really likes you)?

The likelihood of a Royal Flush of a specific suit is 1 outa 2598980. That is P(EH) where E is the fact that you're dealt this Royal Flush and H is the hypothesis that someone is stacking the deck. If those probabilities are not sufficiently astronomically low, consider the hypothetical that one person wins the Lotto six or eight times in a row. It's not impossible, but if it were to happen, based solely on the probabilities, all of us would reasonably suspect that someone is nefariously fixing the game.

I believe that the original intent about the original post is not so much about fairies, but is more about the existence of God. If that is the case, might we discuss this Bayesian reasoning in light of the intended question beneath the surface?

• I am sure more books talk about God than about George Washington, and for much longer. Jul 6, 2023 at 10:50
• I agree. And lotsa books that talk about an alleged historical person is evidence, but that evidence is not the same as proof. No one is gonna prove or disprove God. But there is evidence, sorta like fingerprints. But then again, while fingerprints at a crime scene are evidence, they are not necessarily evidence that some particular suspect is guilty of the ostensible crime. Proving someone is guilty of a crime would be to derive an explanation of the evidence that is solely consistent with the evidence. Jul 6, 2023 at 14:37
• So, for a crime, there is a determination or judgement of what seems to be the case. For God, we need a determination of what seems to be the case. But believers simply request a change of venue if the answer is not what they want to hear. Jul 6, 2023 at 18:36
• Well, don't toss all believers into the same bucket. What I was trying to say is that the fingerprints are evidence, whether or not they are proof of guilt. I am saying that anthropic conditions on the Universal scale (these would include circa 26 dimensionless fundamental constants of physics) are evidence that point to design. The fact that they take on the values that they do is not proof of design, but I dispute when someone claims that they aren't even evidence. The fingerprints remain evidence. Jul 6, 2023 at 19:16

None of the other answers seem to entertain your final question, "is it better to refuse to assign a probability at all?". So here is an argument that probability (or, at least, exact probability) is just not appropriate for questions where there is no evidence.

Bayesian interpretation of probability is extremely flexible. As Simon points out, Cromwell's Rule makes discussion of absolutely certain events very trivial in a Bayesian framework - it is reserved for beliefs that cannot be persuaded by any evidence. That includes evidence that your senses are deceiving you, that the universe is a simulation, that you are a brain in a vat, and so on.

Conversely, if you take a radically skeptic position here and allow for doubt in anything besides your own existence, almost anything can be modelled as a probability between 0 and 1. You have some gut feeling about how likely the event is relative to other events, and that gives you a sort of ordering. The likelihood of fairies existing seems clearly lower than the likelihood of it raining tomorrow (after all, I've seen it rain before, but I've never seen a fairy exist). You keep going with your other beliefs until you've assigned a probability that's lower than any other probability you've ever observed. If you do more trials of seeing fairies not exist, your probability should be going down all the time, but there is no magic way to find out how fast.

If you are happy to take this position, then I cannot talk you out of it. If you are prepared to assign a confidence number, say, 1 in 100 quadrillion, to the event "everything I have ever perceived is a lie", then it is impossible for you to distinguish between your confidence in events that are 1 in a googol vs 1 in a googolplex. Your notion of confidence is just meaningless for probabilities that low.

However, I still claim you should not be happy to take this position. First of all, most people do not behave like radical skeptics. So you might allow some events about which you are sure. For those events, a Bayesian approach does not make sense (again, by Cromwell's Rule).

If you are prepared to behave like a radical skeptic, then you have to explain your confidences. What's the ratio between your confidence that fairies exist and your confidence that the sun exploded in the last 8 minutes? If you think that's a stupid question, I agree with you. But you can only say it's a stupid question if one or both of the confidences I asked about is unquantifiable.

You can avoid most of these problems by assigning confidence intervals rather than precise confidences. So, I know that my confidence of fairies existing is less than 1 in 100 million, but I can't make any claims beyond that. It could in fact be 0, or it could be 1 in a billion, or some other number, I can't tell. You can even apply Bayes' rule using interval arithmetic, but it won't tell you anything interesting.

• Interesting last paragraph. The way I try to avoid it is to refuse to assign confidences to any hypothesis period. It seems more consistent with how reality operates, since for example, fairies either exist or not so it seems nonsensical to attach a probability and is unfalsifiable even as an interval. One may respond with the question “how does one differentiate between hypotheses that “seem” more likely than another without confidence?” My answer is certain forms of bets. For example, I may believe that fairies don’t exist and also believe the sun didn’t explode in the last 8 minutes. Jul 7, 2023 at 9:30
• However, GIVEN a certain kind of bet, where I knew apriori that EITHER the sun did explode in the last 8 minutes or that fairies exist, and I was FORCED to choose one, I would bet on the sun exploding. In other words, when we “experience” relatively different levels of confidence in beliefs, that is truly what in my eyes what it translates to. The problem is that we DON’T know these things apriori and thus these bets never have to be made. “Confidence” levels thus to me are actions we would take GIVEN certain betting scenarios. But those betting scenarios don’t usually occur in reality. Jul 7, 2023 at 9:34
• @thinkingman If you accept bets, then you probably accept confidences too. The precision in your confidence is related to the precision in your bet. You can qualitatively say the confidence in the sun exploding is higher than your confidence in fairies existing without committing to a specific number. But if you would take a 10:1 bet, then that is not very different from saying your confidence in the sun exploding must be at least 10 times as big as that fairies exist (conditional on one of them happening). Jul 7, 2023 at 10:30
• In fact, I think to be consistent with your refusal to assign a probability, I think you shouldn't entertain the bet either. There is no logical basis to make a judgement, so you choose not to. Alternatively, you take it as a sure belief that there are no fairies, and your conditional bet doesn't make sense. You probably should reserve sure belief for those events where, if you were convinced otherwise, the rest of your reasoning would be so disturbed that conditioning based on it is a meaningless activity. Jul 7, 2023 at 10:34
• But the bets I described are conditional on the apriori assumption that one of a certain number of claims are true. But we don’t know this to be the case. For example, if I only know that one and only one of Islam, Christianity, or Judaism is true, and I’m forced to make a bet, it may imply greater confidence in Islam GIVEN that knowledge. It does not imply greater confidence WITHOUT that knowledge. Jul 7, 2023 at 11:52

You cannot assign a probability to an event that has already happened. Fairies either exist (probability 1) or they don't (probability 0).

You could, in theory, calculate a probability for a fairy popping up in existence randomly in the future, or for a current species to evolve to a species like fairy. But this was not in the question, which is about fairies existing now.