3

As a reminder, in Bayesian epistemology, given a hypothesis H and new evidence E, it is recommended to update your degree of belief using the formula P (H|E) = (P (E|H) * P(H))/P(E). P (H) is the prior probability of the hypothesis being true. In Bayesian epistemology, there is the concept of a prior for certain theories. A zero prior is not recommended even for theories that have had no evidence so far. This is because with a zero prior, no amount of evidence would change your degree of belief as per that formula.

But how does this materialize into reality? A zero prior in the case of a hypothesis also means you are simply certain that it isn't true. But why does certainty imply that you can't change your mind? It is not hard to imagine someone being certain that dinosaurs did not exist, come across evidence of fossils, then start believing that dinosaurs did exist.

As such, is it wrong to assume that a 100% degree of belief implies that no amount of evidence can change your mind?

7
  • 4
    No. Bayes's formula does imply that no amount of Bayesian updating can change priors 0 or 1. But Bayes's formula hardly reflects how people actually update their beliefs, they are not ideal Bayesian reasoners. From Bayesian point of view, people are simply wrong to assign 0/1 priors to anything other than, perhaps, mathematical theorems (if that). Hence the prescription to avoid them as priors, which Lindley called Cromwell's rule after Cromwell's quip "I beseech you, in the bowels of Christ, think it possible that you may be mistaken".
    – Conifold
    Commented Jun 14, 2023 at 6:47
  • 1
    100% belief can be confronted by new evidence which makes the believer realise that 100% belief is unjustified. In other words, a certain person can upon encountering new evidence become uncertain. Commented Jun 14, 2023 at 11:10
  • 1
    It's not clear to me how, based on your description of Bayesian epistemology, you arrive at the conclusion that "no evidence for" means you should assign a probability of 0%. In everyday thinking, 0% probability requires not only "no evidence for", but also "irrefutable evidence against". With this interpretation, it becomes easy to arrive at the guidance to assign a small, non-negative prior for hypotheses that seem wrong, even in the absence of any evidence for them.
    – xLeitix
    Commented Jun 14, 2023 at 14:46
  • 1
    Using numeric probabilities for knowledge itself is highly problematic, because you don't know what you don't know, and therefore you can't hope to attach a probability to it that's even remotely accurate (except for 100% and 0% for things you're confident about, perhaps). There's going to be a massive margin of error there.
    – NotThatGuy
    Commented Jun 14, 2023 at 15:50
  • 2
    @NotThatGuy: In base physics, 0% probability events happen. This usually means you set your definitions to be too precise.
    – Joshua
    Commented Jun 14, 2023 at 17:56

3 Answers 3

6

is it wrong to assume that a 100% degree of belief implies that no amount of evidence can change your mind?

You can think of this as a definition rather than an assumption. Within the Bayesian calculus, degrees of belief are handled like this. It is fair enough that you personally would like to handle it differently, however then Bayesian calculus doesn't fit as a model for your thinking.

Accepting it, you may wonder whether you should always avoid assigning 100% probability to anything that is not logically impossible (a Bayesian may say so), but see below.

A zero prior is not recommended even for theories that have had no evidence so far.

In fact, in applied statistics, prior probabilities of one are routinely assigned to nontrivial (and for sure not certain) events such as data being exchangeable, or following certain specific parametric models. In such situations still every isolated observable event can have a probability that is neither zero or one (for example in an exchangeable model with normal distributions), however implicitly assumptions are being made about the underlying process that may not be appropriate.

For example, if you observe binary events, exchangeability implies that if you observe 100 times 1, then 100 times 0, then 5 times 1 in a row, the probability for the next observation to be 1 is the same as had you observed 105 times 1 and 100 times 0 in seemingly random order. This doesn't seem rational, but if in fact you started from a prior assuming exchangeability, whatever sequence of outcomes you then observe, according Bayes' Theorem nothing can switch your beliefs away from exchangeability (in line with what you observed in your posting).

In fact this is a very valid criticism of applied Bayesian statistics (that can be found in some literature). One could respond that indeed "you should not assign probabilities of zero or one to anything conceivable", however in practice this may enforce prior assignments to be so complex that they are ultimately impossible to handle (there are manageable alternatives to exchangeability and even parametric modelling, but they will still exclude with probability 1 certain things that are by no means impossible).

A more practical response is, in my view, to admit that formal probability models are devices for learning rather than meant to be "true" in reality. This includes probability models for (subjective) uncertainty/belief. Such models will always simplify and never match perfectly what they are supposed to model. We may therefore tentatively use a model that assigns probability 0 or 1 to something potentially possible but not certain, as long as it helps us, but keeping in mind that it has the limitation to "forbid" certain things that could in fact happen; and if required, we may drop or update it.

Note that this will violate the Bayesian requirement of coherence, but it is often better to violate a certain "dogma" rather than sticking to it if it becomes apparent that it misleads us. (Some Bayesians would say that the original model, enforcing exchangeability, say, wasn't our "true" model in the first place, but just a convenient approximation of it, and may then claim that dropping it when encountering evidence against it still follows Bayesian logic with respect to some "true model" postulated to exist but not in advance properly formulated by us. This, however, is speculation, and can obviously not be checked because it's post hoc.)

If you have some time in your hand, you may enjoy this (which despite the title also has something about Bayesian/epistemic probability):

C. Hennig: Probability Models in Statistical Data Analysis: Uses, Interpretations, Frequentism-As-Model

4
  • 1
    It's a "life-size model". Just don't try to live in it.
    – Scott Rowe
    Commented Jun 14, 2023 at 11:02
  • @MarkAndrews Your edit has messed up the formatting and I don't see much benefit of it to be honest. I'm happy with punctuation corrections of course. Commented Jun 24, 2023 at 10:32
  • @ChristianHennig. The answer is a good one (I upvoted), but it is buried in a wall of text. Please revise this to make the argument more clear. Commented Jun 24, 2023 at 17:19
  • @MarkAndrews Sorry, I don't see the problem. I don't think the numbering helped a lot, but I wouldn't have rolled it back had the formatting worked properly. If you try again to edit it, please check how it looks like and make sure that all the elements you use are in the right place. Anyway, from my point of view no editing is needed. (I will split the longest paragraph into two, maybe that is a bit of an improvement.) Commented Jun 24, 2023 at 17:54
4

I think we have had a related discussion in which I made the claim that Bayes' theorem is just that- a theorem, so it is mathematically correct; however, as with any faultless method of computation it is still subject to garbage-in, garbage-out. As you indicate, it is possible for priors to be entirely subjective, and therefore to miss the mark. If I take the view that the prior probability of finding a pink unicorn is zero, and then a herd of them is found grazing in the depths of Nottingham Forest, I will conclude that my assessment of the prior was unreliable, and not that there was any problem with Bayes' theorem.

12
  • But in what sense would your assessment of the prior before be wrong? You had never seen pink unicorns and had no evidence for them. Assigning a non zero prior would require evidence. If you can simply assert a non zero prior for unicorns without coming across them, even if there do end up being pink unicorns, it would still be arguably unjustified to assign a non zero prior. But now we're a standstill where it seems to be unjustified to assign both a zero or a non zero prior in the case of undiscovered pink unicorns. Doesn't this indicate a problem with the theorem?
    – user62907
    Commented Jun 14, 2023 at 5:30
  • 2
    No, it indicates a problem with how we judge the probabilities we need to feed into the theorem. Commented Jun 14, 2023 at 5:32
  • 2
    Agreed. There is only any point in applying Bayes' theorem if you can supply it with sensible estimates of the probabilities- ie estimates based on some meaningful rationale that holds water. If all the probabilities involved are just blind guesses, you might as well just guess the overall probability rather than feeding blind guesses into a calculation. Commented Jun 14, 2023 at 8:44
  • 1
    @thinkingman It is not true that "Assigning a non-zero prior would require evidence." If you have no evidence at all, either for or against a proposition, you should assign a non-informative prior, representing complete ignorance. If, on the other hand, by "no evidence" you mean no evidence in favor of a proposition, but possibly some against it, then your prior should be concentrated near zero, but not entirely at zero. How concentrated it should be depends on how much evidence against the proposition you have.
    – Nobody
    Commented Jun 14, 2023 at 13:44
  • 1
    Usually I informally call the non-informative prior 0.5 as "Bayesian zero", since if you do this Bayesian update in the logodds domain, it's simply representing an addition operator, and this Bayesian zero acts as the additive identity. In this logodds domain, 100% positive certainty (probability 1) is positive infinity, and 100% negative certainty (probability 0) is negative infinity. So you can see that no matter how many evidences you collect, you won't ever be 100% certain.
    – justhalf
    Commented Jun 14, 2023 at 15:54
0

What does a 100% degree of belief mean to you, if you don't want to take the Bayesian interpretation of it? You can choose any definition you like for "percentage degree of belief," and you don't have to say it's a probability if you don't want to, but your definition ought to agree with certain intuitions.

First, do you agree that a higher percentage degree of belief requires a higher level of evidence before you would change your mind? Someone with only a 60% degree of belief might be convinced to change his mind just by fairly "suggestive" evidence, but someone with a 90% degree of belief would require much harder proof.

So there is a correspondence between degree of belief and degree of evidence. Someone demanding a higher level of evidence before changing their mind has a percentage higher degree of belief, and vice versa. Maybe you can see where I'm going with this.

You used the example of a person who initially did not believe in dinosaurs changing their mind when given evidence of fossils. But certainly there are Biblical literalists who disbelieve in dinosaurs despite evidence of fossils; they claim the fossils were hoaxes, or were placed by Satan. If the first person initially had a 100% degree of belief that dinosaurs don't exist, what percentage degree of belief would you assign to the second person who refused to change their mind even after the evidence? Also 100%? To agree with intuition, it seems that the second person should be assigned a higher degree of belief than the first.

Since we can't assign a "110%" degree of belief to the second person, the only way to resolve this is to assign a degree of belief below 100% to the first person. So, yes, intuitively, anyone who admits enough doubt to have their mind swayed by evidence should be assigned a degree of belief below 100%.

3
  • In my eyes, one should simply have a complete lack of belief in anything without evidence. I simply do not believe (tongue in cheek) that having a “degree of belief” is ontologically “correct.” Things are either true or not, so why bother having a graded belief? Similarly, I don’t believe in graded evidence. You either have enough evidence to believe in X or you should simply not believe in it. This should be done in an all out or complete sense. I do not believe that fairies exist. I also do not believe it will rain tomorrow even if the forecast says there’s a 60% chance. It’s not enough
    – user62907
    Commented Jun 24, 2023 at 20:10
  • @thinkingman You're the one who introduced the term "100% degree of belief." So it is on you to define the notion of a percentage degree of belief in a way that makes sense to you, or else just accept the Bayesian notion. If you don't want to do either, don't use the term.
    – causative
    Commented Jun 24, 2023 at 21:26
  • Well the question is referring to what it means in the Bayesian sense. The comment I just made was in response to you asking what it means to me so I assumed you were asking about my opinion on degrees of belief in general. Either way, I take a 100% degree of belief to mean having certainty. And I don’t think having certainty implies that you can’t change your mind.
    – user62907
    Commented Jun 24, 2023 at 22:13

You must log in to answer this question.