5

In Bayesianism, every belief in a hypothesis is updated in the same way. You have a prior probability P (H). You have the probability of an observation under a hypothesis P (E|H). And then you update your P(H) using those two factors and P (~H) and P(E|~H) where ~H = the hypothesis not being true.

Now, most Bayesians, even the most objective of ones, refuse to assign a zero prior in general. They recommend assigning a non zero prior to any logically coherent hypothesis. Their reasoning is that otherwise, no amount of evidence would change your mind.

Here is an example of a logically coherent hypothesis: Adam can guess, using psychic powers, what the price of each stock in the world will be at the end of each trading day.

As you can hopefully tell, this isn’t much of a good explanation. It doesn’t explain how he would do this. What would “psychic powers” mean here? What would be the mechanism? It seems to have very low explanatory power.

Now, let us suppose that the next day, Adam gets every single stock price right. If a Bayesian assigns the hypothesis a non zero prior, then they are forced to conclude that his p(H) must increase. Why? Simply because that would be expected if Adam did have psychic powers, and the result by chance otherwise would be very unlikely.

In fact, with enough correct stock guesses, the Bayesian must assume that this hypothesis is more likely than chance. But how can a hypothesis with no explanatory power ever be more likely than chance which at least has explanatory power in the sense of being able to explain the event through normal physical laws, no matter how improbable it is.

The Bayesian of course still has an out here. They can come up with an alternative hypothesis that Adam somehow cheated. After all, if Adam cheated and somehow managed to control the price of every stock in the world (not sure how, but let’s assume so), the data would fit that hypothesis.

But this misses the point. The problem here doesn’t seem to be that Bayesianism allows you to believe the cheating hypothesis. The problem instead seems to be that it allows you to assign a higher overall probability to the psychic hypothesis than chance. But there is no justification for doing so without an explanation.

Now this isn’t the only problem. A Bayesian may still think that chance is more likely than the psychic hypothesis here. However, they may still admit that the psychic hypothesis’s probability should increase. By this, I mean that if an agent had a certain p(H) before observing that Adam got the stock prices correct, a Bayesian may think that an agent should increase their p(H) after the observation.

But here, again, do we have a lack of justification. A particular observation being improbable (note, not impossible) under the chance hypothesis says nothing, by itself, about the likelihood of the psychic hypothesis, even in an infinitesimal sense.

As David Deutsch would put it, science has never been about increasing your credence in a certain belief or maximizing it for true beliefs. Instead, it has always been about good vs. bad explanations and maximizing explanatory power. Does a Bayesian system then give undue credit to theories with no explanatory power that it doesn’t deserve?

22
  • 1
    Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on Philosophy Meta, or in Philosophy Chat. Comments continuing discussion may be removed.
    – Philip Klöcking
    Aug 28, 2023 at 21:27
  • 2
    Does this answer your question? Does Bayesianism not discriminate against ad hoc hypotheses?
    – Roger V.
    Aug 30, 2023 at 8:35
  • 3
    "Now, most Bayesians, even the most objective of ones, refuse to assign a zero prior in general. " is incorrect. Bayesians are perfectly happy to assign a probability of zero to a logical impossibility and one to a logical certainty. "Adam can guess, using psychic powers," is not logically impossible - you can't be certain that it is false a-priori, so you are not representing your true state of prior knowledge if you assign it a probability of zero. Aug 30, 2023 at 9:18
  • 4
    "The Bayesian of course still has an out here. He can come up with an alternative hypothesis that he somehow cheated. " this isn't an "out", the possibility of cheating is part of ~H and should be included in the probability of observing the evidence under ~H. Alternatively, you could have H1 - Adam is psychic, H2 - Adam is cheating, H3 - something else, ... Hn - random chance. The choice isn't simply between psychic and random chance. Aug 30, 2023 at 9:24
  • 2
    I hate to tell you but physics doesn't exist outside our heads either, it is just an explanation for what we see. Nature doesn't care what we think the laws of physics say either. Aug 30, 2023 at 10:49

11 Answers 11

23

The first issue is that the presented hypothesis is actually two hypotheses. While the sentence "Adam can guess, using psychic powers, what the price of each stock in the world will be at the end of each trading day" is logically coherent, it should be broken up into:

  1. Adam can correctly predict what the price of each stock in the world will be at the end of each trading day.
  2. Adam uses psychic powers to achieve (1).

From these two hypotheses, we see two separate avenues of investigation:

First: Test Adam's predictive powers: ask him to predict some number of randomly-chosen stocks at intervals before the close of trading. As the number of correct predictions increases, hypothesis 1 becomes increasingly likely. Further avenues of investigation include attempting to use those predictions to affect the stock price (ie., to see if the prediction takes itself into account), etc..

If Adam passes the first test - that is, if the investigator has a sufficient number of correct predictions - we move on to the second claim. Here, we investigate how Adam predicts the stock prices. We might increase monitoring, possibly to the point that Adam is kept in a sensory deprivation tank for the trading day; we might monitor Adam's use of the Internet or scan for radio devices that might be feeding him information; etc., etc., etc.. The investigator is functionally testing the "psychic powers" hypothesis against "some mundane method that I haven't thought of yet"; at some point, the investigator (hopefully, a team with diverse ideas) may well conclude that there is no other way for Adam to be making his predictions than "psychic powers" (which, to be clear, here means something like "via some mechanism undetectable to the investigating team").

The second issue is the belief that hypotheses must necessarily have explanatory or predictive power. That assumption is quite untrue: a hypothesis is simply a statement whose truth or falsity the investigator whishes to determine. If a hypothesis is proven, deemed sufficiently likely, or disproven, then science starts to ask "why", and searches for a theory that explains the observed behavior. But: the hypothesis itself need not provide explanatory power, it just needs to define an avenue of investigation. The theory of gravity isn't a hypothesis, it stems from hypotheses like "everything falls" and "everything falls at the same rate, if you ignore air resistance".

13
  • 1
    The point is that 2.) can never be shown no matter how many correct predictions are made, as long as those predictions are possible under chance. This is where explanatory power comes in or some observable mechanism comes in. Gravity doesn’t work as an analogy here because we can see the actual process of things falling. There is nothing to see or observe in the case of predictions. From our senses, we are simply seeing John utter numbers and seeing that they match prices. There is no extra physical mechanism being observed. Aug 30, 2023 at 2:11
  • At some point, Newton observed a phenomenon ("stuff falls"). He couldn't explain why that happened, but he was able to gather enough empirical evidence to form some hypotheses ("all stuff falls at the same rate in a vacuum", etc.). He never figured out the causal mechanism, but he got the math right for human-scale purposes. Adam is the same: we may not be able to explain his psychic powers, but we can demonstrate that (1) is true (p > 0.999), and we can demonstrate that any hypothesis other than "psychic powers" is incorrect. It's not a satisfying result, but it's were Newton died.
    – minnmass
    Aug 30, 2023 at 4:41
  • 3
    "The point is that 2.) can never be shown no matter how many correct predictions are made, as long as those predictions are possible under chance." statistics can never prove any explanation to be correct, and does not claim to do so, it just shows that some hypotheses are more plausible than others, given the observations. Aug 30, 2023 at 12:44
  • 3
    "Psychic powers" is ill-defined. But, so were germs when germ theory was new. You keep asserting that "psychic powers" and "extraordinary luck" are different, but there's no particular reason to believe that (see Domino from Deadpool 2 for an example). At the same time, it's so vanishingly unlikely that Adam's predictions are consistently right that "psychic powers" becomes more probable than "chance" - that some as-yet unknown mechanism is at work, which we can call "psychic powers" until we know how it works. "Psychic power" isn't the end of the road, but the start of a new one.
    – minnmass
    Aug 30, 2023 at 14:05
  • 2
    You're conflating your hypotheses again. "Adam can predict the stock market" is analogous to "sick people make people sick". We can demonstrate that Adam can predict the stock market just as surely as we can demonstrate that being around a sick person is likely to make you sick - and with the same symptoms, too! Once that is established, we look at the "why" of it. Adam claims psychic powers; Fracastoro (et. al.) claimed microorganisms too small to be seen by the naked eye can somehow cause illness. Both are clearly ridiculous, except it turns out that one of them is (almost certainly) true!
    – minnmass
    Aug 30, 2023 at 22:04
7

This seems to be less about Bayesian probability analysis and more about forming testable hypotheses.

If we limit your hypothesis to "Adam knows the closing stock prices ahead of time.", Bayesian analysis yields a reasonable result.

An alternative hypothesis would add untestable effects or ones that can result from many causes. For example "Adam uses his psychic powers to not only guess stock prices, but to keep meteors from hitting the Earth." Is every day without a meteor strike a further proof of this?

All probability analysis methods are just a tool, they cannot compensate for defective experiment design or deliberately misleading choice of hypotheses.

6
  • 4
    "All probability analysis methods are just a tool, they cannot compensate for defective experiment design or deliberately misleading choice of hypotheses." Bravo! The OP is indeed presenting a misleading choice of hypotheses by pushing the 'psychic' hypothesis as the only alternative to "no effect".
    – Olivier5
    Aug 28, 2023 at 6:46
  • That’s still an untestable hypothesis. How do you test if one knows something? Correct predictions won’t do. Converting it to a reasonable hypothesis would depend on how one knows. This could be cheating (which we arguably have reason to assign a nonzero prior for) or psychism (which we arguably don’t) or something else Aug 28, 2023 at 6:51
  • 3
    @thinkingman Indeed, your psychic hypothesis is not really testable because it is ambiguous, the meaning of 'psychic' being unclear. What IS testable is the hypothesis that the only factor at play is luck. If that yields a very low probability of generating the observed outcome, then either Adam has been extravagantly lucky, or something else than chance is at play.
    – Olivier5
    Aug 28, 2023 at 7:47
  • 4
    A more likely answer is that Adam (and/or co-conspirators) have a way to control the stock prices. This comes up a lot actually but not in well-run markets. It's pretty rife in the crypto-currency world. There were some traders who made a ton manipulating Libor some years ago.
    – JimmyJames
    Aug 28, 2023 at 20:13
  • 1
    @thinkingman how would the likelihood (the probability of the observations given the hypothesis) differ between the hypothesis that Adam is cheating and the hypothesis that Adam has psychic powers? Aug 30, 2023 at 15:41
6

The correct thought process is, I believe, to start from the so-called Null Hypothesis (H0), i.e. in this case that Adam has no special capacity to predict all these stock values several hours ahead. If Adam's does manage to do so, one day, then there are ways to estimate the probability of H0 based on this one day of correct predictions. Let's say for instance that the Bayesian computes that the probability of H0 is then less than 0.000000001. This in simple English would translate as: the odds of predicting all these values by chance even for one single day, are microscopic. On this basis, the Bayesian would be entirely correct in rejecting H0, and in concluding that most probably, Adam had some way to foretell stock values a few hours ahead, that day.

The question would then become: How did Adam pull the trick? This question would require more than believing Adam's word about it. One could for instance put Adam under strict surveillance and ask him to redo it, to check if he gets messages from the future, gets in some kind of trance, or some other comportmental clue.

1
5

You are making overbroad claims from the rejection of the null hypothesis. The null hypothesis is merely that one cannot predict stock prices at a rate higher than chance. Rejection of that null hypothesis merely means the person is not predicting stock prices by chance. There is no particular reason we should think psychic powers are the mechanism for the non-randomness of the observed outcome.

But wait, you say, I specifically set up the alternative hypothesis to include psychic powers - the hypothesis is that one can predict using psychic powers. If that is the case, you are limited by your own estimation of the likelihood of correct predictions not stemming from psychic powers. To reject the null hypothesis, you'd have to say it's too unlikely that the person was making correct predictions by means other than psychic powers. To do this, you are implicitly stating that you think someone being a psychic is more likely than them being a good cheater.

There is no reason why the information gained from the observations of correct predictions should make any particular explanation more likely. Maybe the person is a psychic, or a cheater, or a time traveler, or a brilliant economist, or anything else - we don't need to know the mechanism, we just know they're not randomly guessing. There's no reason why any and all arbitrary explanations must become more likely by observing correct guesses.

There are an infinite number of possible explanations for how the person is correctly guessing stock prices, it is mathematically intractable that they all become more likely by more than an infinitesimal amount as more evidence becomes available. The evidence indicates some means of non-randomness, but there's no reason we should say that evidence supports any specific hypotheses like psychics, or signals from aliens, or telepathic messages from Bigfoot.

10
  • This is 100% the correct answer. There is absolutely no reason to think that "successfully guessed stock prices" is evidence for "psychic powers" as opposed to, say, insider knowledge. Both Frequentists and Bayesians are still bound by the need for good experimental design.
    – Him
    Aug 30, 2023 at 15:40
  • You guys are both missing the point. The point is that successfully guessed stock prices are not evidence for psychic powers as opposed to even chance! No one denied that insider knowledge isn’t a possibility. But IF cheating wasn’t involved, then it would STILL be reasonable to conclude it was chance, simply because there’s no evidence psychism is possible. Aug 30, 2023 at 20:00
  • @thinkingman There's plenty of evidence that psychism is possible. It's just not thought to be good evidence. But categorically rejecting any evidence that something is possible is not science. If someone can do something that has a 0.00001% of chance, then it's time to stop looking at chance. Much like the OPERA experiment showing neutrinos moving faster than light; at a certain point it's not noise, and it's not scientific to imagine otherwise. If someone can successfully guess stock prices, it's evidence for many things, but not chance.
    – prosfilaes
    Aug 30, 2023 at 20:34
  • Where is the evidence that it is possible? Successful guesses aren’t evidence since they are by definition explainable by chance Aug 30, 2023 at 20:44
  • 2
    @thinkingman What you're describing is a limitation of experimentation. There is no experiment you can design where the only possible explanations are chance or psychic powers. You're suggesting you could design one, find all evidence in favor of psychics and still reject psychic power as the explanation. The problem is, when psychic powers are impossible, and chance is exceedingly unlikely, the most probable explanation is that you ran a bad experiment, not that someone got absurdly and repeatedly lucky. Aug 30, 2023 at 20:56
4

Explanatory power is irrelevant in quantitatively assessing the evidence in favour of a hypothesis. That is because you can only go on what observations the hypothesis allows or forbids, or in some cases we can assign a numeric probability/plausibility to the observations under the hypothesis (the likelihood).

In Bayesianinsm however, "explanatory power" is one of the things we can include when establishing our prior beliefs (which may be subjective and include subjective considerations such as whether an explanation is a good one or consilience with other knowledge). The advantage of Bayesianism is that it gives us a way of explicitly stating and quantifying our prior position and gives a rational means of updating that position as evidence arrives.

If you have a prior on a logically coherent hypothesis that can't be overcome, then you are not rational, but dogmatic. The evidence is irrelevant to your belief. I wouldn't consider that to be a good thing.

“It pays to keep an open mind, but not so open your brains fall out.” ― Carl Sagan

there needs to be a way for evidence to get in!

1
3

Bayesian reasoning doesn't rule out pseudoscience - nor should it. That's not its job. It's important to keep in mind what Bayesian reasoning does, and what it doesn't do.

It doesn't decide correctness, and it doesn't represent strength of evidence. It represents a process for updating probabilities based on evidence in a more formal way than humans would normally do, but in a way similar to how humans do it.

Note that "psychic powers" has strong explanatory powers. Its explanatory power isn't why it's pseudoscience. Indeed, psychic powers are very much testable - the problem isn't whether they're testable, it's what the tests have come back with.

There are precisely two problems with "psychic powers". One of them is the fact that they have failed tests pretty consistently. The other is that no mechanism has been identified for how they would work. However, the second problem is a result of the first one - if tests had determined that psychic powers were real, then attention would be turned to how they work.

What makes psychic powers pseudoscience is a combination of ignoring that experiments have failed to find evidence for them, and the imposition of ideas for "how they work" without any evidence for such a thing.

However, if one admits psychic powers as a hypothetical possibility (that is, if one assigns it a non-zero prior), then Adam successfully predicting stocks would indeed give reason to increase it against hypotheses that don't expect such a result.

However, all of the hypotheses that place such a result at a high likelihood would be similarly boosted, and their relative chances wouldn't change. Which is precisely as it should be - if you feel that "Adam has insider knowledge" and "Adam has a way to manipulate the numbers" are more likely than "Adam has psychic powers", then that will remain true. But Adam correctly guessing all of the stock prices wouldn't, and shouldn't, boost the option of "Adam has no insight into the stock prices", because the likelihood of getting it all correct is very low.

And if Adam kept getting the prices right, repeatedly over many days, "Adam has psychic powers" should be more plausible than "This was entirely just a matter of random chance". But "Adam has insider knowledge" should be far more plausible, and new tests or analyses should be used to distinguish further.

The underlying problem with pseudoscience is the rejection of any test that fails, rather than considering all evidence. If Adam got all of the stock prices right once, that would boost the plausibility of psychic powers. However, if he tried every day for 10 years and only got it right once, then psychic powers should be reduced in plausibility. The pseudoscientific community around the idea of psychic powers would point to the one day and ignore the other ~3600 days.

7
  • I think truth is about explanation, not prediction. Usefulness may be about prediction but when it comes to the truth of a matter, predictions are merely a consequence. Without an explanation for how psychism works, it is never rational to believe it, no matter how many times Adam gets it right. And if the result is possible under chance, it doesn’t even make it more likely than chance. Aug 30, 2023 at 2:00
  • This can be made clear with another example. Suppose you toss a coin 500 million times. The probability of that particular sequence is (1/2)^500 million. Exceptionally low. And yet, this doesn’t increase the likelihood of a hypothesis that says God did it and wanted that to happen. Note that if we apriori had an explanation for how God could do such a thing and why He would have an incentive to, then that does become more likely. Aug 30, 2023 at 2:02
  • 2
    @thinkingman - you seem to be confusing a few different concepts. First, Bayesian reasoning isn't concerned with "truth", it's concerned with interpreting evidence. If someone came to you, said "God has revealed to me that I will flip this coin 500 million times, and the result will be exactly this sequence", showed the sequence written out, and then proceeded to get exactly that result, then it would be the same as if they'd said "God has revealed to me that I'll get 500 million heads in a row" and did that.
    – Glen O
    Aug 30, 2023 at 6:24
  • 2
    The probability of the sequence isn't the relevant part, unless the sequence is itself made relevant (such as it being provided ahead of time). Now, mind you, that person might "just" be psychic, or have developed a technique for controlling the result. But the point is, Bayesian reasoning doesn't seek to address that, and rejecting a hypothesis on the basis that you disagree with it, rather than because the evidence doesn't support it, is pseudoscience. Psychic powers probably don't exist - but to assume they don't a priori is to reject something based on belief rather than science.
    – Glen O
    Aug 30, 2023 at 6:28
  • 1
    Note that this doesn't mean the prior should be high - it just shouldn't be zero.
    – Glen O
    Aug 30, 2023 at 6:29
2

There is no out for pseudoscience here.

When you observe Adam correctly predicting the stock market, you are correct in updating your posterior for him "correctly guessing it using psychic powers". You can represent that probability as a product of the probabilities of the two parts of that statement: P("ability to guess using psychic powers") = P("ability to guess", "using psychic powers") = P(G,P) = P(G)*P(P|G). Since P(G) should obviously increase when you observe repeated correct predictions about the stock market, and the conditional probability P(P|G) "if he can predict the stock market, he would use psychic powers to do so" stays the same*, this means the combined probability P(G,P) needs to increase, unless you would get more suspicious of psychic powers by seeing someone predict stock markets.

Whether the observation has any noticeable effect on your belief in psychic powers P("psychics exist")=P(E), depends on your priors about predicting stock markets and psychics: if psychic powers were the only plausible explanation (P(G|~E)=0), then the experiment would actually "prove" P(E) after sufficiently many correct stock predictions (this is why you should be careful with probability 0). If there are other plausible explanations, the experiment would only give evidence that the individual either cheats or has psychic powers, with the probability ratios being determined by your priors.

* assuming the details of the predictions don't give additional evidence concerning his methods - eg. only being able to predict the stock of a single company might be suspicious, and drop the probability P(psychic|guess) in favor of P(cheating|guess).

1
  • 1
    this is well worded. i suppose the claim in the question is based in the implicit idea that, if there are degrees of belief, then nothing is conclusively falsified. i don't know if popperian philosophy of science thinks anything is
    – user67521
    Aug 30, 2023 at 2:21
2

TL;DR Bayesianism clearly isn't giving pseudoscience an out - if performed even vaguely competently

I'll give a worked example to demonstrate how a Bayesian might actually analyse this problem:

It is a false dilemma to say that either (H1) Adam is psychic or (H2) Adam is guessing, i.e. random chance. There is no reason we can't also have a third hypothesis (H3) Adam is cheating somehow, and compute the posterior probability for all three hypotheses.

We need to start with a prior. Say we are very skeptical that Adam is psychic, being generous, say P(H1) = 0.001. We are more confident that Adam is cheating, but we still have some faith in human nature and want to keep the arithmetic simple, so lets say P(H3) = 0.099. As all of the priors need to sum to 1, this means P(H2) = 0.5. If you are more or less skeptical, feel free to substitute different numbers expressing your belief.

To make it easier, we will get Adam to predict the outcome of 20 consecutive coin flips - he claims he can predict all 20 correctly. So let X be the number of correct guesses, then the likelihood under H1 is:

P(X=20|H1) = 1 and P(X<20|H1) = 0.

Now this would also be the case if Adam was cheating, so

P(X=20|H3) = 1 and P(X<20|H3) = 0.

For random chance, this is a sum of successes in Bernoulli trials, but we are only really interested in the case of X=20, as that is what actually happened, in which case:

P(X=20|H2) = 0.5^20 = 9.5367e-07

Great, we can now work out the posterior probabilities using Bayes rule:

P(H1|X=20) = P(X=20|H1)P(H1)/P(X=20)

and similarly for H2:

P(H2|X=20) = P(X=20|H2)P(H2)/P(X=20)

and H3:

P(H3|X=20) = P(X=20|H3)P(H3)/P(X=20)

Note that the denominator, P(X=20) = P(X=20|H1)P(H1) + P(X=20|H2)P(H2) + P(X=20|H3)P(H3), is the same for all hypotheses, so if we only want to know which is more likely we only need to compute the numerator, which is an un-normalised probability, for which we will use the symbol Q:

Q(H1|X=20) = P(X=20|H1)P(H1) = 1 X 0.001 = 0.001

Q(H2|X=20) = P(X=20|H2)P(H2) = 9.5367e-07 x 0.5 = 4.7684e-07

Q(H3|X=20) = P(X=20|H3)P(H3) = 1 X 0.099 - 0.099

However, it is simply to renormalise the probabilties so they sum to 1 by dividing by Q(H1|X=20) + Q(H2|X=20) + Q(H3|X=20) \approx 0.1, giving (approximately)

P(H1|X=20) = 0.01

P(H2|X=20) = 4.7684e-06

P(H3|X=20) = 0.99

So having seen the evidence, we are pretty sure that Adam cheated (H3), we have pretty much ruled out the hypothesis that Adam was just guessing (H2). The probability that Adam is psychic has gone up, but that isn't surprising because the evidence was very consistent with that hypothesis. Note however the probability of Adam being a cheat is still 99 times more likely that the probability that he is psychic, which is exactly what you would expect as the evidence is equally consistent with both of those hypotheses. The reason both H1 and H3 have increased in plausibility is that we have effectively ruled out H2, which started out as the hypothesis that we believed in most strongly by a large margin.

Seems pretty reasonable to me.

If you think the chance of psychic abilities being only technically possible, try setting P(H1) = 10^88 and adjust P(H2) and P(H3) however you like so they sum to one and see what happens.

I suspect that was a waste of my time ;o)

Caveat lector: been a long day and I'm tired so it is quite likely I have made a mistake somewhere.

1
1

If Adam is consistently predicting stock prices correctly (with people using his predictions making truckloads of money), then Bayes’ theorem will assign a high probability to the hypotheses “Adam can correctly predict stock prices”, but only if you start with a non-zero prior. If you start with prior of zero, then even after a month of correct predictions you will still put his success down to pure chance.

Using a prior of zero is an act of faith. Just like the people of faith who use prior of 1, who would continue to trust Adam’s predictions even if they’re always wrong.

The scientists here are the ones making the truckloads of money.

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Sep 4, 2023 at 16:02
0

In a word, an exception, far from 'proving' a rule, conclusively refutes it. Every genuine scientific theory then, in Popper's view, is prohibitive, in the sense that it forbids, by implication, particular events or occurrences. As such it can be tested and falsified, but never logically verified.

If you agree with Popper that flat earth has been conclusively refuted, then we place a zero percent degree of belief in it. This does not mean Bayesian reasoning is incompatible with falsificationism etc., only that it can be misused into being so (e.g. so that we believe that any of our scientific claims are true).

So in your psychic powers example, sciences should ask

  1. if your claim can be falsified
  2. if it has been falsified

If 2 is the case, then we put zero degree of belief in it. If 1 then it doesn't matter, it's pseudoscience, and while we can apply Bayes to pseudoscience, it will still be pseudoscience. Why would the application of Bayesian theory change anything like that?

1
  • How falsified is falsified? When I was at school I heard the folk version of the "lime juice prevents scurvy" story; according to Judea Pearl (The Book of Why), the lime juice hypothesis was considered to be discredited by 1900CE, in favour of the "fresh meat" hypothesis (the Royal Navy had tried boiling the juice to preserve it, destroying the ascorbic acid, so they concluded that the hypothesis has been falsified). When ascorbic acid was discovered in the 1920s, lime juice was "unfalsified". So it might be a cautious about violating Cromwell's rule. Aug 30, 2023 at 21:25
0

More advanced answer, addressing thinkingman's comment on the OP

I am not arguing for the notion that a zero probability should be attached. I am arguing that a non zero probability shouldn’t be attached. In fact, I am ultimately arguing that no probability should be attached. Probability doesn’t actually exist out there. It only exists in your heads!

Let's start with the last bit "Probability doesn’t actually exist out there. It only exists in your heads!" a Bayesian, especially a subjectivist Bayesian, would say "well, duh!" ;o)

In Bayesian statistics a probability describes your "state of knowledge" or "degree of belief" regarding some quantity or proposition. Yes, it encodes what you know about something, not how reality actually is. This ought to be obvious because in reality a proposition is either true ("Elephants exist in Africa") or they are false ("the largest elephant that ever existed was so large it succumbed to it's own gravity and became a black hole"). There is no probability that a proposition about the real world is true, so what else could it be than an expression of our beliefs about the plausibility of the proposition being true.

So that part of the comment suggests that thinkingman simply doesn't understand probability and has misinterpreted the Bayesian analysis, which says nothing about whether Adam actually is psychic, just what can we rationally conclude about the plausibility of Adam being psychic given our initial beliefs (the priors) and the evidence we have seen (the predictions).

Now if you think your belief about the plausibility of Adam being psychic cannot be represented as a probability, that means the problem is not with the Bayesian framework, the problem is an inability (or an unwillingness) to specify a state of knowledge. Note that not being happy with the posterior is not a good way of deciding an appropriate prior - that should go without saying!

Now in the very basic Bayesian analysis given in my previous answer I took the very simplest option, which was to assign a direct prior probability to each hypothesis and just turn the handle of Bayes rule to get to the posterior, but that wasn't the only thing I could have done.

Another approach would be to say I have no knowledge that us useful for setting the prior probabilities and use a hyper-prior (a prior belief about our prior). Objectivist Bayesians do this all the time to express the state that we know we don't know something. For a prior on a probability, we might use a Beta distribution as the prior:

enter image description here

Note that the Beta(1,1) distribution is uniform on the interval from 0 to 1, so it is a good prior for expressing a complete lack of knowledge of the probability that Adam is a psychic. We can then turn the Bayesian handle again, but this time it will involve performing integrals, so I won't go through the maths here as there is no LaTeX maths translation.

However, the Beta distribution can take on a variety of shapes, so if you wanted to you could pick a hyper-prior that was sharply peaked at zero (Adam cannot be psychic) but still allowed some plausibility to the idea that he might be, and you could turn the handle again.

So there are ways for Bayesians to address the uncertainty that thinking man finds troublesome. However, they will still result in the posterior plausibility of "adam is guessing" going down unless the prior rules out "Adam is psychic" completely.

9
  • I just wouldn’t describe a “degree of belief” as a “state of knowledge”. The latter seems to imply objectivity. Anyways, I’m aware that the Bayesian system represents a degree of belief. But until there’s a principle that actually maps your degree of belief to reality, attaching any measure to your uncertainty doesn’t work for me. If you can’t falsify something, how do you know you’re accurate. How do you show that your 90% degree of belief in something is more accurate than another persons 30% degree of belief in that thing? Sep 2, 2023 at 21:00
  • @thinkingman "I just wouldn’t describe a “degree of belief” as a “state of knowledge”. The latter seems to imply objectivity. ". If I don't know any relevant information about a question and I use a prior distribution that is non-informative, then my conclusion will be objective. I haven't put any "beliefs" into the analysis, just stated that my initial state of knowledge is that I know I don't know anything. Where is the subjectivity/belief introduced? Sep 3, 2023 at 12:53
  • The subjectivity is introduced in the prior distribution. You can’t get an ought from an is. There is nothing in the world that tells you to use a particular distribution. It’s just an opinion you made up. Sep 3, 2023 at 12:57
  • @thinkingman "The subjectivity is introduced in the prior distribution. " if a prior distribution only encodes complete uncertainty (i.e. we know we don't know anything) where is the subjectivity in that? Sep 3, 2023 at 13:34
  • How can any prior distribution encode complete uncertainty? And even if it did, how would you update the belief based on that in an objective way? Sep 3, 2023 at 13:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .