In Bayesianism, every belief in a hypothesis is updated in the same way. You have a prior probability P (H). You have the probability of an observation under a hypothesis P (E|H). And then you update your P(H) using those two factors and P (~H) and P(E|~H) where ~H = the hypothesis not being true.
Now, most Bayesians, even the most objective of ones, refuse to assign a zero prior in general. They recommend assigning a non zero prior to any logically coherent hypothesis. Their reasoning is that otherwise, no amount of evidence would change your mind.
Here is an example of a logically coherent hypothesis: Adam can guess, using psychic powers, what the price of each stock in the world will be at the end of each trading day.
As you can hopefully tell, this isn’t much of a good explanation. It doesn’t explain how he would do this. What would “psychic powers” mean here? What would be the mechanism? It seems to have very low explanatory power.
Now, let us suppose that the next day, Adam gets every single stock price right. If a Bayesian assigns the hypothesis a non zero prior, then they are forced to conclude that his p(H) must increase. Why? Simply because that would be expected if Adam did have psychic powers, and the result by chance otherwise would be very unlikely.
In fact, with enough correct stock guesses, the Bayesian must assume that this hypothesis is more likely than chance. But how can a hypothesis with no explanatory power ever be more likely than chance which at least has explanatory power in the sense of being able to explain the event through normal physical laws, no matter how improbable it is.
The Bayesian of course still has an out here. They can come up with an alternative hypothesis that Adam somehow cheated. After all, if Adam cheated and somehow managed to control the price of every stock in the world (not sure how, but let’s assume so), the data would fit that hypothesis.
But this misses the point. The problem here doesn’t seem to be that Bayesianism allows you to believe the cheating hypothesis. The problem instead seems to be that it allows you to assign a higher overall probability to the psychic hypothesis than chance. But there is no justification for doing so without an explanation.
Now this isn’t the only problem. A Bayesian may still think that chance is more likely than the psychic hypothesis here. However, they may still admit that the psychic hypothesis’s probability should increase. By this, I mean that if an agent had a certain p(H) before observing that Adam got the stock prices correct, a Bayesian may think that an agent should increase their p(H) after the observation.
But here, again, do we have a lack of justification. A particular observation being improbable (note, not impossible) under the chance hypothesis says nothing, by itself, about the likelihood of the psychic hypothesis, even in an infinitesimal sense.
As David Deutsch would put it, science has never been about increasing your credence in a certain belief or maximizing it for true beliefs. Instead, it has always been about good vs. bad explanations and maximizing explanatory power. Does a Bayesian system then give undue credit to theories with no explanatory power that it doesn’t deserve?