Is subjective probability falsifiable?

Question

If I say that I have a high credence of belief in A and a low credence of belief in B how do I justify this? If I cannot, should these be dismissed or kept?

Answer 1

Probabilities are a kind of prediction, but not a prediction that can be refuted directly. However, they can be undermined in various ways, depending on the kind of probability you have in mind.

In the case of mathematical probability, (lotteries, dice, etc.), the argument for them might be undermined; that would do it. Nothing else would. These probabilities are a kind of analytic argument in which probabilities are determined by the rules of the game.

In the case of estimated probabilities based on past outcomes, they could be undermined by rejecting all the data. They might also be need to be adjusted as future experience is added to the pool of data - but not outright falsified.

If you have in mind Bayesian probability, which does base its probabilities on "credence" and on single cases, direct falsification does not seem to me possible - nor, for that matter, does direct confirmation.

However, if I cite the evidence from paranormal phenomena or Tarot card readings as increasing the probability that we survive death, you may well object that these phenomena are not any evidence for that conclusion. So although the weight given to a specific consideration for a specific probability may be subjective, a certain level of rational argument can bring that probability into question.

You might mean something slightly different by "subjective probability" - namely, the probabilities assigned in an entirely subjective way, as when someone thinks that horse H will win the race. True, a certain amount of evidence may be cited and the weight given to that might be questioned. But the prediction is often made on the basis of a feeling or intuition. True, the amount of the bet might be taken as an indicator of the probability attributed. Some rational arguments might have relevance, but that is all one can say.

Answer 2

Subjective probability can be shown by data to be miscalibrated, i.e., if somebody makes a sufficiently large number of probability forecasts, these can be evaluated and it can be checked whether event occurrence frequencies are in line with the forecasts as expected given the subjective probability assignments (which obviously allows for some random variation).

This doesn't falsify the subjective probabilities per se; they could still have reflected appropriately the prior beliefs of the subject, however it raises a doubt about whether the subject's probability assignments as a whole have good prediction ability. The subject themself may realise this and may want to change probability assignments even at the risk of violating coherence because reality has cast doubt on the existing assignments.

Note that if subjective probabilities are operationalised by betting rates, coherence is an implication of the Dutch book argument; an opponent could win money against you regardless of what happens if you offer betting rates based on incoherent probability assignments. However, in real betting it may be even more expensive to bet based on coherent but miscalibrated probability assignments, for which reason recalibration may be advantageous despite violating coherence. A subject who does that would treat their own prior assignments as "falsified" in the sense that they decide to abandon/change them.

Literature: A. P. Dawid (1982) The Well-Calibrated Bayesian, Journal of the American Statistical Association, 77:379, 605-610, DOI: 10.1080/01621459.1982.10477856

Answer 3

One presumable criterion for calling a degree of belief a "subjective probability" is that the different degrees involved should follow the axioms of probability theory. A distribution of belief would be seen to be in error if they violated the Kolmogorov axioms needed to count as a probability distribution. This will thus be relevant for the attributation of complex propositions A and B, where there are basic propositions that A and B have in common.

The thesis that degrees of belief ought to follow probability axioms is known as Probabilism, and one standard argument in favour of Probabilism is Frank Ramsey's Dutch Book Argument. In summary, if your beliefs did not follow the probability axioms then you would be inclined to accept a sure loss in joint wagers relating to the proposition you are holding beliefs about.

This has two implications. First, it creates a kind of pragmatic Hypothetical Imperative: if you don’t want to be exploited by ruthless book-keepers, your beliefs should mathematically map to probabilities. Secondly, it points to a kind of inconsistency standard for how you are inclined to form your beliefs: the mere fact that your reasoning is in principle vulnerable to it, whether there are in fact any actual exploitative Dutch Bookkeepers, is itself indicative of a systematic error in judgement.

However, I've always thought there is something somewhat double-edged in this approach, which is that the Converse Dutch Book Arguments (that classical probability theory should be used because such reasoning is not vulnerable in the same way) appear to be too strong. The systematic evasion of vulnerability to the dutch book attack can be approached through deferring to conventional probability, but one might achieve the same ends through being systematically averse to loss and essentially skeptical. If I am always inclined towards distrust, then this may strictly speaking violate the additive laws of probabilities, but it comes with a different kind of protection from exploitation that seems to carry the same kind of weight that is supposed to be provided by the Dutch Book argument for Probabilism.

So I think there is something here, in terms of there being a requirement on your degrees of belief that you assign them in a way that maps on to the mathematical/logical structure of the propositions in question, but one might dampen the force of that point by saying that a general approach of caution in the strength of your beliefs and your willingness to draw conclusions and to risk stakes over them would also be justified.