How can one make sense of the gambler’s fallacy if probability is ill defined?

Question

The gambler’s fallacy suggests that in the cases of independent events such as coin tosses, the next coin toss’s probability does not depend upon previous ones.

…but there are different definitions and interpretations of probability. If you go the Bayesian/subjective route, your subjective credence is your probability. So if you feel 80% sure that the next coin toss will land on heads, how can someone prove you wrong within that same Bayesian framework? It would be subjectivity vs subjectivity.

So are these fallacies only applicable in a frequentist sense?

Answer 1

In the cases of independent events (...) the next (outcome's) probability does not depend upon previous ones.

This is general probability calculus. It's not confined to a specific interpretation of probability. It's actually the definition of independence.

The question of interest is whether this applies to real events such as coin tossing. Note though that nothing in probability theory, even using the frequentist interpretation, says that coin tosses (or any specific real events) are in fact independent. Independence in probability calculus is a formal construct. So to make the case that the gambler's fallacy is actually a fallacy, more than probability theory plus interpretation is needed. It turns out that it is rather difficult to argue this, as from a frequentist point of view the best you can get is empirical data that does not contradict independence. You can test independence based on data, but not rejecting it doesn't mean it's true (and in fact there are dependence patterns that cannot be empirically identified, see a recent paper of mine). Also of course Thich Nhat Hanh and other sages say with good reasons that everything is dependent on everything else.

A Bayesian might argue that an exchangeability model (i.e., independence conditionally on the probability parameter) should be used because there is no apparent reason how the order of tosses or the toss number could influence the outcome, but neither is this particularly strong, as it basically says "we assume (conditional) independence because we can't think of anything else". If we want to model our own subjective state of knowledge (or even an "objective" one) as precisely as we can, I'd even argue that such a model should not be chosen, as it implies that any sequence of results, how dependent it may look, cannot make us revise our assumption of independence, which doesn't seem particularly rational to me. (The problem is known to some authors, I'm not making it up as I go, even though Bayesians rarely discuss it.)

It turns out therefore that probabilistic arguments that the gambler's fallacy is actually a fallacy are not particularly strong and convincing.

Despite the problems with diagnosing independence empirically, the best case can probably still be made using empirical data and observing that, even with a big data set, results are compatible with independence. The gambler though may not have a big data set corresponding to the specific game, and may therefore legitimately not be convinced that the fallacy is a fallacy.

In fact, should the gambler claim something specific like "after 10 heads in a row it's at least 10% less likely to have another head than before", this can be "disproved" with pretty small error probability by enough data. (This could be set up as a model selection problem also from a Bayesian point of view, and with enough data the Bayesian may come to very similar conclusions as the frequentist, but it would be more complex and less straightforward.)

Answer 2

If you go the Bayesian/subjective route, your subjective credence is your probability. So if you feel 80% sure that the next coin toss will land on heads, how can someone prove you wrong within that same Bayesian framework? It would be subjectivity vs subjectivity.

Because you can have a set of Bayesian probabilities that isn't consistent with itself. For example, if you assign a subjective probability of 0.6 that the coin lands heads, and 0.7 that it lands tails, then you're wrong, because your probabilities aren't self-consistent. As another example, if you assign high subjective probability to A and to A -> B, but low probability to B, then your probabilities aren't self-consistent. The problem of determining when Bayesian probabilities are self-consistent is in general extremely difficult, at least as difficult as the problem of logical inference.

Answer 3

If you accept that objective reality exists (and reality follows some consistent set of rules), then there would be a single objectively correct probability. Different probability models are closer to and further from this true probability.

So, to argue against you "feeling sure" about some probability (or other probability models), one could compare the demonstrated reliability of different probability models, and present a model that's demonstrated to be more reliable.

Bayesian doesn't necessarily mean purely an arbitrary guess. Ideally, the probabilities we use are based on things we know about reality, to get us as close as possible to the true probability.

It's "subjectivity vs subjectivity" to a similar degree that different distance measurements between countries is subjective. With different levels of knowledge, you can come up with better or worse measurements, but ultimately countries are some actual distance from each other, so any estimation would be more or less wrong. You could also measure the distance from different points in each country, similar to how you could measure the probability of something under different conditions (e.g. flipping a coin may have different probabilities based on how it's flipped and where and how it lands).

---

* If one were to flip a roughly-symmetric coin a few thousand times, you're highly likely to get heads around 50% of the time. Very few people would disagree with this, and plenty of people have tested this. So it's reasonably assumed that the objective probability of such an event is 0.5.

Some events may not be as easily repeatable. But we don't actually need to ever know what the probability is for such a probability to exist.

From another perspective, if you hold that determinism is true (and possibly even if you don't), one could argue the "true probability" of any given outcome of a single instance of an event (e.g. getting heads when flipping a coin once) would in fact be either 1 or 0, as in: it happens or it doesn't happen. But we often can't know this ahead of time with a particularly high degree of confidence, so we come up with the best estimation of the probability that we can, instead.

In this case, if you take some method of estimating probability, and you apply it to many events, the method would have a certain reliability for predicting the outcomes of those events. If the method is perfect, it would predict every outcome is correctly with a probability of either 1 or 0. How far every prediction is from the correct value of 1 or 0 would represent how wrong the model is, objectively speaking. We can estimate how wrong a model objectively is by checking how wrong it is over a certain period of time (there's also a lot to be said about how to estimate this well, and that's what a good portion of statistics is about).

Answer 4

The specific case of successive coin tosses with a fair coin is well-enough understood that a Bayesian approach yields no advantage. Ordinary descriptive statistics is adequate and is not "ill-defined."

Answer 5

Probability is subjective because it is based on information and assumptions, which can vary from one person to the next. That, however, does not mean that "anything goes" when it comes to probability. In particular, once you have decided on some set of assumptions, your subjective probability must be consistent with those assumptions, or it is simply wrong. In stating the gambler's fallacy, we assume that the events are independent, so any assignment of probability that takes account history of the events is wrong because it is inconsistent with the assumption.

Now, for any real-world experiment you can debate whether the assumption of independence is appropriate or not, or you could even treat it as an unknown that you must determine by experiment. All of that is outside the scope of the thought experiment described by the gambler's fallacy. If you accept the assumption of independence, then your probability estimates, however subjective they might be in other respects, must be calculated without reference to the event history.