# How can one make sense of the gambler’s fallacy if probability is ill defined?

The gambler’s fallacy suggests that in the cases of independent events such as coin tosses, the next coin toss’s probability does not depend upon previous ones.

…but there are different definitions and interpretations of probability. If you go the Bayesian/subjective route, your subjective credence is your probability. So if you feel 80% sure that the next coin toss will land on heads, how can someone prove you wrong within that same Bayesian framework? It would be subjectivity vs subjectivity.

So are these fallacies only applicable in a frequentist sense?

• Suppose you get n heads in a row, consecutively. What goes through yer mind as n approaches infinity? Commented May 8, 2023 at 4:18
• > suggests that in the cases of independent events such as coin tosses, the next coin toss’s probability does not depend upon previous ones Suggests? This is the definition of independent Commented May 8, 2023 at 5:15
• “If you go the Bayesian/subjective route, your subjective credence is your probability. So if you feel 80% sure […]” — This is an extreme straw-man description of the Bayesian interpretation. Commented May 8, 2023 at 8:53
• Probability in mathematics is not ill-defined; you're talking about interpretation, not about (formal) definition. Also, in general, being "ill-defined" is not the same as having different definitions around that don't agree. Commented May 8, 2023 at 14:23
• "Bayesian/subjective" That's not what Bayesian means.
– Ray
Commented May 8, 2023 at 19:07

In the cases of independent events (...) the next (outcome's) probability does not depend upon previous ones.

This is general probability calculus. It's not confined to a specific interpretation of probability. It's actually the definition of independence.

The question of interest is whether this applies to real events such as coin tossing. Note though that nothing in probability theory, even using the frequentist interpretation, says that coin tosses (or any specific real events) are in fact independent. Independence in probability calculus is a formal construct. So to make the case that the gambler's fallacy is actually a fallacy, more than probability theory plus interpretation is needed. It turns out that it is rather difficult to argue this, as from a frequentist point of view the best you can get is empirical data that does not contradict independence. You can test independence based on data, but not rejecting it doesn't mean it's true (and in fact there are dependence patterns that cannot be empirically identified, see a recent paper of mine). Also of course Thich Nhat Hanh and other sages say with good reasons that everything is dependent on everything else.

A Bayesian might argue that an exchangeability model (i.e., independence conditionally on the probability parameter) should be used because there is no apparent reason how the order of tosses or the toss number could influence the outcome, but neither is this particularly strong, as it basically says "we assume (conditional) independence because we can't think of anything else". If we want to model our own subjective state of knowledge (or even an "objective" one) as precisely as we can, I'd even argue that such a model should not be chosen, as it implies that any sequence of results, how dependent it may look, cannot make us revise our assumption of independence, which doesn't seem particularly rational to me. (The problem is known to some authors, I'm not making it up as I go, even though Bayesians rarely discuss it.)

It turns out therefore that probabilistic arguments that the gambler's fallacy is actually a fallacy are not particularly strong and convincing.

Despite the problems with diagnosing independence empirically, the best case can probably still be made using empirical data and observing that, even with a big data set, results are compatible with independence. The gambler though may not have a big data set corresponding to the specific game, and may therefore legitimately not be convinced that the fallacy is a fallacy.

In fact, should the gambler claim something specific like "after 10 heads in a row it's at least 10% less likely to have another head than before", this can be "disproved" with pretty small error probability by enough data. (This could be set up as a model selection problem also from a Bayesian point of view, and with enough data the Bayesian may come to very similar conclusions as the frequentist, but it would be more complex and less straightforward.)

• Empirical data, as in real-world, is orthogonal to validation here. A coin's shape, wind, force of tossing and whatnot. The "fallacy" is defined and can be checked a priori. Data plays role if we move into physical realizations, but only in sense of quantifying a model's imperfections, as anything else in physics. A priori (simulated) data is fair game, but its role is itself subject to validation criteria. Commented May 8, 2023 at 19:44
• It seems to me the only useful way to use Bayesian methods here is if you are trying to determine whether the coin (or game) is fair. If you are playing a game where you lose 100% of the time, it's completely reasonable to start to believe that it's rigged and you could use Bayesian methods to formally justify that belief. A coin can have heads on both sides. Dice can be loaded. Dealers can stack the deck. The gambler's fallacy is only a fallacy if the game is fair. Of course, if you think the game is rigged, you probably shouldn't assume that you are 'due' to win. Commented May 8, 2023 at 21:00

If you go the Bayesian/subjective route, your subjective credence is your probability. So if you feel 80% sure that the next coin toss will land on heads, how can someone prove you wrong within that same Bayesian framework? It would be subjectivity vs subjectivity.

Because you can have a set of Bayesian probabilities that isn't consistent with itself. For example, if you assign a subjective probability of 0.6 that the coin lands heads, and 0.7 that it lands tails, then you're wrong, because your probabilities aren't self-consistent. As another example, if you assign high subjective probability to A and to A -> B, but low probability to B, then your probabilities aren't self-consistent. The problem of determining when Bayesian probabilities are self-consistent is in general extremely difficult, at least as difficult as the problem of logical inference.

If you accept that objective reality exists (and reality follows some consistent set of rules), then there would be a single objectively correct probability. Different probability models are closer to and further from this true probability.

So, to argue against you "feeling sure" about some probability (or other probability models), one could compare the demonstrated reliability of different probability models, and present a model that's demonstrated to be more reliable.

Bayesian doesn't necessarily mean purely an arbitrary guess. Ideally, the probabilities we use are based on things we know about reality, to get us as close as possible to the true probability.

It's "subjectivity vs subjectivity" to a similar degree that different distance measurements between countries is subjective. With different levels of knowledge, you can come up with better or worse measurements, but ultimately countries are some actual distance from each other, so any estimation would be more or less wrong. You could also measure the distance from different points in each country, similar to how you could measure the probability of something under different conditions (e.g. flipping a coin may have different probabilities based on how it's flipped and where and how it lands).

* If one were to flip a roughly-symmetric coin a few thousand times, you're highly likely to get heads around 50% of the time. Very few people would disagree with this, and plenty of people have tested this. So it's reasonably assumed that the objective probability of such an event is 0.5.

Some events may not be as easily repeatable. But we don't actually need to ever know what the probability is for such a probability to exist.

From another perspective, if you hold that determinism is true (and possibly even if you don't), one could argue the "true probability" of any given outcome of a single instance of an event (e.g. getting heads when flipping a coin once) would in fact be either 1 or 0, as in: it happens or it doesn't happen. But we often can't know this ahead of time with a particularly high degree of confidence, so we come up with the best estimation of the probability that we can, instead.

In this case, if you take some method of estimating probability, and you apply it to many events, the method would have a certain reliability for predicting the outcomes of those events. If the method is perfect, it would predict every outcome is correctly with a probability of either 1 or 0. How far every prediction is from the correct value of 1 or 0 would represent how wrong the model is, objectively speaking. We can estimate how wrong a model objectively is by checking how wrong it is over a certain period of time (there's also a lot to be said about how to estimate this well, and that's what a good portion of statistics is about).

• One can well hold that objective reality exists but that objective probabilities do not exist. Probability is a very problematic concept also for realists as it always partly refers to events that do in fact not happen. There are many realists that are neither frequentists nor adhere to another concept of objective probability. Note that there also is "objective Bayesianism", which treats probabilities as epistemic, i.e., not objectively existing (only as an objective expression of existing knowledge, which will change with experience). Commented May 8, 2023 at 9:36
• @ChristianHennig I added a footnote expanding on that part of my answer. Commented May 8, 2023 at 10:16
• "If I were to flip a fair coin a few thousand times, the probability is highly likely to be very close to 0.5 of landing on heads. " Isn't this just the definition of a "fair coin", i.e., with any given coin it may well not happen but then you wouldn't call it "fair"? As such, it would be a tautology. By the way, I think you mean "relative frequency" here rather than "probability" as you refer to a finite sequence of observable events. Commented May 8, 2023 at 13:15
• @ChristianHennig "Fair" might be a tautology. I changed it to "roughly-symmetric" (which may be a bit fuzzy, but it's probably good enough), and changed the phrasing to getting heads 50% of the time. Commented May 8, 2023 at 14:11

Probability is subjective because it is based on information and assumptions, which can vary from one person to the next. That, however, does not mean that "anything goes" when it comes to probability. In particular, once you have decided on some set of assumptions, your subjective probability must be consistent with those assumptions, or it is simply wrong. In stating the gambler's fallacy, we assume that the events are independent, so any assignment of probability that takes account history of the events is wrong because it is inconsistent with the assumption.

Now, for any real-world experiment you can debate whether the assumption of independence is appropriate or not, or you could even treat it as an unknown that you must determine by experiment. All of that is outside the scope of the thought experiment described by the gambler's fallacy. If you accept the assumption of independence, then your probability estimates, however subjective they might be in other respects, must be calculated without reference to the event history.

• Ed Thorpe and his associates exploited a lack of independence in blackjack, until the casinos restored it by shuffling the deck more frequently. There was a cost to shuffling, since the punters couldn't place bets while the shuffle was in progress. Commented May 8, 2023 at 20:18

The specific case of successive coin tosses with a fair coin is well-enough understood that a Bayesian approach yields no advantage. Ordinary descriptive statistics is adequate and is not "ill-defined."