118

I have always been perplexed by a seeming paradox in probability that I'm sure has some simple, well-known explanation. We say that a "fair coin" or whatever has "no memory."

At each toss the odds are once again reset at 50:50. Hence the "gambler's fallacy." After 10 heads, the odds of another head are still said to be 50:50. The same after 20, 40, 80... heads.

Yet we also know that the series will converge upon an equilibrium of heads:tails. And indeed this is countable in fairly short order. The convergence appears pretty quickly.

How can both be true? Isn't there something in the physical series of tosses that "remembers"? Isn't there necessarily some slightly better chance of a tails after 10 heads?

How does logic resolve this absolute randomness in the particular events with a general law of convergence? I imagine this must be a well-known issue. I suppose it raises the larger issue of what sort of "causality" probability is.

Note that I do not know symbolic logic so, embarrassingly, formal demonstrations are beyond my ken.

4
  • Mod deletes comments. Please take extended discussion to chat. (Comments are for clarifying the question; answers go in an answer.)
    – Joseph Weissman
    Jan 7, 2016 at 18:17
  • The root issue of gambler's fallacy lies in missing the knowledge of statistical independence. Most gambling event during a sequence is independent, thus any event's conditional probability during the sequence is trivially same as its a priori unconditional one. Another aspect is law of large numbers manifested in any sequence which converges fast after over maybe 30 samples for your simple coin toss situation due to its simple Bernoulli probability density... Apr 17, 2021 at 3:01
  • 3
    After seeing 10 heads, there's a slightly better chance that your coin is not actually fair. Mar 6, 2023 at 18:39
  • This is really a math question. But there are philosophical implications to probability. The simple answer is that, if you assume a fair coin (and you have), the probability of heads is 0.5 each and every time. If you toss n times, you will get approximately n/2 heads. The higher the value of n, the less approximate the proportion of heads will become. So if you toss a billion times, the proportion of heads will not deviate significantly from half a billion. The gambler's fallacy arises with smaller numbers. If you predict that you will toss twenty heads in a row, then the chance is one in a m
    – Meanach
    Nov 16, 2023 at 18:21

22 Answers 22

112

Since you have asked for a non-formal answer, I shall try to oblige by not using any numbers or equations.

Fundamentally, your question is, how does it come about that individual events can be completely unpredictable but when you pile a lot of them together, either in a sequence or in a mass, the behaviour of the whole pile becomes, if not totally predictable, at least substantially predictable? The answer is something called the law of large numbers, and it is one of the most fundamental concepts in statistics.

As an illustration of it, imagine something called a Galton box: it is a triangular shaped box standing vertically, with its base on the ground and one vertex at the top. There is hole in the top to allow a ball to be dropped in. A series of pins or pegs are placed such that a ball falls either to the right or left in an unpredictable way until it reaches the bottom. As illustrated in this diagram, when lots of balls are dropped in, the result is a heap in the middle. We cannot predict where a single ball will fall, but put enough balls in and we can be increasingly sure that we'll get a bell-shaped curve, simply because it is very unlikely that a ball will consistently move left, or consistently move right. One way to think of it is to count the possible paths to a point on the bottom. There is only one possible path to get all the way to the right, or all the way to the left, but lots of paths will take a ball to the middle. enter image description here This means we don't need to suppose that a ball is remembering the previous falls. Each ball is independent, and the resulting curve (a binomial distribution) emerges spontaneously from it. This is one of many examples of how apparently orderly behaviour can emerge even when there are lots of disorderly things going on at the micro level. Another is radioactive decay: we cannot predict when one atom will decay, but with a large mass of them we can predict very precisely what proportion of them will decay in a given time interval. Another example arises from the kinetic theory of heat: we cannot predict how individual molecules move around, but put enough of them together and we can say all kinds of useful things about their thermodynamic properties.

So the gambler's fallacy is a real fallacy, even though it is perennially tempting. My favourite way to test peoples' intuitions about it is to ask them this: suppose I decide to play the lottery every week and my preferred strategy for picking the numbers is to look up the numbers that won last week and choose those. You will find many people who think this is crazy because the chances of the same set of numbers winning two weeks running is tiny. But of course the probability of any set of numbers winning is all equal: it is not affected by the previous week's win.

21
  • 4
    In that space of the all, we can see any other process is nudged by its history; by continuity, we now see that memoryless processes are also nudged - they're nudged out of anything that looks like it has a history! Dec 14, 2015 at 1:28
  • 21
    In memoryless cases like the Galton box, the nudging towards convergence is not a feature of any particular ball - the balls are not nudged - but rather the tendency towards convergence is an emergent property of the whole system. It is related to the concept of regression towards the mean: if you have some atypical results, subsequent measurements are more likely to be closer to the mean than even further away from it.
    – Bumble
    Dec 14, 2015 at 3:38
  • 15
    The irony in your lottery example is that it's actually a decent strategy to pick the numbers that won the last time, precisely because most people will think those have lower chance to win again. So if you do win, there's a good chance you will not have to share your prize :P Of course, given the odds, it's still absurd to participate in the lottery in the first place...
    – Luaan
    Dec 15, 2015 at 15:24
  • 18
    @Luaan Exactly. One of my coworkers put it well: "The lottery is a tax on being bad at math."
    – reirab
    Dec 15, 2015 at 15:25
  • 33
    If you are to toss 1000 times, and after 100 tosses you have 60/40 heads/tails (a 20% advantage), then when you reach 1000, the most likely result would be 510/490, not 500/500. Every future toss is independent, so the 20-toss advantage heads has won't disappear - It will just become small in comparison to the total number of tosses (only 2%). If the gambler bets on the next toss favouring heads or tails, that's a fallacy. But if he bets on the total number of tosses favouring heads he's actually on pretty safe ground. The fallacy is to confuse the two.
    – Ben
    Dec 16, 2015 at 10:59
43

If the probability of heads = p , then the probability of tails = 1-p . If it's a fair coin, then p = 1-p and the probability of either heads or tails is p = 1/2.

Now suppose the number of coin tosses is N, and let's say that N is getting pretty large. The expected value of the random variable that is the number heads out of the N tosses is going to be around the mean Np, which for an honest coin is N/2.

The variance of the random variable (the total number of heads out of N tosses) is Np(1-p) (which, for the honest coin, is N/4) which is the square of the standard deviation. This means if N is increased by a factor of 4, then the standard deviation only increases by a factor of 2.

So as the number of tosses increases, the deviation of the number of heads (which is sqrt(N)/2)) from the expected mean (which is N/2) does increase, but not as fast as the number of tosses increases. When you divide by N, the percentage of that expected deviation, inside the total number of tosses, gets smaller and gets closer to the expected 50%. This is because it's (sqrt(N)/2)/N = 1/(2 sqrt(N)) .

From a percentage POV, it looks like you're getting closer and closer to what is expected from an honest coin.

From a count POV, it doesn't look exactly the same. If you toss an honest coin 1,000,000 times, the number of heads will likely be some distance away from 500,000. But the percentage of the number of heads out of the total number of tosses will be very close to 50%. And it will get closer to 50% with more and more tosses, but the absolute distance away from the 50% mark will grow at a rate proportional to sqrt(N). But the number of tosses is growing at a rate of N.

10
  • 14
    I like how you explicitly call out that the deviation from 50:50, in absolute numbers, is not expected to decrease in the first place.
    – Dan Getz
    Dec 13, 2015 at 23:21
  • 5
    Thanks, this looks clear and accessible, due only to my amateur education I'll have to work through it slowly. Perhaps the focal point of my question simply makes no sense, and I am not even sure if you have or haven't answered it. If we have 9 heads or 99 heads, in that finite series there is still exactly 50:50 chance of heads in next toss. So "convergence" has zero causal pull. As with Hume's radical critique of induction, it just seems something is wrong. Dec 14, 2015 at 1:35
  • 3
    "If we have 9 heads or 99 heads, in that finite series there is still exactly 50:50 chance of heads in next toss" -- yes. if it's an honest coin. Dec 14, 2015 at 2:46
  • 1
    Why is the probability of tails 1-p? If there are two sides to a coin aren't the probabilities of each the same? Dec 15, 2015 at 4:55
  • 5
    @JohnPeters, i was being very general (it's called the Binary Random Variable). a dishonest coin would not have the same probability for heads and tails. but assuming the coin never lands on its edge, it has to be either heads or tails so the probability of heads and the probability of tails must add to 1. Dec 15, 2015 at 5:36
28

The convergence appears pretty quickly.

This is your faulty assumption. It does apear pretty quickly. In most cases. But not at all every time.

There are in some sense two layers of likelyhood: In layer one, every single event has the very same probability as its precedessors. In layer two, the sequence of events as a whole has a probability to occur. And every single sequence of a given length has the very same probability to occur, ie. HTHTHTHT has the same probability as HHHHHHHH (H=heads, T=tails), which is 0.5^8. It is only because across all possible sequences of a given length the number of heads and tails is 50% each that generally, a sequence of independent throws converges towards these frequencies. And of course, there are many, many more series of the length of eight tosses which contain at least one tails, which makes us think that there will occur tails pretty soon.

The problem is that you never know which sequence you are in, as it were. That is why, looking to the future, only the probability of the single next event is what should count for the gambler. The improbability of the 11th heads after 10 times heads is purely subjective because there are so many more sequences with at least one tails, but it is still 50%. After all, having 10 heads in a row is the improbable event, not that the next throw will be heads again. But, well, it still has happened, so it makes no difference as for the next throw.

You have to see what exactly the event (and object of probability) is. In the coin-example, the series so far is an event that has occured. So there was a probability for this series to occur before, but now, as it has happened, there is only a frequency of occurence left which we already know. The only probability in the strict sense of the word, which is about predicting future or at least unknown states of affairs is that of the next event or of the upcoming series. As soon as the next toss is made and we have seen the result, there is only the probability of the now next event and the now following possible series.

The fallacy consists in assuming that because there are ever more possible sequences with at least one tails the longer the overall sequence gets the probability of tails after him experiencing a huge number of heads would increase. But no matter how improbable the sequence he already encountered was, it is a category error to apply probability to past and known events, ie. "his" sequence +1 toss compared to all other possible sequences of this length. For each toss, the probability is still 50%, no matter what happened before.

Probability proper does only make sense for/applies to future or unknown matters of facts!

As a sidenote the following "counter-example": Consider three doors, you choose one that has the "probability" of containing the prize of 1/3. Now one door is opened and you have the choice to change the chosen door. What are the chances? Well, you should definitely change, because your door now has the "probability" of 1/3 as before, but the other has 2/3. Here you have to consider the whole series, there is no contradiction. That is because there is no probability anymore: The prize already is behind one door, the event has happened. That is the difference.

TL;DR: Edit and Conclusion

So the fallacy, as expressed by @wedstrom in his comment, is to think that nature will correct itself, will let it happen that the series in progress will converge quickly. But nature is not an actor that does anything. And in the present, there is only past (occured/known events, frequency) and future (upcoming/unknown events, probability). Thus, if the probability is independent, this has to be taken literally as independence from anything that happened in the past, no matter how scarce the occurence of the resulting overall series does appear.

15
  • 2
    The latter is the called, at least in English, the Monty Hall Problem. It too is very unintuitive, and even many mathematicians, including Paul Erdos, refused to believe the "correct" answer. Though I accept the answers, I do not really understand how to think of "probability" and cause. I suppose I am thinking of the convergence as a kind of "attractor." Many people actually do 100 or 1000 flips, and we live in a part of the universe where they do quite evidently converge. So it still seems that in "normalcy" 99 heads 1 tails is more probable "in any next instant" than 100 heads. Dec 13, 2015 at 22:19
  • 1
    @NelsonAlexander: Added interpretation of probability for both problems for better understanding. Probability is only there as long as it refers to future events. After that, that means after the sequence/event has occured, there is only frequency. So inferring probability from occurance isn't wrong, but the probability of an event cannot be inferred from its occurences that happened just before, i.e. the frequencies, if the events are independent.
    – Philip Klöcking
    Dec 14, 2015 at 11:24
  • 4
    Not sure you explained the Monty Hall problem correctly. The explanation that makes the most sense to me intuitively is: when you pick a door, there's a 2/3 chance the other doors contain the prize. Since the host opens one of those two doors, you know that the door remaining 'picks up' the other 1/3 chance and retains an overall 2/3 chance of containing the prize and thus you should pick it. An intuitive version of the problem is using 1,000 doors. If you picked 1 door and the host opens 998 of those remaining that don't contain the prize, you would definitely switch to the last door.
    – Mdev
    Dec 14, 2015 at 21:06
  • 1
    You point out something that's really important - and I think the main point of the problem. The fact that the gambler is "self-selecting" improbable sequences, that is, by the time the gamblers fallacy applies, the unlikely 7 in a row loss is in the past. The fallacy is expecting nature to take corrective action. Or in other words, believing that an 8 in a row loss, because it is unlikely in general, is less than 50 percent likely after a 7 series loss in a fair coin toss.
    – wedstrom
    Dec 14, 2015 at 21:46
  • 2
    @Matthew: Actually what is missing in most explanations of Monty Hall is the importance of the fact that the game show master knows and actively avoids the door with the price. The importance of that gets immediately clear if you think of a sahow master with the opposite goal: He always chooses the door with the price if available. In that case, if he chooses an empty door, you should most definitely not change, as if the prize had been behind the other door he certainly would have opened that. Not changing therefore gives you a guaranteed win. And it's not too hard to show that if the …
    – celtschk
    Dec 15, 2015 at 22:00
20

Yet we also know that the series will converge upon an equilibrium of heads:tails.

I think this is your central problem. This is indeed the most probable result of a series of coin tosses, but probability doesn't apply to things that are already known to have happened.

Imagine this game:

A coin is tossed 100 times. Gamblers can bet on the total number of heads that will be thrown. They can do so at any time before or during the game.

Imagine that you're betting before the game starts. Your best bet is obviously 50 heads (50% of 100 future tosses).

Now imagine that you're betting after the coin was already tossed 10 times and came up heads all 10 times.

What is the best bet now? According to the gambler's fallacy, the coins should even out and so the best bet should still be 50. But in reality, the most likely outcome for the future tosses is still 50% heads, and we already have 10 heads, so the best bet is 55 (10 known heads + 50% of 90 future tosses).

7
  • Great! So you are arguing, it seems, that my "problem" was real. In the "finite wager" with a "history" the next toss odds do change. But you have not fully differentiated between the mathematical average and "in reality." Yet this does seems to be getting closer to the core of my confusion. Dec 14, 2015 at 3:47
  • 4
    No, the next odds don't change, they're still 50:50. What changes with every toss is the odds for the whole series, including the already known tosses.
    – zocky
    Dec 14, 2015 at 4:07
  • 2
    Yes, this is the right answer. If you toss a coin 100 times, the most likely result is 50 heads and 50 tails, GIVEN that you have not yet tossed the coin, or that you don't know what the results of any tosses made were. But AFTER you toss the coin a few times, the most likely probability is NOT 50 and 50. Now you have more information, which changes the calculation.
    – Jay
    Dec 14, 2015 at 6:55
  • 4
    And by the way, in the real world, if I tossed a coin and it came up heads 20 times in a row, my money would be on the next toss being another head. Because at that point I'd think, maybe this is a trick coin that's weighted or something so it always comes up heads.
    – Jay
    Dec 14, 2015 at 6:56
  • 1
    This is a great answer, probably the best on the page, because it concisely illustrates the concept using a very tangible real-world example. Nicely done, thanks!
    – Brad Werth
    Dec 17, 2015 at 7:28
19

If you use a fair coin, the average of heads thrown will converge to 50%. However, the number of heads won't converge to half the coins thrown.

While the percentage comes closer and closer to 50%, usually the number of coins will diverge more and more from exactly half. How can this be? Throw ten coins. You'll probably get 3 to 7 heads. 30% to 70%. Throw 1000 coins. You'll probably got 450 to 550 heads. 45% to 55%. Even though you are closer to 50%, you are actually further away (50 instead of 2) from having exactly half the coin throws being heads. No memory is needed. Your percentage comes closer to 50%, even though you actually deviate more.

Now throw 1000 coins, and then throw another 1000 coins. Say each time you have between 45% and 55% heads. But since there is no memory, there is a fifty percent chance that in the first 1000 throws you had less than 50%, and in the next 1000 throws you had more than 50%, or the other way round. In that case, you get a lot closer to 50%. For example, 45% + 55% means exactly 50%.

2
  • 2
    This is the crux of the misunderstanding, yes. "Converge to 50/50" is ambiguous as a phrase. And in fact the absolute number does not converge, only the ratio converges. It is a "conflation in language" error.
    – Jeff Y
    Dec 18, 2015 at 17:23
  • What do you mean by "However, the number of heads won't converge to half the coins thrown."? Which mathematical notion of convergence are you using? (i'm familiar with a sequence converging towards a number, and with two sequence being equivalent, but you seem to mean converge as in "difference goes to zero")
    – Clément
    Dec 19, 2015 at 22:59
11

This is really math, not philosophy.

Assume that you've tossed the coin so far m times and gotten n heads. The fraction of heads so far is n / m.

Now you toss the coin one more time.

There is a 50% chance that the toss is tails and the fraction becomes n / (m + 1), and a 50% chance that the toss is heads and the fraction becomes (n + 1) / (m + 1).

By linearity of expectation, the expected fraction after the additional toss is then (n + 0.5) / (m + 1).

Now you can verify that if n / m = 0.5, then (n + 0.5) / (m + 1) = 0.5 as well — if we've had an even run so far, then the expected value after one more toss remains even.

If 0.5 < n / m, then 0.5 < (n + 0.5) / (m + 1) < n / m.

If n / m < 0.5, then n / m < (n + 0.5) / (m + 1) < 0.5.

In other words, if we've had an uneven run so far, the expected value after one more toss is slightly closer to even than it was before for no other reason than that the denominator of the fraction increases at a faster pace than the numerator does. You can start out getting 100 heads out of 100 tosses, but 100 independent tosses later you should expect to be at 150/200, which is closer to 50%. And 800 tosses after that you should expect to be at 550 / 1000. The excess is 50 in all three cases, but the percentage of excess got smaller.

11

Because "converge to an equilibrium" doesn't mean an exactly equal number of heads and tails, it means the proportion of heads to tails approaches equality (with probability 1: the meaning of which hides all the mathematical formalism to deal with the possibility of other results). In fact the probability of an exactly equal number of heads and tails after an even number of tosses tends towards zero with more tosses.

Ignore for a moment that there's an initial run of heads. Just start with the score "heads: 10, tails 0", and a fair coin. Then the score still "converges to an equilibrium", because the more coin tosses you make, the smaller is the proportional difference made by the unfair advantage of 10. You're happier to give someone a 10m headstart in a marathon than in a 100m sprint, and in effect tails is happy to give heads any amount of headstart in an "infinite race". As you approach infinity all fixed constants are small, the probability that tails has caught up with heads at least once along the way approaches 1, the probability that tails is ahead approaches 0.5, and that's all we mean by equilibrium.

The same goes for any initial sequence of coin tosses. Whether it's even or not, it gets buried by the unbounded sequence of coin tosses that comes afterwards. Consider, if you have the mathematics to do so, that the limit as x approaches infinity of (x+1)/x is 1. The numerator is given a "headstart" over the denominator, but it makes no difference to the limit.

10

You are comparing two different cases. One is "the probability of landing heads on the next flip" and the other is "sum of the number of heads." The latter is governed by the Central Limit Theorem, which explains why the sum converges so rapidly (in many cases). Summing acts very differently than simply asking "what's the next result," and its the summing that causes the convergence.

From the perspective of freeing ourselves from this "paradox" the key is that for every case where we have N tosses that landed heads, we also have a corresponding case where we have N tosses that landed tails. From the perspective of "sum of the number of heads," this matters. In the case where we discus "the coin has landed heads up 10 times in a row," it does not, because the fact that we have stated it has landed heads up 10 times precludes us from considering the case where it landed 10 times tails up. The 10 tails case doesn't have any effect on our discussion of the next coin flip because it simply didn't happen. We aren't interested in it.

It's a bit easier to visualize the non-paradox if, instead of counting the number of heads and tails, we assign heads and tails numeric values (such as +1 and -1) and take the average. Most humans find it easy to intuit that the average of a sample will approach the average of the random variable as N gets large.

This visualization can be done in many ways. One way is to look at all the different sequences of heads and tails that can occur. Clearly each sequence occurs with equal probability (with a fair coin). However, when you put these into "bins" based on how many heads you see, you find that there are many more sequences with an "average" number of heads than those which have extraordinary numbers of heads. This causes us to see average numbers more often than extraordinary numbers.

To give a concrete example, the strings of length 3: 0 heads = 1 string ({T, T, T}), 1 heads = 3 strings ({H, T, T}, {T, H, T}, {T, T, H}), 2 heads = 3 strings ({H, H, T}, {H, T, H}, {T, H, H}), 3 heads = 1 string ({H, H, H}). 8 total strings, each with a probability of occurring of 1/8. Thus, by addition, probability of 0 heads = 1/8, 1 heads = 3/8, 2 heads = 3/8, 3 heads = 1/8

8
  • Sounds right but no light bulbs click on. I'll just have to chew on it. At least I'm told than probability is not very intuitive, even for some scientists and mathematicians. Dec 13, 2015 at 21:59
  • 1
    It might help to write out a tree of coin tosses, tracking whether each toss was heads or tails and how many heads you've seen. The trees start to merge together (1H 1T is the same as 1T 1H).
    – Cort Ammon
    Dec 13, 2015 at 22:04
  • Your answer engages most directly with the gambler's fallacy and my question, because it includes a finite "history." Given the finite series 9 heads, the next toss is still exactly 1/2 heads. Which seems like Hume's induction problem in reverse. The "tail" that nudges back towards average could drop in anywhere in some assumed "infinite" series. Yet it still seems to me there is something problematic here. As I put it to myself, something "remembers" that it must converge towards 50:50, what is that something? But I would be the first to admit this is likely a case of pure confusion. Dec 14, 2015 at 2:11
  • 1
    Another viewpoint might be that every coin toss is "independent," in that nothing is being remembered from coin toss to coin toss. If I have seen {H, T}, and flip a coin, I am equally likely to see {H, T, H} as {H, T, T}. However, when I take the sum of heads, I "lose" some information because I'm binning these strings of results by "number of heads." If you pick any finite length string, and enumerate the number of series's with 0 heads, number of series with 1 heads, number of series with 2 heads, etc, you find that the number of ways you can make a string with an "average" number of...
    – Cort Ammon
    Dec 14, 2015 at 2:49
  • ... heads is much higher than the number of ways you can make a string with a more extreme number of heads. Thus, if each particular string is equally likely to occur (true, since the coin toss is random each time), when you "bin" them based on number of heads, you find it is more likely to see an average number of heads because that bin contained more strings, and each string had equal probability of occurring. Does that help?
    – Cort Ammon
    Dec 14, 2015 at 2:51
8

You are right: After a series of 10, 20, 40, 80 heads the probability for another head is still 1/2. It is not slightly less or slightly bigger, it is constantly 1/2. Tosses have no memory.

To reconcile this result with the naive expectation one should take into account: The probability of a series of length 10 with 10 heads is (1/2) ** 10, which is about 1/1000. i.e. the probability to get such series when always making 10 tosses is 1/1000.

And the probability of 80 heads is accordingly (1/2) ** 80, which is about 10 ** (-24), a decimal with the ciffer 1 at position 24 after the decimal point.

Hence the contribution of such exceptional series to the limit on all series of equal length is exceptionally small.

3
  • Thank you, I accept these answers though instinct flinches. I'm afraid I don't know enough even to know if you have already answered this. But in, say, a finite series of 40 heads, any next event is still "exactly" 1/2 heads. No change due to this given history. (The "gambler's fallacy" actually takes place within a given finite "winning streak," a history.) So I am having difficulty with the convergence as a clear, empirical "tendency" that has zero causal effect. This is like Hume's induction problem versus common sense. Perhaps one must simply develop a feel for the math. Dec 14, 2015 at 1:47
  • So you had 10 heads in a row, it now looks like your coin is 100% biased towards heads. Toss it another 90 times and lets say it now lands 1/2 heads (so 45). Now your coin looks like 55/45, heads/tails. Toss it another 900 times and it'll look like 50.5/49.5 heads/tails. The convergence comes because, as @robert pointer out, the number of tosses grows faster than how uneven your coin lands.
    – csiz
    Dec 14, 2015 at 11:08
  • @Jmoreno I edited my answer.
    – Jo Wehler
    Dec 16, 2015 at 5:51
8

To build on what celtschk pointed out (and possibly others, I haven't read all of them) with more examples, 'tend towards 50/50' is not something as in the next n throws will negate any off-set that is currently in place, it's rather, when n gets big enough any current off-set becomes insignificant.

I.e.

Let's assume you somehow manage to toss 100 coins and get 100 heads, but from now on, for arguments sake, let's say the coin tosses split exactly 50/50.

This means at 200 tosses, we'd have 150 heads and 50 tails, still biased to heads.

At 500 tosses, 300 heads 200 tails, still biased to heads, but less so.

At 10000 tosses, 5050 heads, 4950 tails, this is almost 50/50.

At 1000000 tosses 500050 heads 499950 tails, with that many tosses, this has effectively converged on 50/50.

This is the convergence you see, the error that is there initially just becomes insignificant the more tosses you add. There is no 'slightly higher chance' of tails.

1
  • Like with Ross' answer, a good explanation of the key issue to the point. Dec 21, 2015 at 13:59
7

You need to be careful to specify the question you are asking. Going forward, the coin has no memory and the chance of heads on any given toss is 1/2. Period. End. The convergence to the mean is because any excess that you have now will be washed out in much larger numbers. Say the first ten tosses come up heads. At this point, if I asked the most probable number of heads after 100 tosses the answer is 55. This is a little high. If I asked the most probable number of heads after a million tosses it is 500005, while before the first 10 flips it was 500000. As the standard deviation of the number of heads in a million flips is 500, an excess of 5 is no big deal. This is what the law of large numbers says. No matter what excess you have now, if you make enough more flips it will be very small compared to the standard deviation of the rest of the flips. Nothing makes it get closer to the mean, but the excess gets washed out when you consider the average.

1
  • 1
    I think the "deluting" of any early chance aberrations in the large number of later tosses is the key and well explained here. No "active correction" is necessary, the noise just becomes insignificant if the underlying tendency (here: 50/50) is given enough time to work. Most of the times :-). Dec 21, 2015 at 13:57
6

Let's assume you have tossed ten heads, and you are about to do a million further tosses. What is the expectation of the difference between heads and tails? Well, it's ten, because you already have ten tosses, and the expectation for the future tosses is as many heads as tails.

Let's for the moment assume that in the next million tosses, you get exactly half a million heads, and half a million tails. This means the difference turns out to be exactly the expectation, as with the first ten heads, you have ten more heads than tails.

However, if you look at the percentage of heads, you'll find that since 500,010 of 1,000,010 tosses were heads, you've got about 50.00005% heads, and 49.9995% tails. So that's pretty close to equal.

But of course it is not exactly the same numbers of heads and tails. Isn't that a problem? Actually, rather the opposite: If in a million tosses, you get exactly half a million heads, and not a single one more or less, you should get suspicious. Because the probability of exactly half a million heads in a million independent tosses of a perfectly fair coin only is about 0.032%. Even worse, that probability even shrinks as the sequence gets longer, and in the limit of infinitely many tosses goes to zero.

The result from a random toss sequence of a fair coin will likely be somewhere around equally many heads and tails. Indeed, that range of head counts likely to be found even grows with more coin tosses. It's just that it grows slower than the number of tosses (that is, if you do twice the number of tosses, the range you'll likely find the number of heads is not twice at large; indeed it is only sqrt(2) times, or about 1.4 times at large), and therefore the range for the fraction of heads goes down.

Now that growing range of likely head counts means that with enough tosses, your initial ten heads will indeed be completely inside the range of likely counts, and that range will eventually be so large that the ten counts are negligible compared to the deviation caused by the random tosses.

6

Series will generally converge but there is always a small probability that a serie does not converge after a finite number of trials, so there is no contradiction. If you already had 100 tails the whole serie will converge more slowly. The interpretation of probabilities (degree of credence? Objective propensity? Frequency?) is an independent matter.

1
  • 2
    It is nice to note that there is no agreed interpretation of probabilities. Perhaps one should add that that they are used 'normatively' like the principle of inertia: it cannot be observed but deviations are explained. If an experiment does not fit probabilities, it is of course that the coin or the dice is biased, or the hand that throws them or whatever.
    – sand1
    Dec 13, 2015 at 21:59
5

How can both be true? Isn't there something in the physical series of tosses that "remembers"? Isn't there necessarily some slightly better chance of a tails after 10 heads?

Note that I do not know symbolic logic so, embarrassingly, formal demonstrations are beyond my ken.

My way of seeing it is by counting the possible outcomes. Let say, you do 10 coin tosses. There are a lot of outcomes; Exactly 1024 of them (2 to the power of 10), of which only:

  • one is made of only heads
  • 10 are made of one tail and nine heads
  • 45 are made of two tail and eight heads

...

  • 120 of them contains two heads more than tails
  • 210 of them contains one heads more than tails
  • 252 are made of as many tails as heads
  • 210 of them contains one tail more than heads
  • 120 of them contains two tails more than heads

...

  • 10 are made of one head and nine tails
  • one is made of only tails

The general formula is obtained using binomial coefficients, but I skipped the formalism.

All in all, there is a higher probability to have approximately as many heads as tails because there are a lot of way to order an even mix of heads and tails while little way to order uneven mixes.

Note: this is related to the concept of entropy, as expected from randomness.

1
  • A common example of this approach to cumulative probability is rolling two (6-sided) dice - it's more likely to roll a total of 7 than 2 or 12. If you roll a 1 on the first die, then roll the second, it doesn't need to "remember" and avoid landing on a 1 in order to "preserve" the low probability of totalling 2.
    – IMSoP
    Dec 14, 2015 at 16:42
3

Not sure if this is the kind of answer you are looking for, but here is a non-mathematical, intuitive explanation.

The tossing of a coin, while random, is still composed of a chain of events that are themselves theoretically predictable to some degree - it's just that these events are very complex and the way they interact (and what they are) are not known.

For example, the outcome of a coin toss could depend on the following properties:

  • The shape and crafting of the coin
  • The material the coin is made from
  • The way in which one moves their hand/fingers to toss the coin
  • The physics of gravity, momentum, air resistance, and other environmental factors
  • The material of the surface the coin lands on

And so on. If you were to know the exact way these properties interact and know the initial conditions for each of these properties, you may have a better sense of how the coin might land (impossible in practice).

In terms of your question, each coin toss is an independent event that cannot dictate the next event. This is because each of the initial starting conditions will be slightly different. But the shape of the object will have a big impact on the possible outcomes. Heads or tails is determined by the precise interaction of all the variables in the process. Because of the structure of the coin, only two outcomes are possible, and neither is really more likely than the other based on the interaction of all the variables which is driven by the shape of the object. What pushes it one way or another (heads or tails) has to do with how physics causes all the parts of the system to interact together.

This means that the contribution of all the other factors when it comes to nudging the coin one way or another is not sufficient enough to make either heads or tails more likely than the other. When it all adds up over thousands of samples you see that both are pretty much equally likely to happen and this is because of the interaction of all the variables involved in this physical system.

3

Yet we also know that the series will converge upon an equilibrium of heads:tails

We don't, actually.

At every flip the probability of the next flip being heads or tails is still 50:50. Can we not flip an infinite number of heads? We say we cannot, because the probability is small, that is, it is the limit as x->infinity on 1/2^x. Mathematically, we can say this limit converges to 0 (if we are in normal math land).

But let us now imagine a dart board that is the unit circle. We through a dart into the board, and it hits the board at a single, random, point. There are an infinite number of points, so the probability of hitting any individual point is 0. Yet we must hit the board somewhere! So, wherever we hit the board, at that point a probability 0 event happened. This would seem to show that not only does probability have no causal power, but even infinitely unlikely events can be forced to occur in finite time, with infinite possibilities.

So, if you flipped a coin an infinite number of times, it is true that we should expect an exact 1:1 equilibrium between heads and tails (for a fair coin), but we would also expect infinite runs of both heads and tails within the larger infinite set, and if you chose afterwards to just look at these infinite sets our expectation for all infinite sets would be violated. So we expect an equilibrium of heads and tails at infinity, but we also expect to be wrong an infinite number of times corresponding to an infinitely small portion of the infinite set of infinite sets.

4
  • This answer appears wrong to me. It is in fact very well-known that the ratio of heads/tails converges to 1, with probability 1. This is a direct result of the law of large numbers. Saying "we don't know" is false, and mentions of the unit circle or subsets of infinite sets are red-herrings. Dec 15, 2015 at 6:32
  • @BlueRaja-DannyPflughoeft The OP means converge in something other than the standard mathematical sense, as they make clear in their question and the resulting comments. I agree that an infinite series of heads/tails would violate the axiomatic Law of Large Numbers, but my point is exactly that the Law of Large Numbers occassionally fails to hold for a random sample of infinitely large sample spaces lim(x->0) of x of the time. What that means depends very much on your philosophy of mathematics, but I find the unsupported accusation that my answer is gibberish offensive. Dec 15, 2015 at 6:53
  • @admins, I don't appreciate my comments being edited. This is does not "appear wrong to me", it is unequivocally incorrect. Dec 17, 2015 at 16:58
  • @BlueRaja-DannyPflughoeft SE wise if you wanna get that fixed you should: a) use a custom flag with a message like the one you put in the comment, b) post a meta on the issue to discuss it, or c) hop over to chat and be like "yo, adminname, can you help me with a thing?". I got alerted to your reply but I don't think anyone else did. Sorry about that. Dec 18, 2015 at 6:49
3

The sequential tossing of a coin creates the impression of "history building." However, if we use the equivalent method, we will clearly see that no history (memory) is built. Let's take the case of tossing one coin 1000 times, the equivalent method would be to toss 1000 coins, one time. With this method, it is clear that no history is built, and if we examine the coins, we should find about 500 heads (or tails)!

3

There are already quite a lot of good math-based answers here, but this is the SE for philosophy, so I'd like to offer a more philosophical one. I think the most interesting part of your question is:

Isn't there something in the physical series of tosses that "remembers"?

because the answer is a surprising "yes!" It just isn't the coin that's doing the remembering.

Suppose I flip a fair coin ten times and get the result 'TTHHHTHTTT'. Now suppose I flip the coin another ten times and get 'TTTTHHTHTH' instead. Nothing unusual so far.

But wait! Each of those two sequences is actually very unusual--in fact, the chances of either one are exactly the same as the chances of getting heads ten times in a row! An outcome like 'TTHHHTHTTT' only seems more "random" than ten heads in a row because your brain unconsciously throws out information about sequence. To our brains, the two outcomes 'TTHHHTHTTT' and 'TTTTHHTHTH' both just look like "messy jumbles of 'T's and 'H's," even though objectively speaking they are completely different.

So the reason you have an equal chance of flipping heads or tails even after flipping nine heads in a row, is simply that the two sequences 'HHHHHHHHT' and 'HHHHHHHHHH' are just as likely to occur as any other sequence of ten flips--that's the "fair coins have no memory" part. But what about the other part? Where does the law of large numbers come from, if all sequences of flips are equally likely?

I mentioned earlier that your brain unconsciously throws out information about sequence when looking at outcomes like 'TTHHHTHTTT' or 'TTTTHHTHTH', and this is why those two results look so similar. Well, the law of large numbers works because it does exactly the same thing! The law doesn't predict the exact sequence you'll get if you flip a coin a large number of times--rather, the law takes the total number of heads flipped, compares it to the total number of tails, then extrapolates that ratio for longer and longer sequences of flips. As far as the law of large numbers is concerned, the sequence 'TTHHHTHTTT' is exactly the same as the sequence 'TTTTHHTHTH'--or, for that matter, 'HHHHTTTTTT'--because they each have six 'T's and four 'H's, and that's all there is to it.

So in fact, the law of large numbers does imply "something that remembers"--otherwise there would be no way to keep track of the totals. The trick is that the "thing that remembers" is you! The law of large numbers relies on your memory for you to derive and make use of it. So in answer to the final part of your question, you might say that the "causality of probability" is just your expectations acting on past results: instead of saying the coin's fairness "causes" it to come up tails 50% of the time, you would say that your previous experience with fair coins causes you to expect the coin to come up heads or tails equally with each flip. (This is the general view taken by Bayseian probability, a fascinating branch of mathematics and one of many possible interpretations of probability.)

2
  • Thank you. This is very interesting, but seems to me to veer towards "solipsism," philosophically. There is a lot to process here, so I'm afraid I must give your very welcome "not entirely mathematical"answer a bit more thought. You are quite right that "re-membering" was a big part of what stirred my curiosity. Dec 24, 2015 at 2:55
  • @NelsonAlexander Nothing solipsistic about a phenomenon that depends on counting requiring a thing that can count! Look at it this way: the law of large numbers does not require your memory in order to make it true, just as three apples are still three apples even if nobody counts them. However, you do need a memory in order to observe the law in action. The gambler's fallacy arises when our observations (and the way our memory unconsciously edits them) interferes with our intuitions about chance.
    – Malcolm
    Dec 28, 2015 at 3:40
2

Suppose I decided to draw a picture, and a drew a line like this: 'I', and then another one exactly like this after it, and then another, and so...

This would be a tedious picture, but this is like a memoryless drawing - with each line being placed as though it was the first line placed.

This is by analogy just like the throw of a fair coin or die, each throw being memoryless.

The question is are there other ways of throwing that take into account history? Sure, not with a dice or coin, but certainly with a virtual dice in a virtual world, and an avatar throwing it.

And this would be like a man doing a drawing knowing the line he placed before and knowing the line he places after, and the line he is drawing right now.

He has before him an aim, and behind him a history; and right now the moment drawn.

1

Most of the times the fallacy -and your problem- become true only when the events reduce the odds of getting the same values in the future, say:

My opaque jar has 100 balls. 50 of them are white, and 50 black. What's the odds of getting a black one when grabbing just one?

This event remembers the history and, if you pick them all, or you pick just one, the odds were the same: 50/50.

But your problem is the contrast between the uncertainity and the already-known events. All the times you should look at the definition of the problem. If the past is not a constraint (as it was in my example) then forget the damn past and move on:

I flip a coin. What's the odds of getting tails when I flip?

It says nothing regarding the past, because flipping an ideal coin has nothing to do with any physical property (a non-ideal coin, i.e. a real world one, perhaps gets their edges less sharpened when they hit the ground and the future result may vary...). Edit: Indeed, this wikipedia article related to entropy contains a graph with the coin flip distribution, and people knowing this would never commit this fallacy again since it is allowed to have coin flips where an ideal coin would have 1 in each flip, although that would only be a limit case.

Most fallacy-holders think the problem like this:

The coin has a quality of a true balance on a specific interval of experiments. If I flip it X times, X/2 of those times will have the desired result.

They take (or observe) the initial problem like this (without math jargon; otherwise they would not incur this fallacy):

  • The prerrequisite is to chose either heads or tails.
  • The experiment consists on flipping a coin and observing the result.
  • It is common to get half times heads, and half times tails.

And convert them to this:

  • The prerrequisite is to chose either heads or tails.
  • The experiment consists on flipping a coin and observing the result.
  • It is guaranteed to get half times heads, and half times tails.

(Most of the times they will know nothing about variance and SD, so there's no need to detail those concepts anymore).

Although the difference is subtle in language, is not subtle about what do you know about your system. You are changing the propositions and adding another constraint (yes: reducing the entropy).

So: Get back to the roots of your problem. Is your system evolving through experiment iterations? If so, you gain knowledge of the system, and you get closer to the overall initial information you know. When you reach that state, your entropy becomes 0 (exactly 0 shannons here): you know what the last ball is.

However, if your system does not evolve with iterations, the overall initial propositions still apply: Same experiment, same odds you already know (1 shannon over and over and over and over and damn over until our deaths and beyond or until the coin stops somehow being ideal).

0

Worth a mention is regression toward the mean, which is a real thing, although completely a-causal. It doesn't meant that if you've had a very unlikely outcome (8 heads out of 10 flips) that the probability of the next flip is biased against heads, but it does mean that it is >>50% probable that the average of any future sample-of-ten will be closer to 50% heads than your up-to-now outcome of 80%.

https://en.wikipedia.org/wiki/Regression_toward_the_mean

2
  • I think your answer has the same confusion as the original question. Note that the wiki links states "The conditions under which regression toward the mean occurs depend on the way the term is mathematically defined". it is very easy to accidentally change the conditions being discussed, for example changing from absolute difference between count and expectation to the ratio, or splitting the running sum into independent runs. In these cases I find it's best not to look into the maths, but to look back at the humans, and see where (and why) the misunderstanding happen. Dec 21, 2015 at 19:29
  • Fact remains that RTM is a factual phenomenon (under suitable definition) where Gambler's Fallacy never is.
    – Jeff Y
    Dec 21, 2015 at 19:36
-2

Anyone who sees 80 HEADS in a row and doesn't expect to see HEADS on the next toss is an idiot, and I would like to gamble with you.

The basic issue is that we don't actually know if the coin is fair or not, we can only make an assumption by assigning a prior probability of 50/50. We must then update our belief about fairness after each toss. The weight that you give to your initial assumption determines how little or how much you should change expectations given prior results.

Interestingly, in the specific case of a coin toss, we need not make any prior assumption about fairness to guess optimally.If the coin is fair, then it never matters if we guess heads or tails.

Therefore, optimal strategy dictates that we should always guess whichever result has been observed most frequently, even after a single toss. If the coin is fair, we lose nothing by doing so. But, if there is even minuscule advantage for one result over another, we are most likely to be right if we guess the most frequently occurring result.

4
  • 3
    This is a very good observation, but I don't actually think it's an answer to the question. Aug 14, 2020 at 4:40
  • Maybe more of a tangent about gamblers fallacy than a direct answer. Hobbs has the actual answer. Aug 14, 2020 at 5:05
  • The fair coin is an unchangeable assumption of the question.
    – tkruse
    Aug 16, 2020 at 1:22
  • Assume all you want. Doesn't change anything. Aug 19, 2020 at 8:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .