14

Isn’t every event by definition improbable in the sense that each event precedes an infinite series of causes that could have (theoretically atleast) been different?

We think of someone winning five lotteries as extremely improbable and a rare event. But the probability of five different, specific people winning five lotteries is the same.

The probability of getting 100 straight heads is the same as any other sequence, yet the other sequences aren’t seen as improbable.

When it comes to other types of mundane events, such as a person walking across the street, isn’t that also an “absurdly improbable” event? Once you factor in the details and specifics of that event, such as the way in which the person is walking, what clothes he’s wearing, what time it is, it also seems very improbable event.

So when, exactly, is an event probable?

10
  • 2
    Reification Fallacy: you are reifying nature according to your subjective interpretation: everything is "absurdly improbable" if the number of improbable facts in nature is larger than the number of probable facts, but such assumption is extremely arbitrary.
    – RodolfoAP
    Jul 31, 2023 at 18:02
  • 4
    There are 80658175170943878571660636856403766975289505440883277824000000000000 different ways to shuffle a deck of cards. Therefore, if we shuffle a deck of cards properly, the specific ordering that results is absurdly improbable. In fact, every shuffled deck of cards is so improbable, we should never expect to see any given shuffling of cards.
    – JimmyJames
    Jul 31, 2023 at 20:33
  • 1
    Although it's improbable, it's no less improbable than any other outcome! P.S. You might enjoy the story "Divided by Infinity". Aug 1, 2023 at 3:21
  • 2
    You may enjoy this quote from Feynman “You know, the most amazing thing happened to me tonight... I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!” Aug 1, 2023 at 18:37
  • 2
    @sehe also some person winning twice is much MUCH more likely than 2 specific people winning because the one person could be anyone--so the first win is a freebee and the second just has to match the first, whereas for 2 specific people to win the first win AND the second win both have to match. This would make a good misleading test question if phrased correctly.
    – Bill K
    Aug 1, 2023 at 22:26

15 Answers 15

19

The probability of getting 100 straight heads is the same as any other sequence, yet the other sequences aren’t seen as improbable.

Yes they are, if you put a piece of paper in a sealed envelope predicting that sequence ahead of time. And conversely, the more coin tosses you make, the higher the probability that any given sequence will happen at random. That's why the total number of tests is as vitally important as the number of tests which meet your hypothesis.

If anyone thinks one is more improbable than the other, with a fair coin, then by definition they don't know anything about statistics or probability and therefore should be ignored. Sorry if this sounds harsh, but a "popular" opinion doesn't count for anything compared to actually knowing the subject.

14
  • 2
    The important part of the results with the 100 coin tosses, is that a "random" string of heads and tails doesn't look interesting to us - basically like any other "random" string. They are completely different events, but do not feel special. So we inherently lump them together as such. 100 times heads obviously picques our interest.
    – Chieron
    Jul 30, 2023 at 17:04
  • 1
    "I try not to make predictions, especially about the future." -- Yogi Berra - If only we were all so prudent and wise!
    – Scott Rowe
    Jul 30, 2023 at 21:56
  • 3
    @Chieron And that's why "feels special" is a poor guide to probability. :) The OP is a catalogue of errors, but this is the easiest to explain, and will tend to point them in the direction of where they're going wrong in their thinking more generally.
    – Graham
    Jul 30, 2023 at 23:09
  • 2
    There's a subtle distinction here that I think trips people up here. What you say is exactly correct if we are talking about a specific sequence. But if the question was is, out of 100 tries, is getting 50 heads more likely than 100 heads (i.e.: ignoring order) then 50 heads is far more likely than 100 (although not the mostly likely result, contrary to popular opinion.)
    – JimmyJames
    Jul 31, 2023 at 20:40
  • 1
    Sorry, I mean to say that the result of getting 50 heads out 100 tries is not particularly likely.
    – JimmyJames
    Jul 31, 2023 at 20:51
16

Isn’t every event by definition improbable in the sense that each event precedes an infinite series of causes that could have (theoretically atleast) been different?

Specific events don't have probabilities.

Probabilities express a relationship between measurements of initial conditions and expectations of future measurements. This relationship is based off of past measurements of one or more countable frequencies in one or more corresponding finite samples.

If you ran the experiment many times with the same measurements of initial conditions (and only the same measurements, not a metaphysically identical experiment in which all the things you don't know are also the same), what fraction of the outcomes will fall into the category being evaluated? That fraction is the probability.

We think of someone winning five lotteries as extremely improbable and a rare event. But the probability of five different, specific people winning five lotteries is the same.

Yes, if you remember to mentally translate the ambiguous tense of the present-continuous winning to a future tense. Common use of probability in the past tense to describe past events is misleading; the correct tense is the past progressive would have been.

When it comes to other types of mundane events, such as a person walking across the street, isn’t that also an “absurdly improbable” event? Once you factor in the details and specifics of that event, such as the way in which the person is walking, what clothes he’s wearing, what time it is, it also seems very improbable event.

So when, exactly, is an event probable?

When the "event" is defined as the category of outcomes of a hypothetical future experiment whose initial conditions have the values you (or your sources) have already measured, the event has a probability.

(Note all measurements have limited precision, so all predictable outcomes of measurements are categories of outcomes. If I predict a measurement of length on a meter stick with millimeter precision, I'm predicting a category of outcomes that would be measured within half a mm of the predicted value on a meter stick with greater precision.)

The probability of the category of outcomes is the ratio of how many hypothetical future experiments have outcomes that fall within the designated category, divided by the hypothetical large number of experiments whose initial conditions have the values you (or your sources) have previously measured. Making the designated category smaller or bigger makes the probability smaller or bigger respectively, which is what you seem to be getting at: for any experiment with a given set of initial condition measurements, the probability of a category of outcomes approaches zero as the size of the category becomes arbitrarily small.

If the "event" is not so defined, probability isn't one of the characteristics it can have.

29
  • 1
    i intuitively agree with this answer, but you might want to cite something for your leading claim
    – user66760
    Jul 30, 2023 at 6:24
  • 3
    @doot_s I would suggest Experimentation by D. C. Baird, pp 40-46 in the 3rd ed, for an excellent discussion of how to move from measurement to statistical statements and vice versa.
    – g s
    Jul 30, 2023 at 16:41
  • 5
    Not downvoting, but this does appear loose to me. Probability is a measure of a state in a space, and it’s absolutely reasonable to talk about the probability of a discrete event that has already happened - c.f. the result of a roll of a dice on a particular game turn. In a sense, yes, we know with certainty that it happened, but that doesn’t mean the probability measure of the event as a function of the sides of the dice cannot be given, or is not a probability, because the behaviour of the measure still satisfies the Kolmogorov axions.
    – Paul Ross
    Jul 30, 2023 at 17:49
  • 2
    @PaulRoss suppose that Honest Bob has rolled a die already and looked at it. Bob tells us that he will write down the number of his dice roll. Bob reveals the note and we measure that it says "5". The probability that another future measurement of Bob's note will indicate that Bob wrote down a 5 is near unity. The probability that Bob would have written down a 5 given what Bob knew at t=0 is near unity. The probability that Bob would have written down a 5 given what we knew at t=0 is 1/6. [...]
    – g s
    Jul 30, 2023 at 18:03
  • 1
    @gs, I appreciate your point on probabilities under descriptions, but my contest is to the idea that you're specifically talking about expectations of future measurements, not that there is no such thing as the abstract or concrete event simpliciter. One can talk about the abstract probability of an event with no specific temporal sense as long as one is talking about a probability space. The topology of the probability space does not necessarily need to collapse down to the temporal idiom of events and points in time - for example, in the use of continuous probability density functions.
    – Paul Ross
    Jul 31, 2023 at 16:45
11

Yes, it's extremely improbable for practically anything to happen in the exact manner that it does. You are correct.

But then what do you do with that information? So it was very improbable for those specific five people to win the lottery, so what? Is that actionable information? Not really.

But if one specific person won the lottery five times, that is actionable. Because we would then suspect that particular person is cheating. The observable results are improbable given the null hypothesis that the person is not cheating, but become far more probable given the hypothesis that the person found some way to cheat the lottery. So, the probability of that hypothesis increases, according to Bayes' rule P(H|O) = P(O|H) P(H)/P(O).

Let's assume we're dealing with 1 chance in a million lotteries here. P(O) is very low because it's so unlikely to win the lottery five times. Crucially, P(H) (probability they're cheating) is low, because most winners of a single lottery aren't cheaters, but with five wins it's not nearly as low as P(O). And P(O|H) is 1. So P(H|O) is quite high, close to 1.

For five different random people winning the lottery, though it is equally unlikely, there's no interesting hypothesis for us to update. The chance of them all cheating is not increased in any remarkable way. Unless, perhaps, the five people share an interesting property, such as being friends and family members of a lottery official.

In the five-winners case, P(O) is about the same as in the single-person five-time winner case. P(H), however, is much lower here, because the chance all five of them cheated is the probability each one cheated (very unlikely, same as in the previous example) to the power of five. P(O|H) is still 1, but in this case that wouldn't be enough; P(H|O) is still much less than 1%.

Anyway, that's the difference. Practically any event can be viewed as unlikely, but few events are "actionable" in the sense that they would cause a Bayesian update to favor an interesting hypothesis.

5
  • well, they're not exactly wrong. but then you do have to ask what's relevant.
    – user66760
    Jul 30, 2023 at 0:35
  • The lesson is, if you're going to cheat, try not to do it in a noticable way. I heard about a programmer or accountant at a bank who adjusted the programming such that the partial cents which came out of some transactions all went in to one bank account, that he then took for himself. Eventually the discrepancy was noticed, but not for a long time.
    – Scott Rowe
    Jul 30, 2023 at 13:49
  • 1
    @ScottRowe I heard the same story, but they were caught quickly because the account grew too fast and attracted attention. It was also a plot device in the movie "Superman III" with a similar outcome. Jul 31, 2023 at 0:00
  • 2
    In Rosencrantz and Guildenstern Are Dead,the two characters are betting on coin flips. Rosencrantz, who bets heads each time, wins 92 flips in a row.. Since it happens in movies more often than in real life, if this happens to you, one possible hypothesis is that you are a character in a movie... Jul 31, 2023 at 2:37
  • Now for a real one, some time in the late 70s a programmer in Germany changed the payroll software in his company so that he got paid more. And this being the 70s, the laws against fraud required that you lied to a real person or made a real person believe something that isn't true. Police looked really hard until they found a secretary who signed all the paychecks on behalf of the company, and she was made to believe that his paycheck had the right numbers on it when it was much too high. Without her, there would have been no fraud case.
    – gnasher729
    Jul 31, 2023 at 15:58
8

This kind of problem arises when there is an attempt to apply probability to the real world. The theory of probability is about precise sample spaces and probabilities and events. The real world, on the other hand, isn't like that.

In order to try to apply probability to the real world, it's necessary to describe precisely the state of knowledge about an event before and after it happens. For example, if a fair coin is flipped, most people would agree that it has an equal chance of landing heads or tails. But if we knew precisely which side was up to begin with, and exactly how much force is exerted on exactly which part of the coin, as well as the structure that it lands on, then physics could say with very high probability exactly which side it will land on.

So: Given full knowledge of the universe at time t, the future is virtually determined and not probabilistic at all. Given partial knowledge, that can be quantified well, we can estimate the probability of future events given the present as we understand it.

So there is no god-given probability associated with any event in the real world; its probability depends on how much is known about it before it happens.

It does not make sense to claim that a real-world event has a given probability without referring to some state of prior knowledge.

4
  • This doesn't really address the problem. Yes, probability varies depending on your state of knowledge, but it is also true that from any particular human's state of knowledge, the specifics of virtually any event are extremely unlikely to have turned out exactly that way.
    – causative
    Jul 30, 2023 at 4:35
  • You speak of a deterministic world. But the quantum world is truly probabalistic, & quantum particles can be truly indistinguishable in ways that fundamentally shape their properties.
    – CriglCragl
    Jul 30, 2023 at 10:35
  • 1
    @CriglCragl paradoxically, the non-predictability combines with the huge numbers of the particles to make the outcomes predictable again, in general. So, even though all the people who shop in a grocery store each week make independent decisions of what to buy, the store owner can order an amount of foods in advance that is generally appropriate. I think this is the direction an answer to the question should take, but I'm not feeling up to it at the moment. In short, if there are huge numbers of people, most of them will be pretty ordinary, thus, predictable in some sense.
    – Scott Rowe
    Jul 30, 2023 at 13:44
  • 2
    @ScottRowe: In this answer I use the example of traffic being modelled as a gas, & roads as pipes: 'What's the "opposite" of emergence?' philosophy.stackexchange.com/questions/81417/…
    – CriglCragl
    Jul 30, 2023 at 19:56
7

The questions you need to ask are "improbable given what?", and "improable relative to what?"

  • Suppose I flip a coin. The probability that it will be heads, is 0.5.
  • Suppose I flip 10 coins. The probability that they will all be the sequence HTTTHTTHH is 2^-9.
  • Finally, suppose that I flipped 9 coins, and they came up HTTTHTTH. The probability that my next coin flip, ignoring what came before, will be heads is P(x_9 = H) = 0.5. Further, the probability that my total sequence will be HTTTHTHTHH, given that I already flipped HTTTHTTH, is P(HTTTHTTHH | HTTTHTTH) = 0.5.

When asking the probability of a specific event happening, where that event depends on billions and billions of other things happening first, don't disregard the fact that those billions and billions of things already happened1. You don't want P(universe reaches the point where this particular 1997 quarter is minted and ends up in my pocket AND I flip it AND it comes up heads)2. You want P(heads | all that). Yes, all prior distributions for things in the universe are extremely small, but once you condition your distributions on all the prior events we don't care about, the giant numbers mostly cancel out and we end up with useful probabilities.

We think of someone winning five lotteries as extremely improbable and a rare event. But the probability of five different, specific people winning five lotteries is the same.

"Specific" is the key word there. If the probability of winning is 10^-6, and you have 10^6 people with 5 draws each, the probability that each draw will have at least one winner is about 10%. If you want your intuition to match the math, suppose that you named 5 specific people in advance, and they all won. You'd consider that just as suspicious as one person winning 5 times in a row (and it is exactly as improbable if you specify which person wins which draw. A bit (~3125x) more likely if you don't specify which draw each person wins)).


1 And were observed, but more on that later.

2 Note that in this case, we don't need to have observed which specific sequence of events happened so as to lead to that quarter being in my pocket, just that some such sequence happened. Actually computing the probability of the end result would require knowing the probability of the various sequences summing over them, so it's really convenient that we won't need that number. But to head off related questions, it's probably worth noting that the result of that sum will be astronomically higher than the prior probability of any specific sequence in it.

5

Maybe you should read about logical operators like AND in the context of statistics: there is absolutely nothing paradoxical about what you claim.

when is an event probable

let's suppose when its chance is 50% or more.

Any two coin tosses will result in an event that is "improbable". Indeed, I am less likely to reply to a question than not. Yet no-one is very impressed when I do.

5
  • 1
    I was pretty sure you would weigh in on this one. :-)
    – Scott Rowe
    Jul 30, 2023 at 13:32
  • 1
    hahah i have no idea if that's a compliment @ScottRowe
    – user66760
    Jul 30, 2023 at 13:57
  • It might or might not be ;-)
    – Scott Rowe
    Jul 30, 2023 at 14:21
  • What does "I am less likely to reply to a question than not" mean? Than not what? Jul 30, 2023 at 17:05
  • 2
    reply to a question @DanielAsimov
    – user66760
    Jul 30, 2023 at 20:03
4

The field of algorithmic information theory gives an answer to this question: we expect "random" events to be complex. On the other hand, simple events are not likely to arise from random chance, so they are surprising.

Here "complex" is measured by compressibility: how concisely we can describe the event. For example, a string of 100 heads can be described very succinctly. However, a more typical result of 100 fair coin flips will not be compressible at all, and we will have list out the entire sequence of results manually. If the coin flips are actually random, we do not expect to get any simple-to-describe result such as "alternate heads and tails", "heads at each Fibonacci number and tails otherwise", etc. Any such result would be "surprising", or, we can say, "improbable".

3
  • "An optimally compressed signal is indistinguishable from random noise." (shiver)
    – Scott Rowe
    Jul 30, 2023 at 23:10
  • 1
    @ScottRowe Incidentally, this has been brought forward as a reason why no sign of intelligence has been found so far by listening to radio signals from space. The time window during which radio signals emitted by a civilization are not close-to-perfectly compressed and hence indistinguishable from noise is perhaps only a few decades, that is, infinitely short in cosmological terms. Jul 31, 2023 at 12:39
  • I completely agree with this answer, and perhaps one way to reconcile this with the OP's "The probability of getting 100 straight heads is the same as any other sequence, yet the other sequences aren’t seen as improbable." is this: 1) if I write a sequence down, then throw the coin 100 times, it is improbable that the 100 throws will follow my sequence; 2) If I throw the coin 100 times first, without writing my sequence down, then I can't claim that the result is improbable; 3) The set of low-information sequences can be considered as already written down (in our minds) by convention.
    – Stef
    Jul 31, 2023 at 19:00
3

I would dispute the notion that all events are equally improbable or absurd in probability. While in an absolute sense, any precisely specified event may be seen as improbable, that is not how our minds apprehend events in the real world.

Our intuitions about probability are contextual - we perceive events against a backdrop of patterns, regularities, and hierarchical abstractions. Walking across the street seems mundane because it aligns with the common abstractions of 'human behaviour' and 'everyday activities'. It is embedded within a known context.

Winning the lottery five times disrupts our established patterns massively. It deviates from what we expect, grabbing our attention. The greater the deviation, the more improbable and meaningful an event becomes within its context.

So I would argue that our minds are attuned to hierarchical thinking about probability based on conceptual patterns, not just specific details. We do not treat all events as equally improbable, because we inherently recognize that some occurrences align with intelligible contexts and regularities, while others are radical departures demanding explanation. It is these improbable disruptions that reveal the gaps in our knowledge

3

I don't think you can discuss probability in the real world without at least a passing reference to entropy.

Imagine a sealed airtight container filled with a gas. If you could take a snapshot of all the states of all the molecules, the likelihood that they would return to exactly that configuration again in the next five minutes, is the same as that they would all simultaneously end up in a particular corner of the box in the next five minutes.

However there are astronomically more ways that molecules can be evenly spread out in the box than that they can all be in one corner.

Entropy in one sense is a measure of how things can be distributed throughout space.

The human aspect is that we classify things. We can classify many different arrangements of molecules as being the same when in fact they are all completely different. E.g. We can have "the molecules are evenly spread out" as a category and "the molecules are all in one corner" as another. Naïvely these are equal categories whereas in truth they are vastly different.

The probability that a person crosses a street is contingent on a person existing, and a street existing for them to cross. Those are contingent on there being a planet that supports human life orbiting a sun that provides energy, and so on.

In terms of entropy, if you have enough time and enough matter that interacts and combines according to certain rules then most things that are possible within the rules will happen somewhere and somewhen.

Conclusion

Paradoxically, within a suitable dynamic system, any event, no matter how improbable, is nevertheless inevitable given sufficient time.

1
  • You can never step in the same street twicet. Wait, can you answer the same question twice? What are the odds?
    – Scott Rowe
    Jul 30, 2023 at 21:05
3

Of course the reality we inhabit is fantastically unlikely. I thought that's obvious after a casual look around. ;-)

1
  • 2
    I was going to answer: "Of course not, things happen all the time. " ;-)
    – user64314
    Aug 2, 2023 at 5:24
3

The probability of getting 100 straight heads is the same as any other sequence, yet the other sequences aren’t

Other sequences are seen as improbable. At least they are for anyone who has studied probability.

Something that you haven't taken into account is the number of coin flips.

If I make 100 tosses, the probability of getting 100 straight heads is approximately 0.0000000000000000000000000000008 - For practical purposes it is zero.

If I make an infinite number of tosses, the probability of getting 100 straight heads is 1, i.e. it is certain.

In order for your question to be answerable, you must define your sample space and specify the number of trials.

2
  • Did you mean 0 instead of 1? If you make an infinite number of tosses, then the probability of getting only heads is 0, not 1.
    – Stef
    Jul 31, 2023 at 18:55
  • No, I meant 100 straight heads. I've added that in. Aug 2, 2023 at 20:49
2

Given the premise "there is a universe", the probability of a mammal calling itself Joe Smith walking on two legs across the bit of space I know as "6th Avenue" at the moment I reckon as "12:33pm on July 31st" is ludicrously, incalculably small.

Conditioned on the premise "there is a universe, and it has matter as we know it, and planets, and cities, and streets, and human beings, and calendars, and Joe Smith — and Joe Smith works at 6th Ave and 33rd Street from Monday to Friday, and has a lunch break at 12:30, and likes Japanese-Mexican fusion", the probability of Joe Smith walking across 6th Avenue at 12:33pm on any given weekday, on the way to Takumi Taco, is not so small after all. The "unlikelihood" of the premise and the conclusion do a good job of canceling one another out.

Probabilities are only usefully considered in light of information we already have. Thinking about a probability conditioned on nothing doesn't get you much of anywhere.

0

For any event or definition of an event where you're adding on successive conditions, where those conditions are either mutually exclusive (at a given time in a given place, only one object can occupy that space, and someone cannot be on a flight to Melbourne and floating through the Horsehead Nebula at the same time) or the negation of that condition is true for at least one thing in your potential sample, the probability of all of those conditions being true will always go down, because you're multiplying probabilities that are less than 1 together or creating a set of conditions that cannot happen together.

When observing a real world event, the observer can describe that event using as many or as few of the actual preceding conditions as they like, generally removing all of the conditions for which we do not observe a causal link (we observe that gravity is 9.81m/s^2 on every day where at least one person wakes up, but most other things also happen when that's true, and we have no known laws implying one should follow the other).

A "probable" event then is, in general, one with as few conditions as possible, where each individual condition is itself "probable", and where those conditions are not mutually exclusive with each other. This is why abstracting away conditions makes something more likely, i.e. what are the odds that my left foot will be directly over the 52th sidewalk pad of North Avenue Chicago at 5:12pm on March 19th 2034, vs. what are the odds that anyone walks down any street at any time in the future. So there is at least one probable event in the universe: Somewhere, sometime, something will happen!

0

You throw a coin. Before it lands, you describe three results: It ends up "heads", it ends up "tails", or something entirely unforeseen happens (just in case to cover all possibilities). The first two have probabilities about 49.999999999%.

Now you throw the coin, and look for the event "coin falls on the table and comes to rest after between 1.3746 and 1.3747 seconds". That is much much less likely. There are also a few thousand similar events, all with very small probabilities. But the add up to a probability very very close to 1. You know have a very unlikely event.

Now you throw the coin ten times in a row, and look for the event "coin rests after 1.3746 to 1.3747 seconds on the first throw, 1.0853 to 1.0854 seconds on the second throw, ..., xxx on the tenth throw". This is incredibly unlikely to happen. But there are gazillions of similar events with a total probability of close to 1. (You might have a heart attack after the seventh throw, that's why the total probability is not 1).

So you can describe events with arbitrary small probabilities, but events that have no interesting meaning. You need to look at interesting events.

0

I suck at statistics. But perhaps my simplistic explanation will be useful if correct. Please comment if I need to fix this.

Suppose you flip a coin five times, and the coin has two faces, tails (0) and heads (1).

For example, I could flip the coin five times and end up with this sequence:

01101

So now let's ask two different questions about this experiment.

  1. How likely is it that we achieve the sequence 11111 versus all other sequences?
  2. How likely is it that we flip 5 heads?

This sounds like the same question, but I suggest to you that it is not. The underlying question to ask in both cases is how many ways there are to achieve that outcome versus the other competing outcomes?

For 1, it's simple. Every possible sequence of 5 flips is unique, so they are all equally probable. That is to say there is only one way to flip 01101 - one tail, then one head, then another head, then a tail, then a head. It's an identity.

For 2 it's different. How many ways are there to flip exactly one head? five: 10000 01000 00100 00010 00001

But how many ways are there to flip five heads? Exactly one: 11111

We can definitively say, having written out all applicable outcomes, that flipping one head is 5x more likely than flipping all five heads.

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Aug 1, 2023 at 7:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .