3

There seems to be a paradox in my head when it comes to evaluating independent probabilities and it's sort of boggling my head. I am curious as to how to solve it.

Suppose I tell my friend to think of a number between 1 to 100,000 and I try to guess it every day. Suppose I fail for 100,000 straight days. Today, I guess it correctly. I claim that I predicted this using mind powers. A skeptic says, "You guessed 100,000 times! You were bound to get it right! The probability of you getting a correct guess out of 100,000+ guesses is pretty high". Now, that same day, Jane asks her friend to think of a number between 1 and 100,000 and tries guessing it. She guesses it correctly. People are astonished. After all, she guessed it correctly. Even the skeptic starts believing that she has mental powers. She got 1 out of 1 right and the probability of her getting it right was 1 in 100,000!

Yet today, the probability of me guessing the correct number and her guessing the correct number were the exact same. Why should we care about my previous trials? After some thinking in my head, it seems that I, and possibly others, incorporate my previous trials since they are similar. But why does similarity matter as long as the guesses are happening on independent days. If say, instead of me guessing every day the last 100,000, a new and different person did. Then, we perhaps would be just as surprised that I guessed it today as we would with Jane. But what really is the evidential difference here?

One could respond to this and say that my trials are similar to Jane's. Hence even though Jane correctly guesses the first time, it's not unsurprising for atleast one out of both Jane and my trials to be guessed correctly. My counter to this would be to imagine the following scenario: you guess once and guess it correctly. Most people would be surprised if you did. But unbeknownst to you, in the history of humankind, about 100,000 people had made those same guesses at different times and failed. Different humans in different times in different areas but the same kind of guess. In this case, it seems ludicrous to include these trials, but why or why not? Should this change your level of surprise?

EDIT: For the purposes of this example, let's assume true independence here. Let's assume the guesses are being done based on a random machine spitting out numbers for example to reduce effects of patternized guesses.

4
  • 1
    I do not follow what the paradox is. The probability is 1/100,000 in both cases, whether the guess is correct or not is irrelevant, whether people are surprised or not is also irrelevant to that. The "surprise" happens not under the contrived stated assumptions, but in superficially similar situations in the real world, which are quite different. One is never assured of fairness and independence by a voice from above, so observing ten tosses of heads in a row, or guessing a # out of 100,000, makes it far more plausible that the game is rigged. Surprise!
    – Conifold
    Dec 22, 2022 at 13:15
  • Why people are surprised is a psychology question. Your questions about probabilities should be submitted to Math Exchange. I see no philosophy in your question
    – user59124
    Dec 22, 2022 at 14:35
  • I don't agree with @SteveSaban. Although this question is worded in terms of surprise, it strikes me as an attempt to get at an important issue in the philosophy of probability, namely how one defines a trial, and why number of trials is significant. Dec 22, 2022 at 18:07
  • I'll drop this here. It's unsophisticated philosophy that doesn't recognize naturalization of epistemology. Anyone who doesn't at least partially naturalize their philosophy isn't living in the current century. or is still struggling with the appropriate use of the law of the excluded middle. I'd ignore them. plato.stanford.edu/entries/epistemology-evolutionary
    – J D
    Dec 30, 2022 at 20:34

7 Answers 7

2

It's probably more of a question of psychology, expectation and human biases.

Like humans are really good at pattern matching and not nearly as good when it comes to dealing with randomness. Just look into a cloud or against a rough wall and you'll see faces, structures, animals and whatnot that aren't really there but you brain makes up patterns as it tries to sort the chaos.

Now the only known way to make sense of randomness is through statistics as randomness itself might follow some meta patterns. So that for example the individual sample is random but and aggregated average might follow a pattern.

However even the assertion that a coin flip is 50:50 or that a dice roll is 1 in 6 is technically a lie. You can roll the dice and get your predicted number immediately and you can roll it 20 times in a row and never get it. Even if you'd use the same number and the same dice and whatnot.

So that probability ratio is actually just half of the truth, the other half that is often forgotten is "(in the limit of infinitely many tries) the ratio is x". And infinity or even an approximation of that, with very large numbers, is something that is hard to grasp intuitively for a human. Like in the majority of cases we don't get to collect enough data for a reliable statistic, but instead we extrapolate from our biased sample data.

Like in terms of intuition it's probably more like having categories in your brain which you assign the probabilities to, like idk "never, seldom, often, always".

And if it happens 1/1 it's always, if it fails after that it's first treated as an outlier (the failure), then probably moves to often (if failures persist) and then moves to seldom and probably stays there for a long time. While if failed 100,000 times in a row it is at "never" and if it succeeds after that it either moves to "seldom" or is treated as an "error in the measurement" and is thus neglected. So the pattern/categorization that you form depends on the order of events while the statistics doesn't.

Whether it's the first or last try isn't important what matters is the ratio for large numbers. Which you rarely get to see.

The other thing is that we live in a physical reality and probability is a mathematical concept. So as I guess conifold has mentioned, we can't really be sure of the randomness of an event either. Like if something happens more often than predicted that could be an outlier or it could be a pattern (that game could be rigged). Like if a dice shows the 6: 1; 5; 10; 100; 1,000,000 times in a row would you bet on another number because this event was so unlikely that something else has to follow (which it doesn't as the events are independent) or would you bet the pattern continues because the dice is clearly rigged?

So in the physical world it often makes more sense to adapt to the anomaly of statistics and assume a pattern than to expect a uniform distribution of randomness that you cannot work with. However just because it works in some instances doesn't mean it's not going to bite you in others (gambling).

0

Unlike a roulette wheel, which doesn't 'remember' your previous attempts; you may well remember some or all of your previous attempts (especially if you guessed in some sort of sequence/pattern). Each of your guesses in such a scenario would have improved odds, as they are informed by previous, incorrect, eliminated guesses.

5
  • I don't feel like this is addressing the meat of the question. Sure I agree with this, but let's assume true independence here. Let's assume the guesses are being done based on a random machine spitting out numbers. The paradoxical intuition remains in place. If I hadn't guessed it 100,000 times before, almost anyone would be surprised if I guessed it correctly. Yet if I did, most wouldn't. Is this intuition accurate? It seems to be. But my trials are just as "valid" as Jane's trial. And if my trials are independent from Jane's, does this mean we should be surprised by Jane? Dec 22, 2022 at 12:19
  • 1
    Sure. Now you've edited the question, my response is of little use. By 'intuition', do you mean 'surprise'? It is more surprising if the number is guessed the first time, because the odds of guessing the number in 100,000 attempts is vastly greater than guessing 1/100,000 in a single attempt, even though, when measured on an individual level, the odds of each guess are the same. See The Gambler's Fallacy for a vaguely related issue. You could replicate the test with ten playing cards to measure the effects over a hundred or so attempts. Dec 22, 2022 at 12:30
  • Yes, I edited it because I wanted to make sure the crux of the question is stayed put. The question then is is this "surprise" coherent with reality? I.e. "should" we be surprised? Intuitively, it seems yes, we should be surprised more if someone guesses it correctly the first time vs. after 100,000 times. But if they're independent, why should we be? What if I did correctly guess it on my first try? Then people would be surprised. Not let's assume that in the history of humankind, about 100,000 people unbeknownst to me also had guessed and failed. Does this change things? Dec 22, 2022 at 12:38
  • I'm not knocking the edit. It was a good idea. The trouble seems to be you're comparing the analysis of one guess in isolation with one guess in the context of 100,000 guesses. With each guess, the odds are 1 in 100,000 if viewed in isolation, but with every new attempt, the odds decrease, just as the odds of rolling the same number again decrease. This is more for a mathematics stack and someone here or there will soon answer it far better than I can. Dec 22, 2022 at 12:47
  • This just begs the question of what to include in that context. How many games should we include? Should we only include guessing games? Dec 22, 2022 at 13:20
0

As far as I can tell, you guessing 100,000 times in a row is consituted of a series of dependent events. Let's use a smaller number (say 5). Your task is to correctly guess which number 1 through 5 is the one held by someone in his hand.

Probability of guessing the correct number with your first guess is 1/5 (20%). You say a number out loud, "3!". "No, incorrect" announces the judge.

Next guess, but now 3's eliminated i.e. your probability of guessing the right number with your second try is 1/4 (25%). You shout "2!" The judge looks at you, says "nope!".

On your third try (the numbers 3 and 2 are no longer in your list of possible correct answers), the probability will be 1/3 (33%). Your odds are improving. Suppose you're an unlucky guy and this time too your guess (say it was 1) is wrong. That eliminates 1, 2, 3 from the list.

Fourth try now (remember 1, 2, 3 are out). Your chances of guessing correctly are now 1/2 (a whopping 50%). Suppose again you fail to guess the right number (you'd said 5).

Fifth attempt. This is a no brainer. There's only one number left (4), so obviously 4 is the answer. Probability of you "guessing" the right answer this last time is 1/1 (100%).

With each attempt your odds go up i.e. the scenario in your question consists of dependent events or, to answer your question explicitly, your previous trials (guesses) matter.

See how your chances went from 20% to 25% to 33% to 50% to 100%?

0

Granted your guessing is random, with no memory there is still a big flaw.

You are comparing one guess with a group of guesses. At one side, there is one guess only; at other side, there are thousands of guesses taken together. So, you are comparing a soldier with an army.

Guessing a number when the solution-space has 100,000 numbers in one guess is obviously 1/100,000. It's not, however, 1/100,000 when you take 10,000 guesses in accumulation. Even when each guess is random the accumulation has probability of correct guess equal to 1/10.

Anybody who met you on the lucky day only and doesn't know that you have been guessing the number all your life will be equally as surprised as when he met the girl that does luckily guess the number at first try.

0

Sometimes, people consider epistemology to be prescriptive rather than descriptive. (Worrisomely frequently, many fail to distinguish between the two at all.) I have never found my way through the Münchhausen trilemma, so I cannot speak about prescriptive epistemology. Some, however, include descriptions under the mantle of epistemology, and feel more comfortable speaking to that.

Jane asks her friend to think of a number between 1 and 100,000 and tries guessing it. She guesses it correctly. People are astonished.

If you ever witness this happening, I encourage you to note what happens next. My expectation is that this event is immediately followed by the demand that Jane do it again. The probability of this happening two or more times consecutively is vanishingly small, and that might explain our human reaction of disbelief, which is often appropriate: sometimes it is chance or coincidence. Repetition-- replication-- is how we understand how many trials something required. When Jane gets the answer right, we do not know many trials led up to that. When we demand that she repeat her performance, we are immediately starting a new trial.

However, there are very important, real-life examples of this principle-- not a paradox, but perhaps a cognitive deficit- that we can examine to talk further about it.

Let's say you design trials for a pharmaceutical company and you're told to get research showing that some drug works, when that drug is actually ineffective. Is there any way you can do that? Sure. You could run twenty different trials, perhaps distinguished in slight ways like outcome measurement, and then abandon any trials that are unpromising, or simply fail to publish any trials with a negative result. Even if you did try to publish all trials, journals are not interested in negative findings for a new drug, so those wouldn't get published anyways-- no blood on your hands. You're left, probably, with a single published trial, demonstrating only that people get better on Placebo A than they do on Placebo B about half the time.

Does this happen in reality? We can't look into anyone's mind to see if anyone is intentionally trying to push drugs that they believe are ineffective. However, yes, pharmaceutical companies run many more trials than are ever published. In combination with publishing bias, journal editors' biases toward positive findings, this leaves us with a misleading picture of research. This fact is perhaps part of what's responsible for the replication crisis in psychology and the claim that, probably, more than 50% of published medical research is incorrect. I don't expect that these issues are limited to either field, although I do expect them to be worse in some fields than in others.

Note that we don't even have to run multiple trials to do this; we can choose, after we have gathered data, how to interpret that data. We can easily imagine twenty different outcomes to measure for a drug-- perhaps all-cause mortality, perhaps days called in sick, perhaps hospital time. These are referred to as "researcher degrees of freedom." The more degrees of freedom the researcher has, essentially, the more tries they have at a positive result; those degrees of freedom are not restricted to any particular domain. They might even decide, after the fact, that some of the people enrolled in the trial weren't really eligible for the trial; maybe those eligibility rules were decided after the researchers already knew who got better and who got worse. Doing this intentionally is often referred to as "p-hacking," but there's no reason to assume this only ever happens intentionally-- my introduction to this problem was a doctor who, after categorizing the diets of pregnant women into a few hundred categories, proudly reported that breakfast cereal increases the likelihood of a male child. I don't believe that doctor was intentionally trying to mislead anyone.

Some of the most fun, headliner findings of science over the last few decades have been related to these degrees of freedom, from positive ESP findings to the brain activity of dead salmon. We might say in these cases that Jane tried to guess the number of 100,000 people, but none of those people were aware of the other guesses, and that Jane managed to stun one of them.

One of researchers' first acts to address this problem was correction for multiple comparisons. We can calculate the probability of getting one hypothesis correct by chance; we can also calculate the probability of getting one of any number of hypotheses correct by chance. So we can run a trial and consider multiple outcomes or multiple subsamples and still accurately report how likely the results would be if our explanation was false. However, this is of limited use with regards to other degrees of freedom, which are potentially uncountable, and especially, with regards to intentionally misleading anyone.

Perhaps the better solution is to demand that researchers rigorously document and share their plan prior to collecting any data. In the USA, this is now legally required by the FDA for many pharmaceutical trials (but keep in mind, there is a big difference between making a law and enforcing a law; I don't know the state of current compliance, but it has been spotty in the past.)

If we were to consider Jane's case, were she required to register her trial, she might document that she would make 100,000 attempts to guess a number, and then count how many times she got it right. Could she instead say that she would guess until she got it right? She could, but she would be criticized for doing so. For one thing, such a trial could potentially never end, and so it would call her methodology into question: if she never guessed correctly, she would never finish her trial, biasing the trial toward a positive result. So high quality research is never designed this way, and even when there are reasons for ending a trial early-- ethical reasons, perhaps, in the case of a drug that seems to save lives-- the decision to do so is still a reason to suspect the measured outcomes.

But note that researchers do not all necessarily agree that this is a problem, or that even if it is, that it is possible or worthwhile to do anything about it. And Society or Science aren't entities that can believe in anything; it is people and scientists who believe things, and they don't have to believe in those things for the same reasons. There remains significant resistance to trial registration and correction for multiple comparisons. And there are fields, say paleontology, where the concept of registration before data collection doesn't make a lot of sense. If there is some flowchart we can follow to true statements, then I don't know where it's kept, but perhaps we can make do with getting a lot of smart, educated people in one place and listening to them argue.

0

You seem to be assuming that your chances of guessing correctly increases as your number of attempts increases. That is simply not true. Assuming your friend thinks of a fresh number every time you guess, the odds are always 1:100,000 regardless of how many times you have previously failed to guess other numbers imagined by your friend.

It is rather counter intuitive. Suppose you toss a coin and get heads fifteen times in succession. The chances of that are very low- about one in 33,000. You might be strongly tempted to guess that it is time for tails to come up, but the odds of the next toss being heads is still 0.5, as it has been for the previous 15 tosses.

You also seem to misunderstand how probabilities should be construed. You can calculate the probability of your friend guessing the number correctly at a single attempt. You can calculate the probability of you guessing the correct number at least once in 100,000 attempts. You can calculate the probability of either you or your friend getting the number correct in 100,001 attempts. You can calculate the probability of either you or at least one of n other people guessing the right number at least once when you have had x attempts and they have had y attempts each on average. You can even, if you wish, calculate the probability of you guessing the correct number in x attempts and someone else throwing ten heads in a row (ie combining a totally different set of possible outcomes). The resulting probabilities will be different in all of the five scenarios I have mentioned. But the chance of any one person guessing the correct number on any one attempt is always 1 in 100,000.

-2

We decide the probability of an event on the basis of the nature of the event: a die has six faces designed to be physically very similar to each so we think all faces have the same probability of turning up and so we say that there is a 1/6 chance of getting any particular one. Past results are irrelevant to the next throw of the die. This means that there is nothing to marvel at from getting a 6 rather than a 2. Thus, the probability of correctly guessing the result in advance of the throw is 1/6.

This also means that the probability for one event, the event of guessing correctly the result of the next throw of a die is 1/6.

For multiple events, the probability is totally different. If you try to guess each time you throw the die and you throw the die six times, the probability of guessing right one of the six throws is 6 times 1/6, that is, 1.

No paradox.

4
  • The final sentence in this answer is incorrect. Dec 23, 2022 at 22:45
  • @MarcoOcram Please explain. Dec 24, 2022 at 16:33
  • In the six-throws case, the probability of guessing right at least once is not 1. (Would the probability of guessing right at least once if you flipped 7 coins somehow be greater than 1?) Instead, it's 1-(5/6)^6 - that is, the probability that it isn't the case that you guessed wrong six times in a row - which works out to about 2/3. Generally, "or" is more complicated than "and" in probability. Dec 25, 2022 at 3:46
  • @NoahSchweber Interesting... Thanks for explaining. I'll think about it. Dec 25, 2022 at 10:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .