# How do you prove probability? [closed]

I have this thought that has bugged me for a couple of days and could not find any answers on the internet. I thought this would be the best place to consult.

How does one go about proving probability? My family members are frequent casino goers and I was thinking of a way to show them how they won't ever win. Until I came across this problem myself.

Lets take a normal dice for example. There is a 1/6 chance of getting a random number from 1-6. But how can anyone be sure of this? Meaning there is 1 in 6 chances that the dice you threw will land on the number "1", but it could also be 10 in 60, 100 in 600, 1000 in 6000 and so on. So you may actually get any number other than "1" for the first 50 throws, and then getting a "1" on the next 10 throws to give you a 1 in 6 probability.

But OK, lets say you get lucky and you do hit a "1" in 6 throws. But why do you determine that is the probability of the throws? If you throw it for another 6 times, you may hit "1" two more times, giving you 3 in 12 and if you stop right there, the probability then is 1/4 or 1 in 4. If you don't get any "1" then your probability becomes 1 in 12.

So who really determines/proves that the probability of a random number of a dice thrown is 1/6? Who decides how many times you have to throw it?

Just FYI all, I am no mathematical genius in probability and this is probably explained by some theory that I have never come across in my life. Would appreciate if you can point me in that direction if so. Thank you all.

Edit: Was pointed here by Mathematics StackExchange so...

• If your goal is "to show them how they won't ever win" it is futile, you can not show that because it is false. With randomness involved there is always a chance they will win. What you can show is that this chance is very small, so small that it is unreasonable to pursue it because the risk of losing is much higher. This is determined not by looking at 6 throws but at a very large number of such throws ("trials"). Then it can be shown that as this number goes to ∞ the relative share of 1-s approaches 1/6. Jun 2, 2017 at 21:03
• The problems you are discussing are known at least for a frequentist interpretation of probability. See philrsss.anu.edu.au/people-defaults/alanh/papers/comp_logic.pdf pp. 9-10 for a short discussion, the author indicates that the problems are difficult to solve for frequentism. Btw probability calculus only determines probabilities of certain and impossible events, you have to bring in additional assumptions to determine them for other kinds of events, i.e. you might have to presume a probability distribution and revise it when new evidence comes in I think some answers mentioned this Jun 4, 2017 at 16:07
• Btw it's not hard to prove that the probability of rolling "1" on an unbiased dice is 1/6. You only need the following premises: P1. Probabilities lie between [0,1]. P2. Probability 1 is assigned to certain (or "necessary") events. P3. The probabilities of mutually exclusive events add up. P4. The dice has six faces 1,2,... , P5. It lands only one face up on any roll. P6. Each face is equally probable. P1-P3 are versions of Kolmogorov's axioms. Of all of these premises only P3 seems worth doubting on intuitive grounds, and it has been doubted but arguments can be and have been given for it. Jun 4, 2017 at 16:19
• @Johannes I read the link and it was really good. I think i especially connected with 3.3 Frequency interpretations, just like you said. I read through several others but couldn't quite understand some but i think the frequentist interpretation was what i was looking for. Never knew there were so many interpretations of probability. Super grateful for the link! it was a great read. It blows my mind how much a single individual can only know, and even then, it is only in his own field of study.
– MH.Q
Jun 4, 2017 at 16:33
• plato.stanford.edu/entries/probability-interpret/#FreInt by the same author covers the same ground in more detail. Jun 4, 2017 at 16:40

This really belongs in the stats stack exchange (a.k.a. cross validated). But I'll give you an answer here.

Before getting into any math, I'll just say that there's a problem of epistemology here. You can't ever know anything about the real world with perfect confidence. The only thing you can ever know with perfect confidence is whether the axioms imply the conclusion. This is what a "proof" always consists of. That is, you can prove that "if `A` then `B`", where `A` is the set of axioms and `B` is the conclusion. You are not 100% confident in `A`, but you are 100% confident that `A -> B`. Premise `A`, itself, may have been the result of another proof, which means that you are 100% confident that `A` is true if the premises that led to `A` are true. If you follow the chain backwards, you eventually get to the original axioms, which were either a claim about the real world (which you cannot be 100% confident in) or a definition (usually in math). So any conclusion you make from a chain of proofs is either one in which you're only as confident in as the original set of axioms, or it's tautologically true. For example, if I posit `C` as an axiom of which I'm 70% true, and I prove that `C -> D` then, assuming this is a valid proof, we are 100% confident that if `C` then `D`. However, because we are only 70% confident in `C`, we are consequently 70% confident in `D`. However, instead of a true proof, I might have made a probabilistic claim. That is, `C -> D` with 90% confidence. In this case we are 63% confident in `D`, because 90% of 70% is 63%.

We can always form hypotheses and conduct experiments that are true tests of the hypothesis. This procedure can be used to increase our confidence in a claim, even push it to near-certainty. But it will never be certain.

The consequence of this fact is that we are not certain about anything, even our best scientific theories. We merely get to a point that a certain claim is so negligibly unlikely to be wrong given the set of data we currently have testing the claim, that our best bet is to accept it. If you google "most accurate theory", you will find resounding results that it is QED (Quantum Electrodynamics). But how can there be a "most accurate theory" if we can know that certain theories are true? Well, we cannot. However, QED makes certain statistical claims on what you would observe in a particle collider, were it true. From there, you can calculate the probability of observing the things it claim, were it not true (by random chance.) In QED's case, the probabilities of its claims across the board were generated by chance, by non-QED hypotheses, are so dramatically low that not believing in QED is as bad an epistemological bet as you can make. But it's not guaranteed to be wrong with 100% certainty.

You can apply this to anything involving probability. So, for example, we might come up with a probability that the sun will rise tomorrow, given the set of historical observations. A frequentist would predict 100%, and a Bayesian would predict something so close to 100% that the difference is negligible. But we cannot know for sure. Assuming we have not yet discovered celestial mechanics, we can make inferences on the probability of the sun rising given previous events.

Now for a more grounded example, to help cement this concept. Suppose we take a coin. We can assume it's a "fair coin" (i.e. 1/2 chance of landing on each side) but we don't know that for sure. (In fact, it's probably not exactly true for any coin because there are subtle imperfections that make it not a perfect thin cylinder.) However, its probability of landing on heads is a purely empirical claim. We can do something like a hypothesis test, which is similar to what I described in QED, but much simpler.

To give you an idea of how a hypothesis test works, consider this. Suppose we start with a coin, and ask "is this a fair coin?" (i.e. 50% chance to land on either side.) One thing we can do is flip it `n` times and ask "what is the probability that a fair coin would have been at least this deviant from the expected result?" The expected result of `n` flips of a fair coin is `n/2` heads, but we know that it can deviate from that. There's nothing abnormal about that. When I say "at least this deviant", what I mean is that the number of heads is at least as far from the expected result as we have observed. So suppose, for example, that we flip it 100 times, and we get 55 heads. To be "at least this deviant" simply means outside of the range of 4 heads away from the expected result of 50. That is, below 46 or above 54.

Suppose we flipped it four times, and it landed on heads only once. The probability of being "at least this deviant" is the sum of the probability of landing 0 heads, 1 heads, 3 heads or 4 heads. It turns out that the probability is 62.5%. That is, a fair coin would have a 62.5% chance of being at least one off the expected result of two heads. That's pretty high. So is this grounds to reject the "fair coin" hypothesis? No. But that doesn't mean we should blindly accept it. We have to keep going.

So now we flip it 10 times, and we get 2 heads. This deviates from the expected result by three. What is the probability of deviating from the expected result of 10 flips by at least 3? The answer is ~10.9%. That's not very low; things of that probability happen all the time in our daily lives. Should we reject the fair coin hypothesis on this basis? It would be a weird standard if we did.

Now let's say we flip it 100 times and get 30 heads. We have deviated 20 from the expected value. The chances of deviating at least that much is ~0.003%. That is, a fair coin has roughly a 3 in ten thousand chance of it's "head-count" in 100 rolls to deviate by at least 20 from the "fair coin" expected value of 50. But this coin did. This should constitute "strong evidence" that this is not a fair coin. Now, does that mean it definitely isn't? Well, I mean, the chance of a fair coin landing like that is not 0. So it's still possible that it's a fair coin. But given the result we just observed, it would be a fairly bad epistemological bet to conclude that it's fair. Some people say "we should reject the hypothesis" because the result was so low. I, personally, don't take such a binary view, but I would say that I'm so confident that it's not a fair coin that I will act as though I'm certain (even though I'm not). After all, that's how we treat epistemology in many aspects of every day life. Just keep in mind that, with more rolls, we can become even more confident in our claims. But never 100% confident.

Then the question is, what do we conclude about the coin's probability of landing on heads? Well here's the tricky thing about adjusting our views based on evidence. It's far easier to make a negative claim than a positive claim. That is, if we take a theory (e.g. "this coin is a fair coin with a 50% probability of landing on heads"), the best we can do with that is formulate an experiment to see if the claim of this theory holds up. If it does, we haven't exactly confirmed the theory (other theories could make the same claim) but we failed to reject it. However, if the claim appears to be unequivocally false, we can easily reject the theory. So the question of "after the coin experiment, what do we choose to believe is the true probability of landing on heads?" is far harder to answer than "is the probability of landing on heads an a priori given value?"

There are several ways of addressing this problem. Bayesian statistics is one such novel way. Instead of claiming "the probability of landing on heads is `x`", we instead have a probability distribution for the value in question. Then, as we obtain evidence from new experiments, we adjust that distribution. In the coin example, we might say "given no knowledge, we'll assume that the probability of landing on heads is equally likely to be anywhere between `0` and `1`". That's our probability distribution. Then we conduct an experiment by flipping the coin. Every time the coin is flipped, our distribution shifts slightly. After, say, 100 flips, if we have 30 heads, our distribution would be somewhere centered around `0.3`, but it's not fixed at that value. The more times we conduct this "experiment" the more confident we become in our assessment, and the narrower this distribution becomes. (Recall that flipping 2 out of 10 heads wasn't enough for us to reject the fair coin hypothesis, but flipping 30 heads out of 100 was more than enough.)

This goes to something called the "Law of Large Numbers", on which we're relying to pan out to the truest answer in the limit of many experiments. You can look it up, but loosely speaking, it claims that if an event is drawn from a probability distribution, as the number of times we generate this event approaches infinity, the distribution of the outcome of this event approaches the true probability distribution. So, if we perform the experiment a few times, there's no guarantee that its distribution will look anything like the true distribution. But we expect it to look more and more like that distribution as the number of experiments grow. In the coin example, we would often expect a fair coin to give something like 2 out of 10 heads, but we would almost never expect it to give 30 out of 100.

Going back to the casino discussion. If we take a casino game of pure chance, involving dice or coins, once we learn the parameters of that game we can easily prove that the game favors the house using simple math. However, as I mentioned in my first paragraph, that proof has axioms. So we're only as confident in the conclusion of that proof as we are in the axioms. Most of the time, those axioms are that the tools of the game are fair tools. (Fair dice, fair coins, etc.)

• DV - this answer is very confusing. Jun 2, 2017 at 21:03
• Wow that is a lot of effort given into this answer. Thank you so much for that! I definitely agree with the Bayesian statistics part as one of the better ways to find the "true probability distribution". Thanks!
– MH.Q
Jun 4, 2017 at 6:33

The key issue here is tying physical events into mathematical concepts. Mathematically, the probability of drawing a 1 from a uniform distribution between 1 and 6 is 1/6. Period. Why? Because that's the definition of a random variable with a uniform distribution between 1 and 6. The hard part is arguing why that random variable is a useful model of a die.

For that, we have an unfortunate definition in mathematics: If you threw the die an infinite number of times, you would see that it landed on a 1 exactly 1/6 of the time. The feasibility of throwing a die an infinite number of times is, of course, questionable at best.

There is a mathematical tool for this, which makes me surprised Mathematics.SE sent you here. Stats.SE would have been a good place too. The tool is called a p-value. It's a powerful tool which tells you how likely it was that a particular result occurred. So if you roll a die 18 times, and see 5 1's, it can tell you how "unusual" that event was.

Generally speaking, for Casino dice, the random variable is a very good model. The house always wins, but to do so they need to be confident that the dice they put on the table are sufficiently well modeled as random so that they can use these statistical laws to fleece your purse. They have a fiscal interest in making sure these dice are very random.

For computerized games, it's the same: the house wants it to be very random. The algorithms used in these games go through incredibly strict statistical testing to ensure they are highly random. In many states, the algorithms used are audited by the state.

In all cases, however, we don't prove a probability. We assume that a random variable is a good model of the physical game we are playing, and then we do testing (a. la. the scientific method) to develop confidence in that model. Or, we rely on greed, and make sure that someone else (i.e. The House) has a fiscal incentive to make the game random.

• "If you threw the die an infinite number of times, you would see that it landed on a 1 exactly 1/6 of the time". When randomness is a factor, "exactly" may not apply. Rather, it could be described as asymptotic; i.e. the more you throw the dice, the closer to 1/6 is predicted (given a fair system), but it wouldn't be exact and random. Jun 3, 2017 at 13:51
• @alampert. When infinities get thrown around, exact can be thrown around as well. I agree that a limit notation like you suggest would be more precise, but I felt the colloquial language around infinity was sufficient for this case. Jun 3, 2017 at 19:32

Your initial intuition is an example of a logical fallacy known, aptly, as the "Gambler's fallacy".

the incorrect belief that separate, independent events can affect the likelihood of another random event. If a fair coin lands on heads 10 times in a row, the belief that it is "due to the number of times it had previously landed on tails" is incorrect.

In actuality (in a fair game), no previous result has any influence on following results. That is why the statistical odds are consistent, regardless of previous outcomes. Each chance is exclusive.

Oppositely, the experiment you suggest of rolling an actual dice is not a determination of the general probability but an analysis on the "fairness" of the specific dice tested - whether it is balanced properly to give equal odds to all outcomes. It is possible that the dice may be flawed, where something is affecting the outcome to skew the results - for instance, one side is sticky or weighs more, leading that side more likely to land faced down.

I am sure the Mathematics.SE can provide a better explanation of how to design this experiment. There are guidelines in statistics on how to determine the parameters of what would be considered a legitimate sample size. The larger the sample size and the more controls established in the experiment to eliminate other variables, the smaller the degree of statistical error, and therefore the better you can determine if the dice is fair or flawed.

To the point you are trying to prove to your family, while it is smart to point to the probability of winning in general, it is more significant to compare the probability that you will win, versus the probability the house will win. For instance, let's consider the following game:

The game is to guess what number will appear when rolling a single, fair 6-sided dice. If you guess correctly, you win. If you guess incorrectly, the house wins. Therefore, the probability you will win is 1/6, but the probability the house will win is 5/6 (they win with every number other than the one you chose).

The casino will try to entice you to bet by offering you a high payout; e.g. "if you win, you will get 6 times the amount you bet". However, because you are much less likely to win, they will more often collect your bet, than they will payout to you. Furthermore, if you do win, and they entice you to continue to play with the money you've won, they are more likely to retrieve your winnings in the end.

There is no point where repetition becomes proof. So more testing does not prove anything.

Randomness encountered in physical processes does not proceed from mathematical or philosophical principles. Nor is it ensured by testing, as your test has a definite likelihood of coming out in various misleading ways.

The distribution or probabilities proceeds from the symmetries in a situation and the fact that on a macroscopic level physics treats symmetrical situations symmetrically. The die has to come down on some side, and the process deciding this is symmetrical between sides.

Also, if the situation is not symmetrical, the outcome is not uniformly distributed. Since measurement is never exact, a die is not perfectly symmetrical. So to some very fine level of detail, the odds for each side of the die are not equal.

• @alampert22 I just meant to point out that there is a continuum, there are not random-behaving objects, and deterministic-behaving objects. Dice are not somehow not about physics, and instead about statistics.
– user9166
Jun 2, 2017 at 23:32
• I understand your point about the most minute imperfections of anything man-made (or that there is no such thing as the idyllic dice) but if it is so small that it has not conceivable difference or impact on the outcome, isn't it moot? Isn't it like saying, you are not 6 feet tall, you are 6 foot and 1 micrometer. Jun 2, 2017 at 23:33

The philosophical problem here (insofar as one exists) as opposed to the mathematical one, is the confusion of incidence with probability.

As specified if you were to roll a fair die a large number of times the incidence of any given number would tend towards one in every six rolls as the number of rolls in the test approaches infinity.

In an actual event (in a fully deterministic universe) the probability of the die landing on any given number for a specific roll is either 0% or 100%, it is governed by the laws of physics acting on the die up to the time it lands. We just don't have the data to calculate what side it will land on so we substitute the proportion of incidence in an infinite test series for the data we don't have, presuming that we know the variance of those factors (the range of options). In this case probability is just a statement about the extent of unknown factors. If, for example, we had a small amount of that data (say we had some good model of how a die will land given it's starting position and we knew that position) then the probability of it landing on a particular number would not be 1 in 6, it would be some other figure because we know some of the factors affecting it but not others. This would continue up until we had a complete model of the throw of a die when the probability of it landing on any given number would be either 100% or 0% depending on the starting points of all those factors in the model.

In a non-deterministic universe but still a realistic one the situation would be different, but not significantly so. It may be that uncertainty could affect the factors that influence where the die will land, but given the weight of factors affecting it which have already been determined (the starting position, the speed and path of every air molecule it passes etc.) I do not see that the extent to which uncertainty in those factors which have yet to be determined as the die is thrown could weigh that heavily on the outcome.

In a deterministic non-realist universe, of course, we have no idea what the probability is because either the entire event is taking place in our minds anyway, or some divine being has willed the entire event to take place, in which case the whole subject is moot.

• That's a interesting answer (building off of the "deterministic-object" jobermark describes in his answer. When you say 1, is that a "100%" likelihood? Jun 4, 2017 at 11:54
• Yes, I generally deal with probabilities as values from 0 to 1, but percentages are more readily understood by non-statisticians I think, I've edited the answer to be consistent, thanks.
– user22791
Jun 4, 2017 at 12:47