# Why does statistics work?

Statistics deals with probability, where even an extremely unlikely event has some chance of happening. What if there's a series of these unlikely events going on for thousands of year. I mean it could happen, there is some chance. For example, what if the entire history of the universe is just an unlikely sequence of events that has absolutely no patterns in it. That means, every statistical model that tries to model reality or data is plain wrong. So why does statistics work at all?

For example, imagine you a flip a coin for millions of years and you only get heads. It is possible. Then you make a statistical model that says the probability of flipping a coin that lands on head is 1. You keep using this model for thousands of years, believing that it's correct. Then, you get 1000 tails in a row all of a sudden. Everybody would be amazed how this is possible. I know this is an unlikely event happening, but the point is that it could happen. Is all of statistics flawed by the survivorship bias? How can we take statistics seriously? Or is statistics rather about trying random things out, and if it works we're happy until it stops working?

• @user603 Perhaps - or at least I'd think a candidate for community wiki. As written, I think it falls under several of the "not constructive" items on the vote-to-close-dialog, suggesting it should be closed on that basis, but perhaps siamii can reword it to be a better fit for the Q&A form of stackexchange (making it less of an editorial, and more specific in its questions). I don't think philosophy of statistics questions are necessarily off-limits, but the particular format of these questions concerns me. – Glen_b May 13 '13 at 23:09
• Seein a coin tossed comes so many times heads in the past and than assigning it a probability of 1 is not a statistical model development. You get "statistical model development" wrong. In the example you gave, people would be suprised in two ways when they see that now get tails: 1-) A deviation from historic behaviour 2-) IF they developed a (scientific/mathematical) model that was consistent with earlier common heads and having now 1000 tails is unexpected based on model, they suprise as they model turned out to be wrong. (See next comment pls). – mami May 16 '13 at 6:43
• Your suprise is different. In your example you know a priori that your coin is 50% heads/tails. If the people you mention share this knowledge, why do you suppose that they will develop a model (before seeing 1000 tail) where in their model the probabilities will be 100%-0%??? I guess you create inconsistency yourself in your example and then start being suspicious about statistical analysis. – mami May 16 '13 at 6:44
• They don't work. I took statistics in college. First day of class our professor made it very clear, "Statistics can be altered to meet the means of the maker, thus statistics is not a perfect science." I'll never forget that, and if you look at the news, you'll see just how true it is every day. – SpYk3HH Jun 7 '13 at 14:04
• @SpYk3HH This squarely (and rather naively) confuses a scientific field and its (mis-)applications. – Did Jul 28 '13 at 10:20

I have the impression there's a general confusion between probability theory and statistics. The two are of course related, but not identical. Basically, statistics is about existing data, while probability is about prediction. The connection is that if the number of repetitions of a random experiment goes to infinity, the statistics is expected (not guaranteed!) to converge against the probability. That is, if you've assigned the probabilities correctly, the probability that the statistics deviates from the prediction from probability theory goes to zero in the limit of infinite experiments (note however that the probability that you get an exact agreement between probability and statistics goes to zero!). And due to that connection, if we have sufficient statistics, we take the statistics results as probabilities for further predictions, assuming that the statistics closely agrees with the probabilities, because provided that we are not missing any important influence on or regularity of the outcome, it is unlikely that the sufficient statistics deviates significantly from the actual probability.

Now let's look at the example from the question. The coin in question is supposed to be a fair coin. Nevertheless it is assumed that the coin has produced only heads for millions of years. I'll assume it has been tossed frequently during that time (because otherwise, a coin that is thrown once every ten million years and produced heads three times in a row would also fit that description).

So the statistics says that the coin showed 100% heads for very many tosses. There's nothing wrong with that statistics, it tells us the facts.

Based on that statistics, people made a model that this coin is not a fair coin, but actually is a coin that almost certainly produces heads on each toss. If we take as two competing models the model of the coin as a fair one, and the model of a coin as a head-producing one, probability theory tells us that the second one is much more likely to be true than the first one. Note that it doesn't tell us that the second one is true. It just tells us that we most likely get better prediction with that assumption. And for millions of years that was actually true for this coin.

Now that coin suddenly starts to produce tails. And not just one or two, but a thousand in a row. Now, this doesn't invalidate the statistics. The tosses before still are 100% heads. And even including those 1000 heads, the statistics is still damn near at 100% heads. So the heads-only model still works better than the fair coin model. However, the most likely explanation ion that case would be that something affecting the coin changed (although nobody knows what did change), and thus the probabilities of the coin after the change are not the same as the probabilities before. Note that the fair coin assumption still isn't ruled out (it even got slightly less improbable), but it's still very unlikely (because also 1000 tails in a row are a very improbable event for a fair coin).

And BTW, even if the inhabitants of that world were wrong with their model of the coin, for their purposes the wrong model was better than the correct one for millions of years, because during that time their wrong model actually made the correct predictions. Indeed, given that fact, one may actually question whether their model really was "wrong".

• "something affecting the coin changed". Why? Nothing has changed. it's just randomness. The probabilities are still the same, which is 1/2. "one may actually question whether their model really was "wrong"" Indeed. The point is that we cannot know the true probabilities, which can be anything. Any model we base on observations might be totally wrong. Their model worked for millions of years, then it stopped working. There's no explanation why it worked or why it stopped working. It's just pure chance. Do the models that WE use now work by chance too? – siamii Jun 1 '13 at 15:26
• ""something affecting the coin changed". Why?" Because that's the reasonable assumption if you see such a behaviour. Remember, the presumption was that the people observing the coin do not know from somewhere else that the coin is indeed fair. "Do the models that WE use now work by chance too?" We never can exclude that possibility. But it is extremeyly unlikely, just as the scenario that you constructed (a fair coin giving only heads in millions of years, then giving 1000 tails in a row) is extremely unlikely. It is much more likely in that case that there's something that determined … – celtschk Jun 1 '13 at 15:44
• that behaviour (and indeed, there is: It was you who determined it, to provide an example :-)). Nothing about the world is ever 100% certain. But if something is sufficiently unlikely given the evidence, it is reasonable to act as if it wasn't possible. – celtschk Jun 1 '13 at 15:45

There are very few things beyond mathematics that involve certainty statements. Even most physics, which many would likely consider the next best thing wrt "purity" is based on numerous observations that are accounted for, warranting the formulation of a hypothesis, after which this hypothesis is tested repeatedly (e.g. by making predictions based on the assumed truth of the hypothesis, and then verifying these predictions).

This is nothing but the scientific method.

Statistics is just a tool (pardon me, Mr Fischer and so many other good folk) that will help you make less errors, or at the least to some extent inform you of how likely it is you've made an error. Not, and to my knowledge never (but I will welcome any nontrivial counterexample), will it lead to absolute truths. Statistics is exactly (in my vision) about deciding how big you will allow your error margin to be (note that setting the heads probability to 1 implies that you are exactly sure of this value). Making the wrong kind of conclusions, but informedly so, is embedded in statistics. Making the wrong kind of conclusion, but wrongly so (turning a high likelihood into an absolute certainty) is human error (or at best, a computational artifact), and doesn't make statistics unuseful or incorrect, just abused.

Did I just write a poem on statistics?

Statistics, in common with most if not all mathematical tools, is a means of constructing models. Any one statistical model is in fact a scientific theory, which makes predictions.

The predictions of statistics are less ironclad, harder to falsify than deterministic models, for exactly the reason you describe. But just because an event is supposed to happen with some tiny non-zero probability, does not mean we should simply shrug when it occurs, just because it's possible. If an event which is meant to have probability 10-100 per year occurs, this is a hint that the model which assigned that probability is (probably) wrong. Sure, that rare event could happen, but it also could happen that we are mistaken, and we are much more often mistaken than once every 10100 years.

In the end, as with any model, a statistical model is only worthwhile to the extent that it is useful. If it does not predict frequencies to within "reasonable" tolerances, it is not likely to be useful. If you have stronger requirements for small tolerances than statistics can provide, that just means you need a model which is not statistical; though perhaps, if you're lucky, such a model might exist.

• Does every statistical model make predictions? Can't statistics be just about the past as well? Or how do you justify this claim? I would like it to be true :) – Kriss Jun 1 '13 at 11:58
• @Kriss: To the extent that what you have is a model, rather than just a list of measured facts that happened at some point, what you have is a claim to knowledge of regularity: that if the same situation were to hold again as what was described by the model, that the probabilities and correlations described by the model would continue to hold true, and were not simply accidents when they occurred in the past. – Niel de Beaudrap Jun 1 '13 at 12:45

I know this is an unlikely event happening, but the point is that it could happen.

If the many-worlds interpretation of quantum mechanics is correct, then this does happen!

Is all of statistics flawed by the survivorship bias? How can we take statistics seriously? Or is statistics rather about trying random things out, and if it works we're happy until it stops working?

I suggest viewing all math as modeling a possible world. In your hypothetical scenario, one cannot use a static probability to back the coin. Interestingly enough, your hypothetical is virtually identical to a hypothetical discussed in the history of the philosophy of science:

Consider the physical law that F = ma up until 1916, after which General Relativity holds.

There is nothing wrong with this statement, logically. But we greatly dislike discontinuities like this, because they make it hard if not impossible to understand the past. So we do our best to work under the assumption that no fundamental physical laws take this form. This being said, it is possible that fundamental physical constants could change slightly from one part of space to another—perhaps in a continuous fashion. This, we are more likely to be able to understand.

I wonder if you're really asking: Why does reality seem to be rationally understandable?

My claim is that if statistics didn't describe our actual world, that it would not likely be rationally understandable. This has hints of the anthropic principle, given that if the universe weren't rationally understandable, we wouldn't be observing that fact. So, I would suggest a reformulation that is probably too broad for Philosophy.SE: What makes a possible world comprehensible to a mind? It could turn out that only comprehensible worlds can contain minds—that you simply cannot construct a mind within an incomprehensible world. At the very least, you'd have to provide an "island of comprehensibility" where the mind would dwell—probably unsatisfactorily.

Therefore, I say that statistics probably (heh) works necessarily, because you are asking. (!)

• right... i should have read your reply before answering straight to "why reality is rationally understandable" :) – Yuri Alexandrovich Oct 21 '20 at 16:08

Statistics works within the constraints of the distribution of the underlying population and the information that can be derived and or inferred from it. Mathematics is an abstract logical process. Statistics is primarily concerned with the empirical application of logical relationships. This means that Mathematics lends itself to formal logical tests and Statistics is best suited to Empirical Science both of which are required for the investigations that are deemed to be scientific.

In the case of the coin one would discuss the probability of chains occuring and the probability of an individual coin toss seperately as they are two different questions which you shouldn't conflate - experiment and hypothesis design is vital in statistics as you need to understand exactly what you're measuring.

Fundamentally, statistics (since it works on observations) is a question of induction - there are a priori rules, but the application is typically based on the data you to have hand. At this point you could say you are Russell's Inductive Chicken and take a grim view of things but being able to allow for doubt is fundamentally what statistics is about - it is prediction not fact assertion. Induction is a favourite topic of mine, and I can highly recommend reading some 'classic' works on it - Russell and Hume

Statistics is the formal mathematical apparatus encoding the idea of the necessary, the probable, rare and impossible; plus all the possibilities in between.

But note that not all possibilities can be thus encoded. For example, an alien spaceship orbiting Mars is significantly more probable than a Victorian era teapot orbiting Jupiter.

• I'm not really sure what you mean by your second paragraph (e.g. how the two sentences relate to each other). – Niel de Beaudrap May 15 '13 at 13:59
• Are you saying that its not significantly more probable that an alien spaceship is orbiting Mars than a Victorian era teapot - and we can't quantify the probabilities of each? – Mozibur Ullah May 15 '13 at 14:12
• You wrote "not all possibilities can be thus encoded". What does it mean to encode a probability? Aside from the difficulty of discovering the probability of these two (or distinguishing them), I do not see why there is any theoretical problem with these events. – Niel de Beaudrap May 15 '13 at 14:40
• Maybe I should say assign? I just mean p=0.01 or p=0.1 etc. I don't think that the probabilities are discoverable, but I can distinguish them, and assert that one is less than the other. There is no problem with understanding what these two events are or what they mean. The theoretical problem here is whether there are quantifiable probabilities that can be assigned. You admit as much by saying 'the difficulty of discovering' them. It appears that you're saying that there are probabilities to these two events we just don't know them. Whereas I'm saying they're not there to be discovered. – Mozibur Ullah May 15 '13 at 14:53
• So what you're claiming is that probabilities are objectively real phenomena (as opposed to a part of the mathematical models we use to describe phenomena), except that in the case of Jovian Victorian teapots and Martian extrasolar UFOs, the probabilities don't actually exist? I would sooner say that these are phenomena which, to the best of our knowledge, have probability not practically distinguishable from zero, but in principle positive (essentially infinitesimal for the teapot, and simply quite small for the UFO, but no evidence to support a non-zero lower bound in either case). – Niel de Beaudrap May 15 '13 at 16:16

We can never absolutly define something as true or false since everything is based on previous assumptions which so far can't be disproven as true.

That being said statistics is built on the supposition that the universe isn't just a huge pattern-free coincidence. If it was, then everything explained by statistics would be false and there would be no way for us to know it.

Note: This is my first post on the philosophy site, and english isn't my first lenguage. Please don't kill me.

The short answer -- no reason. There is no -- and never will be -- a rational explanation of why things don't happen at random.

However, one must have faith.

One must believe that while things don't necessarily happen for a reason,1 there must be, nevertheless, a reason why things happen.2 In other words, one must believe that everything must have had caused by some past event(s) -- if only for one's own existence3 to make any sense at all.

I elaborated on the last point elsewhere, but in short, To Be is to Do, and we must believe that things do not happen at random in order to have any freedom whatsoever, the freedom to do anything at all.

1 "for a reason" as in "leading to some specific outcome"

2 "Through it ["it" as "ὁ Λόγος(Lógos)", "the Reason", also known in those days as "the Divine Spirit", of which every human would get their own personal copy, known in those days as "a human Soul"] all things were made; without it nothing was made that has been [ever] made." -- John 1:3
(not trying to promote Christian (or Hindu, or any other) faith in particular, but merely pointing out that the above is by no means a novel concept)

3 ... and we ourselves do exist -- in fact, our own existence is the only fact we know for sure, a premise otherwise known as "Cartesian doubt"

• I do not see any substantial point or argument about statistics, probability, and prediction here, only zealotry. You do not have to have faith to understand how statistical analysis and prediction works (either it is not the case within certain bounds of uncertainty - alpha-error - that the null hypothesis is false or it is within reasonable certainty false that the null hypothesis is true) and that it works in practice. For all we observe, most events are randomly distributed. – Philip Klöcking Oct 21 '20 at 12:22
• @Philip Klöcking "For all we observe, most events are randomly distributed" -- like things falling up or down at random? for all you observe? – Yuri Alexandrovich Oct 21 '20 at 15:51
• Come on, you can do better than that. Since "a thing falling up" is oxymoronic and not an event proper since it literally never happens, this attempt of ridicule is poorly chosen. – Philip Klöcking Oct 21 '20 at 17:27
• The first week of April in Berlin being warmer than the last week of February? – Yuri Alexandrovich Oct 21 '20 at 17:58
• Firstly, statistics is about observed events, not events observable according to some thought experiment. Secondly, hasn't been the case in 2003. – Philip Klöcking Oct 21 '20 at 18:27