Why does statistics work?

Question

Statistics deals with probability, where even an extremely unlikely event has some chance of happening. What if there's a series of these unlikely events going on for thousands of year. I mean it could happen, there is some chance. For example, what if the entire history of the universe is just an unlikely sequence of events that has absolutely no patterns in it. That means, every statistical model that tries to model reality or data is plain wrong. So why does statistics work at all?

For example, imagine you a flip a coin for millions of years and you only get heads. It is possible. Then you make a statistical model that says the probability of flipping a coin that lands on head is 1. You keep using this model for thousands of years, believing that it's correct. Then, you get 1000 tails in a row all of a sudden. Everybody would be amazed how this is possible. I know this is an unlikely event happening, but the point is that it could happen. Is all of statistics flawed by the survivorship bias? How can we take statistics seriously? Or is statistics rather about trying random things out, and if it works we're happy until it stops working?

Answer 1

I have the impression there's a general confusion between probability theory and statistics. The two are of course related, but not identical. Basically, statistics is about existing data, while probability is about prediction. The connection is that if the number of repetitions of a random experiment goes to infinity, the statistics is expected (not guaranteed!) to converge against the probability. That is, if you've assigned the probabilities correctly, the probability that the statistics deviates from the prediction from probability theory goes to zero in the limit of infinite experiments (note however that the probability that you get an exact agreement between probability and statistics goes to zero!). And due to that connection, if we have sufficient statistics, we take the statistics results as probabilities for further predictions, assuming that the statistics closely agrees with the probabilities, because provided that we are not missing any important influence on or regularity of the outcome, it is unlikely that the sufficient statistics deviates significantly from the actual probability.

Now let's look at the example from the question. The coin in question is supposed to be a fair coin. Nevertheless it is assumed that the coin has produced only heads for millions of years. I'll assume it has been tossed frequently during that time (because otherwise, a coin that is thrown once every ten million years and produced heads three times in a row would also fit that description).

So the statistics says that the coin showed 100% heads for very many tosses. There's nothing wrong with that statistics, it tells us the facts.

Based on that statistics, people made a model that this coin is not a fair coin, but actually is a coin that almost certainly produces heads on each toss. If we take as two competing models the model of the coin as a fair one, and the model of a coin as a head-producing one, probability theory tells us that the second one is much more likely to be true than the first one. Note that it doesn't tell us that the second one is true. It just tells us that we most likely get better prediction with that assumption. And for millions of years that was actually true for this coin.

Now that coin suddenly starts to produce tails. And not just one or two, but a thousand in a row. Now, this doesn't invalidate the statistics. The tosses before still are 100% heads. And even including those 1000 heads, the statistics is still damn near at 100% heads. So the heads-only model still works better than the fair coin model. However, the most likely explanation ion that case would be that something affecting the coin changed (although nobody knows what did change), and thus the probabilities of the coin after the change are not the same as the probabilities before. Note that the fair coin assumption still isn't ruled out (it even got slightly less improbable), but it's still very unlikely (because also 1000 tails in a row are a very improbable event for a fair coin).

And BTW, even if the inhabitants of that world were wrong with their model of the coin, for their purposes the wrong model was better than the correct one for millions of years, because during that time their wrong model actually made the correct predictions. Indeed, given that fact, one may actually question whether their model really was "wrong".

Answer 2

There are very few things beyond mathematics that involve certainty statements. Even most physics, which many would likely consider the next best thing wrt "purity" is based on numerous observations that are accounted for, warranting the formulation of a hypothesis, after which this hypothesis is tested repeatedly (e.g. by making predictions based on the assumed truth of the hypothesis, and then verifying these predictions).

This is nothing but the scientific method.

Statistics is just a tool (pardon me, Mr Fischer and so many other good folk) that will help you make less errors, or at the least to some extent inform you of how likely it is you've made an error. Not, and to my knowledge never (but I will welcome any nontrivial counterexample), will it lead to absolute truths. Statistics is exactly (in my vision) about deciding how big you will allow your error margin to be (note that setting the heads probability to 1 implies that you are exactly sure of this value). Making the wrong kind of conclusions, but informedly so, is embedded in statistics. Making the wrong kind of conclusion, but wrongly so (turning a high likelihood into an absolute certainty) is human error (or at best, a computational artifact), and doesn't make statistics unuseful or incorrect, just abused.

Did I just write a poem on statistics?

Answer 3

Statistics, in common with most if not all mathematical tools, is a means of constructing models. Any one statistical model is in fact a scientific theory, which makes predictions.

The predictions of statistics are less ironclad, harder to falsify than deterministic models, for exactly the reason you describe. But just because an event is supposed to happen with some tiny non-zero probability, does not mean we should simply shrug when it occurs, just because it's possible. If an event which is meant to have probability 10^-100 per year occurs, this is a hint that the model which assigned that probability is (probably) wrong. Sure, that rare event could happen, but it also could happen that we are mistaken, and we are much more often mistaken than once every 10¹⁰⁰ years.

In the end, as with any model, a statistical model is only worthwhile to the extent that it is useful. If it does not predict frequencies to within "reasonable" tolerances, it is not likely to be useful. If you have stronger requirements for small tolerances than statistics can provide, that just means you need a model which is not statistical; though perhaps, if you're lucky, such a model might exist.

Answer 4

I know this is an unlikely event happening, but the point is that it could happen.

If the many-worlds interpretation of quantum mechanics is correct, then this does happen!

Is all of statistics flawed by the survivorship bias? How can we take statistics seriously? Or is statistics rather about trying random things out, and if it works we're happy until it stops working?

I suggest viewing all math as modeling a possible world. In your hypothetical scenario, one cannot use a static probability to back the coin. Interestingly enough, your hypothetical is virtually identical to a hypothetical discussed in the history of the philosophy of science:

Consider the physical law that F = ma up until 1916, after which General Relativity holds.

There is nothing wrong with this statement, logically. But we greatly dislike discontinuities like this, because they make it hard if not impossible to understand the past. So we do our best to work under the assumption that no fundamental physical laws take this form. This being said, it is possible that fundamental physical constants could change slightly from one part of space to another—perhaps in a continuous fashion. This, we are more likely to be able to understand.

I wonder if you're really asking: Why does reality seem to be rationally understandable?

My claim is that if statistics didn't describe our actual world, that it would not likely be rationally understandable. This has hints of the anthropic principle, given that if the universe weren't rationally understandable, we wouldn't be observing that fact. So, I would suggest a reformulation that is probably too broad for Philosophy.SE: What makes a possible world comprehensible to a mind? It could turn out that only comprehensible worlds can contain minds—that you simply cannot construct a mind within an incomprehensible world. At the very least, you'd have to provide an "island of comprehensibility" where the mind would dwell—probably unsatisfactorily.

Therefore, I say that statistics probably (heh) works necessarily, because you are asking. (!)

Answer 5

Statistics works within the constraints of the distribution of the underlying population and the information that can be derived and or inferred from it. Mathematics is an abstract logical process. Statistics is primarily concerned with the empirical application of logical relationships. This means that Mathematics lends itself to formal logical tests and Statistics is best suited to Empirical Science both of which are required for the investigations that are deemed to be scientific.

Answer 6

In the case of the coin one would discuss the probability of chains occuring and the probability of an individual coin toss seperately as they are two different questions which you shouldn't conflate - experiment and hypothesis design is vital in statistics as you need to understand exactly what you're measuring.

Fundamentally, statistics (since it works on observations) is a question of induction - there are a priori rules, but the application is typically based on the data you to have hand. At this point you could say you are Russell's Inductive Chicken and take a grim view of things but being able to allow for doubt is fundamentally what statistics is about - it is prediction not fact assertion. Induction is a favourite topic of mine, and I can highly recommend reading some 'classic' works on it - Russell and Hume

Answer 7

Statistics is the formal mathematical apparatus encoding the idea of the necessary, the probable, rare and impossible; plus all the possibilities in between.

But note that not all possibilities can be thus encoded. For example, an alien spaceship orbiting Mars is significantly more probable than a Victorian era teapot orbiting Jupiter.

Answer 8

We can never absolutly define something as true or false since everything is based on previous assumptions which so far can't be disproven as true.

That being said statistics is built on the supposition that the universe isn't just a huge pattern-free coincidence. If it was, then everything explained by statistics would be false and there would be no way for us to know it.

Note: This is my first post on the philosophy site, and english isn't my first lenguage. Please don't kill me.

Answer 9

The short answer -- no reason. There is no -- and never will be -- a rational explanation of why things don't happen at random.

However, one must have faith.

One must believe that while things don't necessarily happen for a reason,¹ there must be, nevertheless, a reason why things happen.² In other words, one must believe that everything must have had caused by some past event(s) -- if only for one's own existence³ to make any sense at all.

I elaborated on the last point elsewhere, but in short, To Be is to Do, and we must believe that things do not happen at random in order to have any freedom whatsoever, the freedom to do anything at all.

 

¹ "for a reason" as in "leading to some specific outcome"

² "Through it ["it" as "ὁ Λόγος(Lógos)", "the Reason", also known in those days as "the Divine Spirit", of which every human would get their own personal copy, known in those days as "a human Soul"] all things were made; without it nothing was made that has been [ever] made." -- John 1:3
(not trying to promote Christian (or Hindu, or any other) faith in particular, but merely pointing out that the above is by no means a novel concept)

³ ... and we ourselves do exist -- in fact, our own existence is the only fact we know for sure, a premise otherwise known as "Cartesian doubt"