How does Bernoulli's theorem make an inference from chance to frequencies and not vice versa?

Question

I am currently working on an assignment of mine about Frequentism and Bayesianism. In my studies I've found this source: http://statweb.stanford.edu/~cgates/PERSI/Courses/Phil166-266//IV-Frequency2013.pdf. Particularly, the following quotation on page 8 is quite perplexing.

What does it mean to determine chances a posteriori from frequencies? The question is, given the data - the number of trials, and the relative frequencies of success in those trials - what is the probability that the chances fall within a certain interval. It is evident that this is not the problem that Bernoulli solved. He solved an inference from chances to frequencies, not the inverse problem from frequencies to chances.

My first question is, I understand that 'a posteriori' means to obtain your knowledge through observation. But what is so perplexing about that? How are calculations of chance through frequencies somehow counter-intuitive and nonsensical to the frequentist interpetation?

My second question is, how exactly did he solve an inference from chances to frequencies and not the other way around? Clearly in the set up of his Theorem he takes observed values to somehow infer a value for the chance? Isn't that the whole idea behind the Weak Law of Large Numbers?

Answer 1

To understand this, it helps to be clear about the terminology. The term "chances" is usually used to mean a physical interpretation of probabilities. The idea is that a symmetrical, unbiased coin has a physical chance of landing heads with a probability of 0.5, or that an urn containing 1 million red marbles, 1 million blue and 1 million green, has a physical probability that a marble drawn at random will have a probability of 1/3 of being red. The first thing to note here is that this is only one way of understanding probabilities: on an epistemic account there are no chances, and what we call probabilities are statements about how much information we possess. I'll leave that thought to one side for a moment and proceed with chances, since the author of the article you reference believes in them.

The important question is, if I don't have any direct access to the physical chances, i.e. I can't test the symmetry of the coin or count all the marbles in the urn, and the only information I have is gleaned from performing sampling experiments and observing the frequencies in the sample, what inferential relationship is there between the physical chances and the observed frequencies of the samples? What Bernoulli showed is that one can draw an inference from the chances to the observed frequencies: this is the law of large numbers, and it implies that if you keep tossing the coin, the long run frequency of the sample result will converge to the physical chance. Or if you keep drawing marbles from the urn, the long run frequency of your sample will converge to the actual proportions of the marbles in the urn. Note that this is a convergence result only: it doesn't mean that your sample frequency must agree with the chances, only that it will tend towards doing so in the long run.

What Bernoulli doesn't tell you is how to perform the inverse inference from information about the observed frequencies of the sample to information about the chances. This is a very common problem in real life: often we want to know something about a system or a population, perhaps because we want to make future predictions about it, but we only have access to samples of data. Solving this problem is much more problematic and there is no perfect way to do it, or even a general consensus about how to go about it. Fisher held that the best approach is to make hypotheses about the chances, design experiments to test them, and reject the hypotheses if the test results are significant. Neyman and Pearson used the approach of defining type 1 and type 2 errors and of approaching the values of chances in terms of false positive and false negative error rates. Bayesians use Bayes' rule by specifying prior probability distributions and using the data to update those distributions. Likelihoodists take two rival hypotheses and calculate a likelihood ratio that allows one to say which hypothesis is confirmed relative to the other.

Answer 2

It is not clear that Bernoulli takes frequencies as proof of the probability. The idea that measuring frequency is what indicates the probability is the gambler's fallacy in disguise.

Any experiment meant to determine the probability from observed frequencies (and not a census) is still random, and will randomly mislead you as to the base probability a certain percentage of the time. Right? You can make the odds small, but you can never get rid of them.

So, if you want a known probability, then in principle, at some point, you need to base the probability on some cause other than observation of outcomes, such as the physics of a coin's construction, or an actual known distribution of incidences in a finite population.

In theory, you cannot use past observations to predict any single result in the future, because there is always a tiny chance your history is cursed, and you have no way of telling a representative history from a terribly biased one.