Philosophy Stack Exchange is a question and answer site for those interested in the study of the fundamental nature of knowledge, reality, and existence. It only takes a minute to sign up.
Sign up to join this community
I am studying for an exam and I ran into frequentism. Honestly, I don't understand anything about that. Is frequentism related to probability only? Why are probabilities understood as frequencies? I think they are different from frequencies. Furthermore, what are frequentism's flaws?
Sorry for the question but I am so confused about this topic.
Wikipedia provides a useful stake in the ground for this query insofar as it notes the probabilistic grounding of frequentism (e.g., see link 1 below). That is where I would suggest the OP begin his exploration. Not being one to make fine and hard distinctions between disciplines I would only note that Wiki's summaries move the debate away from the philosophical and towards more mathematical and statistical terrain.
Indeed, in the last few decades there have been wars of religion between two schools of statisticians, one calling themselves frequentists -- associated with classic ideas of inference and probability as typically taught in Stats 101 classes, e.g., p-values less than 0.05 as a decision tool -- versus Bayesians who reject this classic framework by substituting credibility or strength/degrees of belief as opposed to cut-offs. Bayesians also set up an iterative, tri-partite framework for inference based on Bayes' Rule (e.g., see link 2 below) originating with prior or "subjective" assumptions and beliefs (usually defined by a theoretical statistical distribution for the data or information under analysis), followed by empirical estimation of a likelihood as a function of those prior beliefs and ending with summarizing the posterior results of that estimation which is used to update the priors. Bayesian approaches to estimation resemble simulations and can be quite computationally intensive, making them cumbersome and slow with large amounts of data. Frequentists, on the other hand, strenuously object to the use of subjective assumptions.
That said, philosophers of science have weighed in on these issues. Among the important frequentist contributions is Deborah Mayo's recent book Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars. Here is the Amazon abstract:
Mounting failures of replication in social and biological sciences give a new urgency to critically appraising proposed reforms. This book pulls back the cover on disagreements between experts charged with restoring integrity to science. It denies two pervasive views of the role of probability in inference: to assign degrees of belief, and to control error rates in a long run. If statistical consumers are unaware of assumptions behind rival evidence reforms, they can't scrutinize the consequences that affect them (in personalized medicine, psychology, etc.). The book sets sail with a simple tool: if little has been done to rule out flaws in inferring a claim, then it has not passed a severe test. Many methods advocated by data experts do not stand up to severe scrutiny and are in tension with successful strategies for blocking or accounting for cherry picking and selective reporting. Through a series of excursions and exhibits, the philosophy and history of inductive inference come alive. Philosophical tools are put to work to solve problems about science and pseudoscience, induction and falsification.
On the Bayesian side I like McCloskey's The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives which, among other things, is highly critical of classic, frequentist hypothesis testing as developed by R.A. Fisher, one of the great geniuses of 20th c statistics. Again, the Amazon abstract:
The Cult of Statistical Significance shows, field by field, how “ statistical significance,” a technique that dominates many sciences, has been a huge mistake. The authors find that researchers in a broad spectrum of fields, from agronomy to zoology, employ “ testing” that doesn’t test and “ estimating” that doesn’t estimate. The facts will startle the outside reader: how could a group of brilliant scientists wander so far from scientific magnitudes? This study will encourage scientists who want to know how to get the statistical sciences back on track and fulfill their quantitative promise. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots.
As with many human endeavors there are no unambiguous answers to this debate.
Gamblers, actuaries, and scientists have long understood that relative frequencies bear an intimate relationship to probabilities.
Frequency interpretations posit the most intimate relationship of all: identity. Thus, we might identify the probability of ‘heads’ on a certain coin with the frequency of heads in a suitable sequence of tosses of the coin, divided by the total number of tosses.
A simple version of frequentism, which we will call finite frequentism, attaches probabilities to events or attributes in a finite reference class in such a straightforward manner:
the probability of an attribute A in a finite reference class B is the relative frequency of actual occurrences of A within B.
Thus, finite frequentism bears certain structural similarities to the classical interpretation, insofar as it gives equal weight to each member of a set of events, simply counting how many of them are ‘favorable’ as a proportion of the total.
The crucial difference, however, is that where the classical interpretation counted all the possible outcomes of a given experiment, finite frequentism counts actual outcomes.
It was developed by Venn (1876), who in his discussion of the proportion of births of males and females, concludes: “probability is nothing but that proportion” (p. 84, his emphasis).
Finite frequentism remains the dominant view of probability in statistics, and in the sciences more generally. Finite frequentism gives an operational definition of probability, and its problems begin there.
For example, just as we want to allow that our thermometers could be ill-calibrated, and could thus give misleading measurements of temperature, so we want to allow that our ‘measurements’ of probabilities via frequencies could be misleading, as when a fair coin land heads 9 out of 10 times.
More than that, it seems to be built into the very notion of probability that such misleading results can arise. Indeed, in many cases, misleading results are guaranteed. Starting with a degenerate case:
according to the finite frequentist, a coin that is never tossed, and that thus yields no actual outcomes whatsoever, lacks a probability for heads altogether; yet a coin that is never measured does not thereby lack a diameter.
Perhaps even more troubling, a coin that is tossed exactly once yields a relative frequency of heads of either 0 or 1, whatever its bias. Or we can imagine a unique radioactive atom whose probabilities of decaying at various times obey a continuous law (e.g. exponential);
yet according to finite frequentism, with probability 1 it decays at the exact time that it actually does, for its relative frequency of doing so is 1/1. Famous enough to merit a name of its own, these are instances of the so-called ‘problem of the single case’. In fact, many events are most naturally regarded as not merely unrepeated, but in a strong sense unrepeatable —
the 2000 presidential election, the final game of the 2001 NBA playoffs, the Civil War, Kennedy's assassination, certain events in the very early history of the universe. Nonetheless, it seems natural to think of non-extreme probabilities attaching to some, and perhaps all, of them. Worse still, some cosmologists regard it as a genuinely chancy matter whether our universe is open or closed (apparently certain quantum fluctuations could, in principle, tip it one way or the other), yet whatever it is, it is ‘single-case’ in the strongest possible sense.
The problem of the single case is particularly striking, but we really have a sequence of related problems:
‘the problem of the double case’, ‘the problem of the triple case’ … Every coin that is tossed exactly twice can yield only the relative frequencies 0, 1/2 and 1, whatever its bias… A finite reference class of size n, however large n is, can only produce relative frequencies at a certain level of ‘grain’, namely 1/n.
Among other things, this rules out irrational probabilities; yet our best physical theories say otherwise. Furthermore, there is a sense in which any of these problems can be transformed into the problem of the single case.
Suppose that we toss a coin a thousand times. We can regard this as a single trial of a thousand-tosses-of-the-coin experiment. Yet we do not want to be committed to saying that that experiment yields its actual result with probability 1.
The problem of the single case is that the finite frequentist fails to see intermediate probabilities in various places where others do.
There is also the converse problem: the frequentist sees intermediate probabilities in various places where others do not. Our world has myriad different entities, with myriad different attributes.
We can group them into still more sets of objects, and then ask with which relative frequencies various attributes occur in these sets. Many such relative frequencies will be intermediate; the finite frequentist automatically identifies them with intermediate probabilities. But it would seem that whether or not they are genuine probabilities, as opposed to mere tallies, depends on the case at hand.
Bare ratios of attributes among sets of disparate objects may lack the sort of modal force that one might expect from probabilities.
I belong to the reference class consisting of myself, the Eiffel Tower, the southernmost sandcastle on Santa Monica Beach, and Mt Everest. Two of these four objects are less than 7 ft. tall, a relative frequency of 1/2; moreover, we could easily extend this class, preserving this relative frequency (or, equally easily, not).
Yet it would be odd to say that my probability of being less than 7 ft. tall, relative to this reference class, is 1/2, even though it is perfectly acceptable (if uninteresting) to say that 1/2 of the objects in the reference class are less than 7 ft. tall.
Some frequentists (notably Venn 1876, Reichenbach 1949, and von Mises 1957 among others), partly in response to some of the problems above, have gone on to consider infinite reference classes, identifying probabilities with limiting relative frequencies of events or attributes therein.
Thus, we require an infinite sequence of trials in order to define such probabilities.
But what if the actual world does not provide an infinite sequence of trials of a given experiment? Indeed, that appears to be the norm, and perhaps even the rule.
In that case, we are to identify probability with a hypothetical or counterfactual limiting relative frequency. We are to imagine hypothetical infinite extensions of an actual sequence of trials;
probabilities are then what the limiting relative frequencies would be if the sequence were so extended. We might thus call this interpretation hypothetical frequentism.
Note that at this point we have left empiricism behind. A modal element has been injected into frequentism with this invocation of a counterfactual; moreover, the counterfactual may involve a radical departure from the way things actually are, one that may even require the breaking of laws of nature. (Think what it would take for the coin in my pocket, which has only been tossed once, to be tossed infinitely many times — never wearing out, and never running short of people willing to toss it!) One may wonder, moreover, whether there is always — or ever — a fact of the matter of what such counterfactual relative frequencies are.
Limiting relative frequencies, we have seen, must be relativized to a sequence of trials. Herein lies another difficulty. Consider an infinite sequence of the results of tossing a coin, as it might be H, T, H, H, H, T, H, T, T, … Suppose for definiteness that the corresponding relative frequency sequence for heads, which begins 1/1, 1/2, 2/3, 3/4, 4/5, 4/6, 5/7, 5/8, 5/9, …, converges to 1/2.
By suitably reordering these results, we can make the sequence converge to any value in [0, 1] that we like. (If this is not obvious, consider how the relative frequency of even numbers among positive integers, which intuitively ‘should’ converge to 1/2, can instead be made to converge to 1/4 by reordering the integers with the even numbers in every fourth place, as follows: 1, 3, 5, 2, 7, 9, 11, 4, 13, 15, 17, 6, …) To be sure, there may be something natural about the ordering of the tosses as given — for example,
it may be their temporal ordering. But there may be more than one natural ordering. Imagine the tosses taking place on a train that shunts backward and forwards on tracks that are oriented west-east. Then the spatial ordering of the results from west to east could look very different. Why should one ordering be privileged over others?
A well-known objection to any version of frequentism is that relative frequencies must be relativized to a reference class. Consider a probability concerning myself that I care about — say, my probability of living to age 80. I belong to the class of males, the class of non-smokers, the class of philosophy professors who have two vowels in their surname, … Presumably, the relative frequency of those who live to age 80 varies across (most of) these reference classes.
What, then, is my probability of living to age 80? It seems that there is no single frequentist answer. Instead, there is my probability-qua-male, my probability-qua-non-smoker, my probability-qua-male-non-smoker, and so on.
This is an example of the so-called reference class problem for frequentism (although it can be argued that analogs of the problem arise for the other interpretations as well[9]). And as we have seen in the previous paragraph, the problem is only compounded for limiting relative frequencies: probabilities must be relativized not merely to a reference class, but to a sequence within the reference class. We might call this the reference sequence problem.
The beginnings of a solution to this problem would be to restrict our attention to sequences of a certain kind, those with certain desirable properties.
For example, there are sequences for which the limiting relative frequency of a given attribute does not exist; Reichenbach thus excludes such sequences. Von Mises (1957) gives us a more thoroughgoing restriction to what he calls collectives — hypothetical infinite sequences of attributes (possible outcomes) of specified experiments that meet certain requirements. Call a place-selection an effectively specifiable method of selecting indices of members of the sequence, such that the selection or not of the index i depends at most on the first i − 1 attributes. The axioms are:
An axiom of Convergence: the limiting relative frequency of an attribute exists.
An axiom of Randomness: the limiting relative frequency of each attribute in a collective ω is the same in any infinite subsequence of ω which is determined by a place selection. The probability of an attribute A, relative to a collective ω, is then defined as the limiting relative frequency of A in ω. Note that a constant sequence such as H, H, H, …, in which the limiting relative frequency is the same in any infinite subsequence, trivially satisfies the axiom of randomness. This puts some strain on the terminology — offhand, such sequences appear to be as non-random as they come — although to be sure it is desirable that probabilities be assigned even in such sequences. Be that as it may, there is a parallel between the role of the axiom of randomness in von Mises' theory and the principle of maximum entropy in the classical theory: both attempt to capture a certain notion of a disorder.
Let us see how the frequentist interpretations fare according to our criteria of adequacy. Finite relative frequencies, of course, satisfy finite additivity. In a finite reference class, only finitely many events can occur, so only finitely many events can have a positive relative frequency. In that case, countable additivity is satisfied somewhat trivially: all but finitely many terms in the infinite sum will be 0. Limiting relative frequencies violate countable additivity (de Finetti 1972, §5.22). Indeed, the domain of definition of limiting relative frequency is not even a field, let alone a sigma field (de Finetti 1972, §5.8). So such relative frequencies do not provide an admissible interpretation of Kolmogorov's axioms.
Finite frequentism has no trouble meeting the ascertainability criterion, as finite relative frequencies are in principle easily determined. The same cannot be said of limiting relative frequencies. On the contrary, any finite sequence of trials (which, after all, is all we ever see) puts literally no constraint on the limit of an infinite sequence; still less does an actual finite sequence put any constraint on the limit of an infinite hypothetical sequence, however fast and loose we play with the notion of ‘in principle’ in the ascertainability criterion.
It might seem that the frequentist interpretations resoundingly meet the applicability to frequencies criterion. Finite frequentism meets it all too well, while hypothetical frequentism meets it in the wrong way. If anything, finite frequentism makes the connection between probabilities and frequencies too tight, as we have already observed.
A fair coin that is tossed a million times is very unlikely to land heads exactly half the time; one that is tossed a million and one times is even less likely to do so! Facts about finite relative frequencies should serve as evidence, but not conclusive evidence, for the relevant probability assignments. Hypothetical frequentism fails to connect probabilities with finite frequencies.
It connects them with limiting relative frequencies, of course, but again too tightly: for even in infinite sequences, the two can come apart. (A fair coin could land heads forever, even if it is highly unlikely to do so.) To be sure, science has much interest infinite frequencies, and indeed working with them is much of the business of statistics.
Whether it has any interest in highly idealized, hypothetical extensions of actual sequences and relative frequencies therein is another matter. The applicability to rational opinion goes much the same way: it is clear that such opinion is guided by finite frequency information, unclear that it is guided by information about the limits of hypothetical frequencies.
For much more extensive critiques of finite frequentism and hypothetical frequentism, see Hájek 1997 and Hájek 2009 respectively.
Ref.- https://plato.stanford.edu/entries/probability-interpret/#CriAdeForIntPro
