Suppose there is a society where most metrics of science are kinda distorted. For example you cannot trust on university degrees, resumes etc.

Can more accurate predictions (relative to other predictions) somehow be used as a basis for measuring knowledge between individuals (and not just between scientific theories)? Is there already a framework for it? Has it any special term in philosophy?


The term of art is "predictive success", see e.g. Statistical Inference for Measures of Predictive Success by Demuynck:

"We provide statistical inference for measures of predictive success. These measures are frequently used to evaluate and compare the performance of different models of individual and group decision making in experimental and revealed preference studies."

Older Selten's papers are still often cited in this context, e.g. Properties of a Measure of Predictive Success:

"Area theories for the prediction of experimental results delineate regions of predicted outcomes within the set of all possible outcomes. The difference measure of predictive success for area theories introduced by Selten and Krischker (1983) is the difference between hit rate and area. The hit rate is the relative frequency of successful predictions and the area is the relative size of the predicted region within the set of all possible outcomes. It is argued that other measures proposed in the literature are unreasonable with respect to the implied structure of unimprovable theories."

Searching for "predictive success of science" brings up more references for various other frameworks. However, finding a satisfactory metric is not straightforward, and all of them have their flaws. On this note see Christensen's Measuring Confirmation, which shows how difficult it is to craft a quantitative confirmation measure that reflects our intuitions about how much given evidence "confirms" a conclusion (and hence can validate "success" of theoretical predictions):

"But our ordinary concept of evidence involves a certain kind of epistemic asymmetry. Hypotheses are commonly taken to be supported by evidence, but not vice versa... It has long been clear that the traditional probabilistic account of confirmation would fail to capture the epistemic asymmetry of our ordinary notion. It now seems that probabilistic accounts will also miss our ordinary notion's dependence on the distinction between background beliefs and specific evidence. I would not claim that these are the only aspects of the intuitive conception that the measure fails to match. But we should not hastily take further mismatches as vitiating the philosophical interest of probabilistic models of quantitative confirmation."

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.