Does prediction really have epistemic value?

Question

I am having trouble understanding why the act of predicting something gives it any sort of value or makes a theory more likely to be true. If a scientific theory explained everything in hindsight, such as evolution, why would it matter even if it didn't make any new novel predictions? You could argue that you can come up with many theories that accommodate data but that isn't really true in the sense that not all theories explain everything. For example, theoretically, you can posit that some magical force caused all these life forms. But you can't actually see or verify that happening. It's not that it's a weak explanation because it accommodates data. It's a weak explanation because it doesn't actually explain anything,

Suppose I'm thinking of a number between 1 and 100 and I enter a restaurant. I check the table number and it's exactly the number I was thinking of. Suppose I find out in hindsight that the table number is randomly assigned between 1-100. I fail to see how I should be less surprised in this case compared to the scenario of me predicting a number between 1 - 100 beforehand and it being the same number in my head.

Answer 1

You can always find a model that explains the past, but is wrong (so fails to explain the future).

Prediction is such a strong indicator of correctness because it means the model still works on previously unseen data.

Answer 2

Does prediction really have epistemic value?

Anyone can make a prediction. I can predict it will rain today. Whether it does or not, the prediction has no epistemic value.

If I create a model for the weather, and it consistently provides accurate predictions for rain, then the science behind the model will have the real epistemic value. The predictions made by model will provide useful information but I wouldn't call it epistemic.

A broken clock will predict the time correctly twice a day.

Answer 3

But you can't actually see or verify that happening. It's not that it's a weak explanation because it accommodates data. It's a weak explanation because it doesn't actually explain anything,

And precisely THAT is what people mean when they talk about a prediction.

Like picture a coordinate system, your regular 2d one with an x- and a y-axis, and then add dots in it and pretend they are events. Now a theory is just any line(s) that connects these dots. And there are infinitely many lines that you could draw. Now you could say the x-axis is time and that a theory preserving cause and effect would need to pass these points without having more than one y-value for any x-value (just don't go straight up down or backwards). And you'd still have infinitely many candidates. Like just picture a wave pattern connecting them and make the peaks and valleys a parameter that you can vary between 0 and infinity and that should be obvious. And science, in that regard, is even "worse" because your theory doesn't even have to hit these points it's good enough if they are within a region of error around that line. So still infinitely many options.

The juicy part is that however you draw that line, by connecting or coming near to these points you would have also covered domains where there are no points, either before the first, after the last or in the spaces between them. And THAT is what we call a "prediction".

Because it prescribes a behavior for an event that we haven't seen or analyzed yet. That's what separates the theory from the raw data and which enables you to "verify" it (you don't) or at least to assess it's usefulness.

Also why would you be surprised if the restaurant had numbered tables? If you'd on the otherhand made a prediction beforehand and then find your prediction confirmed you might get the impression (justly or unjustly) that you are on to something.

Answer 4

True vs. Useful

Most theories are not strictly "true" or "false". Newton's theory of gravity depends on a view of spacetime that is generally agreed to be false (roughly true in everyday life but not exactly true). Einstein's general relativity is considered to be a better approximation, but may well still be "false" in light of how spacetime is structured at the Planck scale. But even if they are "false", we will continue to use these theories, because being "true" or "false" is different from being useful. We can never know that a scientific theory is completely true. But we know that both Newton's and Einstein's theories are useful because we can use them to understand what we see as well as to make accurate predictions.

Understanding vs. Prediction

Some theories are more about predictions, and others are more about understanding, but it is hard to be 100% one way or the other, because predictions are typically enabled by some kind of understanding, and an ability to predict also represents a form of understanding. The theory of evolution is largely about understanding why species so frequently appear to be "designed" for their niche, as well as why there are so many similarities between different species. Big bang theories are also mostly about understanding, not prediction, since we will never see another big bang. Of course these theories can be used to help make predictions, and correct predictions may increase our belief in them, but that is not their main contribution. Other theories, like the theory that smoking increases the chance of lung cancer, are focused much more on prediction than on understanding. For these theories, predictive power is in fact their main contribution.

Theories don't need to make predictions

A theory doesn't have to make predictions to be a good theory. If you see a broken egg on the ground, and then you see a nest in a tree directly above the egg, you will probably form the theory that the egg fell from the nest and broke. Most people would say this is a good theory, and there is no point trying to test the theory. Of course this theory can be used to make a prediction, for example that you are more likely to find someone who says "Yes, I saw the egg fall from the nest" than someone who says "Yes, I saw the egg ooze up out of the ground and then a nest materialized in the tree" but trying to confirm this prediction is a waste of time. In fact the theory is so good that even if you met a person making the latter statement, you would probably keep believing your original theory about the egg falling out of the nest, and you would form a new theory that the person you are talking to is pulling your leg or is crazy.

Similarly, the heliocentric theory of the Copernican revolution was a big intellectual advance, although it didn't make any different predictions. It was just a much simpler understanding, and a much simpler way of making the same predictions.

Predictions strongly influence our acceptance of theories

If one theory makes better predictions than another, then the one that makes better predictions is typically preferred (but simplicity of the theory is also a factor). For example, we could consider 3 theories:

• sunny days are more humid than cloudy days on average, because the sunlight warms the ground and accelerates evaporation of moisture on the ground
• cloudy days are more humid than sunny days on average, because the clouds, being formed of water, are a direct indication of humidity in the air
• whether a day is sunny or cloudy is unrelated to the humidity

To decide between these theories, you would probably start collecting data and see which theory fits the data the best. If the data is very close to what the third theory would predict, you may prefer the third theory because of its simplicity, even if the prediction is not absolutely perfect. If you talk to a meteorologist, you may decide that all of these theories are wrong, not because their predictions are wrong, but because the understanding they offer is woefully incomplete. Both predictions and understanding are important, when choosing a theory.

When theories give clearly different predictions, testing these predictions is a strong influence on what we believe. The theory of general relativity gives a prediction of the precession of Mercury's orbit over 20 times closer to the measured value than the prediction given by Newton's laws. This was a strong influence on people choosing to believe in general relativity over Newton's laws.

Similarly, big bang theory predictions such as the cosmic microwave background radiation have strongly influenced people towards believing the big bang theory, even though the purpose of the theory is not to go around making predictions like that, but rather to provide an understanding of how the universe began.

Answer 5

The final goal of mathematics is to predict the future, which is useful for survival (I predict where a cannonball falls, so, I defeat my enemy, or, I can calculate how much will this air-conditioning installation costs, then I can predict precise earnings).

The scientific method (repeating tests to validate an experiment) is exactly what validates a theory: each test implies predicting future result; if predictions are correct, the theory is considered valid for the context of the experiment.

For your example, the evolution theory is evidently based on the past, in order to predict the future, providing multiple correct predictions as for now (example, look for the word prediction; you can Google for many other proofs).

Answer 6

Evaluating the reliability of predictions forms a huge part of science, statistics, epistemology, striving towards truth and just daily life (whether you're consciously aware of it or not).

Let's say I come up with some prediction method, where I simply predict something to be as likely as it was in observed data.

Now I roll a die a whole bunch of times, record the results, predict results based on that, roll it a bunch more times, and check how well the prediction performs.

Now I can do the same for a different die or a coin or whatever else.

Or we could do some analysis of the results to see whether this predicted pattern would hold over different periods, when that period is excluded from the set that we determine the prediction from. (This could also form part of a bigger prediction method.)

Doing this when predicting different things would not just give us the accuracy for predicting those things, but would also give us an overall accuracy of our prediction method and might highlight some weakness of it (e.g. if something is constantly increasing, you don't just want to use the same frequency as you saw in past data).

With this, we can gain confidence in some prediction method actually being correct, and being able to predict the future accurately.

Beyond this, if we have an explanation for something that includes some predictions (e.g. evolution), evaluating the accuracy of those predictions could also give some confidence that the explanation is correct, and those predictions could tell us what we can expect in future (e.g. the evolution of viruses for virology and epidemiology), so we can prepare for what we expect to happen.

---

The act of predicting something doesn't really "make a theory more likely to be true", but it can help us figure out how likely it is to be true.

Answer 7

I think @haxor789 and @kutschkem give the basic answer, but to expand on some of your points:

Suppose I'm thinking of a number between 1 and 100 and I enter a restaurant. I check the table number and it's exactly the number I was thinking of. Suppose I find out in hindsight that the table number is randomly assigned between 1-100. I fail to see how I should be less surprised in this case compared to the scenario of me predicting a number between 1 - 100 beforehand and it being the same number in my head.

Here is the difference: you are always thinking about something, and observing things. Sometimes you're thinking about a number, sometimes you're seeing a number, sometimes the two will coincide. That's not the same as anticipating you'll see a specific number in a specific situation. The latter is a much more constrained situation that will happen a lot less often than the former. Simply because there are a million reasons you might think of the number "4", and, well, maybe not a million numbers you might run into for a restaurant table but at least like 10. And there are plenty of situations where you'll be thinking thoughts that involve the numbers 6, 3, 323, 42, 7, and then go into a restaurant with the table number "5" and not notice anything interesting about this sequence of events whatsoever. And then sometimes one of your number thoughts might involve a "4" and you'll see a street sign showing the number 4, and you'll notice. But you probably won't be amazed because on some level you'll recognize this as A Coincidence That Happens, for the abovementioned reason that we're always thinking, often those thoughts feature numbers, and we're also always perceiving and often those perceptions feature numbers and once in awhile the two will coincide (if this isn't true and you would be amazed I would like to exchange more because we have probably a lot more to talk about in terms of epistemology and probability).

On the other hand you'll very rarely have the thought "I bet the number of the table in the restaurant we're about to go into is 4". I would even hazard the guess that you've never had this thought before making this post or even reading the previous sentence, and failing that, that many people reading this post have never had this thought. I don't think I've ever had it before writing this paragraph, that's for sure. Why would I, it's such a weirdly specific thing to think. By the same token, going into a restaurant and the table number being 4 is a much rarer event than "running into some number in the world". As such, I think it's pretty clear that the combination of having this thought, and very shortly after having the corresponding experience, is really, really rare. Not so rare that it couldn't happen by coincidence, mind you - there are 8 billions of us having thousands of thoughts and experiences every day, and some of us are people who, unlike me, are inclined to try and make such coincidences happen by going to a lot of restaurants and routinely betting on their table number. But much rarer than the more generic "I thought of a number and shortly after saw that number somewhere somehow" situation for sure. (this kind of highlights a constraint on the epistemic value of prediction - it's not so much about predicting things per se, but about the probability of your prediction being correct if your theory is true vs if it isn't. For example if I look in my crystal ball and predict the sun will rise tomorrow, that's not evidence my crystal ball has an epistemically superior theory of the world because anybody could make that prediction, crystal ball or no. On the other hand if I correctly predict seven lottery numbers in a row that suggests I do have a superior epistemic link to whatever makes lottery numbers what they are, because those are supposed to be unpredictable by ordinary means).

But you can't actually see or verify that happening. It's not that it's a weak explanation because it accommodates data. It's a weak explanation because it doesn't actually explain anything,

I would challenge you to explain what "explain" means in that sentence. "Predict" has the benefit of being pretty simple to test - if you say a thing will happen in situation X you don't have prior knowledge of, and it happens or doesn't happen, you succeeded or failed to predict it. How strong the epistemic value of that prediction is can then be debated, but the fact you made a prediction is much easier to narrow down. "Explain", now... what does that mean? It's not "see or verify that happening", that's for sure; there are plenty of processes we think of as "explained" by science or just everyday reasoning but that didn't involve "seeing anything happening" (if only because the explanations involve past processes) and whose "verification" relied entirely on predictions.

I would argue that "explanation" is actually a very complicated concept that has subjective elements - that in part it involves people feeling like an explanation is satisfactory, and that this feeling is not entirely reliable in terms of matching up to what other people feel is satisfactory or other, more "objective" assessments of an explanation such as its predictive value. So I'd want any claim that something "explains" a phenomenon better or worse than something else to give more detail as to what makes the difference. Personally I always end up coming back to predictions when I try.

Answer 8

Humans don't like surprises. Most surprises that make a difference are not birthday parties or checks in the mail. They are volcanoes erupting, pouncing carnivores, financial crashes, and spouses who run off with yoga teachers. We thus create categorically an environment of "common sense," ordering our senses such that there is a high probability that the future will resemble the past.

Predictions thus have limited value in proportion to their certainty. The sun will rise. People will run out of a burning building. A thrown rock will travel some roughly known distance and fall to the ground. Excellent predictions, but with little value in the reduction of uncertainty. Trivial, we say. (Though it is well to remain conscious that these are only highly probable inductions and the natural habitat of "black swans.")

Theories that have value make unexpected predictions. In Shannon information theory, one way to measure or even define "information" is by how improbable or surprising it is. This is not prediction, for the confirming event or news comes first. How "informative" it is is the quantity of "surprise" it removes.

My name on an envelope in my mail box provides little of what we call "information." Nor does the coincidence that all the letter in my mailbox bear these exact same pronouns. The curt letter inside the envelope telling me of the yoga elopement contains even fewer words, perhaps, but much more "information." It is "surprising" and adds far more to my fund of knowledge and quite probably "makes a difference."

The value of a scientific theory is the amount of knowledge or certainty it brings into human affairs, which is the same as the quantity of surprise it removes. The theory that is very surprising and, like general relativity, triumphs in confirming a highly improbable prediction has removed a large slab of uncertainty, subject to repetition or "experimental duplication."

It can now become mere common sense, altering our environment of probabilities, advancing the frontiers of "causality" or surprise-free operations, from which even more unlikely predictions can be launched, steadily draining the universe of flux, shock, bumps in the night. I am not sure that I understand your table number prediction. But we usually find a temptation to cheat even ourselves when it comes verifying uncommunicated predictions "in our heads," and our exaggerated sense of clairvoyance can lead us into very bad surprises.

Answer 9

The temporal veil of ignorance between the present and the future gives true predictions their epistemic value. It's an undeniable fact that nobody really knows what's gonna happen the next instant let alone what might transpire tomorrow, a few days on, etc. To predict and to be right means the temporal veil of ignorance has been penetrated.

For science which adopts a generally deterministic stance, predictions are routine, part of the landscape, for theories that have made the cut like general relativity.

Answer 10

Lot's of good, intuitive answers here. I'm going to take things a little more formally.

Epistemic force is a term you might hear regarding assertions. It essentially means that some assertions have bearing on what we know. Your question is 'how precisely does a prediction have epistemic value?' You then give an example where you exemplify your contention that it seems not with a context about a guess. This is a good question, and is related to the scandal of induction which asks, how can inductive arguments have any epistemic value if they fundamentally can be wrong?

First, let's examine your example. It's a bit of a simple notion of prediction.

A prediction is usually more than a guess as you present it. It's often an assertion that comes from a theoretical framework in practice. For instance, mathematical conjectures aren't "guess a number to 10". And scientific hypotheses aren't "I think it might be blue over there". They are often quite involuted, theory-laden statements that can be difficult to understand if you're not trained in mathematics or the physical sciences. For the former, consider the continuum hypothesis. It's a prediction. For the latter, consider the somatic marker hypothesis. Would you feel confident assessing either hypothesis if, given the judgement of experts, you could loose your car or house? I wouldn't. The predictions themselves may take years to understand, and in the pursuit of understanding, one is forced to sort out one's presumptions. Thus, the epistemic value of the prediction is examining the theory-ladenness of one's observations. In fact, without prediction, there is no problem in science at all.

I cannot stress how, in philosophical circles, this was a shock to some of the greatest thinkers who were sure science could be brought inline with hypothetical-deductivism. Thus, just the philosophical examination of what a prediction helped to end the overconfidence of some scientists in science itself. The movement, since Decartes and rationalism, has fallen firmly in the favor of moderate empiricists, and is known in philosophy as fallibilisim (IEP).

Another aspect of prediction is that it functions in developing scientific explanation. Predictions aren't just checked. They often involve social activity to construct apparatuses to determine what is real. So, predictions motivate the construction of knowledge in a general sense. That also takes on the role when there are competing hypotheses or interpretations of experiments in helping create objective points of agreement in discourse. So, if knowledge is constructed socially, that is, it's a social venture which has social factors, then one of the factors driving the development of knowledge is prediction, because predictions make arguments objective, and thus subject to public access. There are no private languages according to Wittgenstein.

You talked about a guess coinciding with a random selection. But this example is an occurrence of prediction. What makes prediction epistemically valuable is that guessing a number from 1-100 is coincidental in the case of a match, but coming to understand a frequentist's notion of probability (SEP) entailed the invention of a formal theory. More specifically, predictions can be evaluated to create theories which discuss dispositions of things Now, that theory can be used to talk probabilistically about phenomena. I won't tell you how much money casino's make off of the mathematically illiterate, but their predictions about peoples' predictions in gambling is easily in the billions of dollars of profit. That's why gambling is often referred to as a tax on the mathematically illiterate.

So, the overarching point here, and just one aspect to demonstrate the origins of the epistemic force of prediction, is that the instance of being right or wrong about something is not the same as a sustained effort to build models to generate predictions. The former is epistemically trivial, and the latter may be at the heart of the accumulation of all human knowledge ever. The former is a child guessing a number, and the latter is a multi-generation project to develop pragmatic knowledge about the world. And at the heart of that construction of knowledge, which is historically seen as justified and true belief. Thus, without prediction, most of what constitutes as social knowledge that you have inherited, not excluding language itself, revolves around the act and art of prediction.