When given limited information, is the simplest solution that matches that information most likely correct?

Question

Is there any basis in philosophy for the idea that when given limited information, the simplest solution that matches that information should be presumed correct or most likely to be correct?

For example, a complete search is conducted in the space of algorithms that generate series of numbers. Take the simplest algorithm thats number sequence starts "1, 1, 2, 3, 5, 8" and then goes on for at least three more elements.

I'd like the seqence to continue "13, 21, 34" etc... However, it might instead continue in some other equally valid and simpler direction that hadn't previously occurred to me. It might even seem to me to be wrong.

How often am I going to be disappointed?

Answer 1

The question you ask refers to what is known as Occam's Razor or the law of Parsimony (esp. psychology). It is not an unwavering principle but more of a general heuristic or "rule of thumb". Various formulations of this idea have been offered since ancient times and various justifications have been used to support it, despite the fact that it (historically) appears to never have been intended to be a proof but rather a way of simply avoiding needlessly overloaded explanations.

As Michael pointed out in his comment, Occam's Razor is about entities (concepts/assumptions) in an explanation, such that—all else being equal—the explanation for a phenomenon which poses the least new concepts/assumptions should be favored.

Consider this image:

You might wonder, why is the metal orange?
When examining the potential explanations for this, we might come up with the following reasons:

1. someone painted it orange
2. it is coursing with electricity, causing the molecular structure of this metal to excite and emit an orange glow
3. an evil daemon is causing an illusion, tricking us into thinking it is orange

All these possible explanations could be true. But—as with most situations—one of the explanations is more likely to be true than the others. This is where Occam's Razor comes in.

In a superficial sense, the first answer seems the simplest but in principle it is just as simple as the second explanation, as neither explanations propose radical assumptions (that is, presuming is it relatively equally possible for someone to have painted it and that metal to be of the type that glows). The third explanation—while theoretically possible—is distinct from the first two because with it we have the burden of proving that an evil daemon exists, that he has the power to give us illusions, and that he has the desire to do so. These are additional entities, or assumptions, within this explanation. Occam's Razor would tell us to favor the other two explanations over this one, all else being equal.

But it doesn't have to always be about outlandish new assumptions; in principle even trivial assumptions could be examined with this principle. Between answers 1 and 2, the outcome of applying Occam's Razor to these explanations would depend on the particular situation. I.E., is it practically possible that someone could've painted the stove while you weren't looking? Is it April Fool's Day? Is one of your roommates/family members a good artist who likes to pull pranks? etc.

Answer 2

How often am I going to be disappointed?

That depends on what degree of rigor you are aiming for.

Wittgenstein famously demonstrated that there is no 100% reliable way to intuit a rule from a pattern, or to explain what it means even "to follow a rule"; this means that we are often working in the realm of "best guesses" rather than "absolute certainty." This is generally not a problem in our daily lives, and doesn't necessarily need to be a problem in our philosophy, unless we are insisting upon a degree of certainty that remains (necessarily) beyond our grasp-- if this is the case, you will always be disappointed.

Answer 3

There is a curious pair of results in classical, first order logic known as the Lowenheim-Skolem theorem and the Compactness theorem (see section 5 of that article for details). Together, these results tell us that our standard logical reasoning actually isn't strong enough to let us tell the difference between incredibly large infinite models of a given theory and merely countably large infinite models of the theory.

Abstractly, we might conclude that since our logic allows that countable systems are as useful as uncountable ones, we can make do with the countable models and ignore the unneeded complexities that these larger models introduce. This seems to licence a form of Occam's Razor as concerns redundant information in an infinite universe.

However, this argument hinges on whether we accept that conventional first order logic is the right background framework for scientific model building. This position was frequently defended by Quine, and features heavily in debates about the idea of Realism in science and metaphysics - Classical logic maintains axioms of both non-contradiction and the excluded middle, such that every potential proposition is exactly one of either of two truth values (True and False). Again, some interesting discussion can be found in the SEP article on the relationship between Logic and Ontology.

Answer 4

There's lots of stuff on the use of simplicity as a virtue in science. Are simple theories likely to be correct? Why would that be?

My take is that things like simplicity might not necessarily be truth conducive, strictly speaking, but there are still good methodological reasons for opting for them. Leaving aside Occam's razor, which has been covered, consider the following: simple theories are easier to falsify that complicated theories. So, if I stick to simple theories, I will falsify false theories quicker, so I will progress faster. I think Ernan McMullin kind of says this, but I can check references if anyone's interested.

Answer 5

Michael Dorfman's comment on Stoicfury's answer is important, indeed for me it is the crux of the question, so I thought I'd expand on it.

'Simplicity', when applied to a set of rules, is an inherently meaningless concept. One might ask for example which is simpler out of the integers and a general ring- the former requires more axioms than the latter, but arises more naturally than the class of all rings: to say one is simpler, more likely a priori, would be wrong.

Occam's Razor works (at least, can be seen to work) because it is a statement in conditional probability: each additional entity X we posit to be at work adds another factor P(X¦ All the other entities posited so far) to our probability, each of which must be less than or equal 1. From another point of view, an additional entity occurs in a subset of instances, with lower of equal probability (by the monotonicity of probability measure [if A is a subset of B, P(A) < or= P(B)]).

Our problem comes from the fact that purely abstract entities cannot be ostensively defined- 'twoness' for example is a property that varies in its sense (if I break something in half, is that one or two?) and so their instances are not well-defined, making subsethood (and hence a priori relative likelihood) an impossibility.

We can rescue some semblance of a notion of likelihood, though, by positing the existence of entities which can be placed in ostensive correspondence with our abstractions: concrete mechanisms, for example, that produce the fibonacci sequence (or, more mundane, collections of objects that combine like the natural numbers under addition). And although it is somewhat surprising that such entities should exist, they most certainly seem to.

The OP cites a truncated fibonacci sequence and asks 'is the next likely to be fibonacci?'-naturally this depends on what our sequence is of (dividing cells? number of hats by size?) and the complexity of the mechanism he could propose by which fibonaccis should be produced.

Answer 6

I am not a very knowledgeable person nor am I a very good philosopher. However, I am able to think logically. Based on given information, one can conclude certain things or speculate different aspects. When one is faced with limited knowledge/information, the most logical thing to do is to extrapolate the existing data to match possible results. Please note that this will not ensure that the conclusions are correct; this just gives you a range of possibilities that are more likely to be correct than other answers. Unexpected answers to simple questions may not be common, yet they do exist.

Furthermore, when generally stating something as drastic as "when given limited information the simplest solution that matches that information is most likely correct", you must back your point up with proof. In certain situations, your statement may prove to be correct however, in many cases, the integrity/truth of such a statement can be questioned. A very important principle in philosophy that must be noted is that the lack of knowledge does not justify the truth of another. For example, one cannot argue to prove the existence of God by saying: "Well, since we don't know how the first atom was created, then God must have created it." One must approach such argument from a different direction.

Back to your question specifically about the sequence, I see your logic behind what you deem to be the pattern/algorithm of the sequence and I agree because I do not see other possibilities. Although I may be completely wrong, It seems to me that the simplest answer (in this case) will most likely be correct. However, this is case sensitive and in relation to other situations, this theory may not necessarily hold true.