5

When two or more hypotheses can explain the same evidence, it's commonly affirmed that simpler hypotheses should be preferred over more complex ones.

For example, in response to my previous question How do we know (i.e. justify our belief) that time exists without "proving too much"?, this answer suggested:

Appealing to our subjective personal experience of time is perfectly fine, and doesn't prove too much (but you're only justified in believing the simplest explanation for the evidence).

Given our subjective experience, the existence of time is a simpler explanation than the non-existence of time.

Regarding experiences of ghosts, angels, demons, aliens, past lives and religious beings, those can all have a simpler explanation of hallucinations, dreams, misinterpretations of one's experiences, etc. As we gain a better understanding of reality, life, death, our bodies, our brains, the universe, our origins, beliefs of different cultures, etc., fabrications of our minds becomes a more simple explanation, while actual supernatural beings becomes less simple due to a lack of strong supporting evidence and how their existence might conflict or be difficult to explain with our current understanding of reality based on the available evidence.

Similarly, the Wikipedia article on Occam's razor states:

Occam's razor, Ockham's razor, or Ocham's razor (Latin: novacula Occami), also known as the principle of parsimony or the law of parsimony (Latin: lex parsimoniae), is the problem-solving principle that "entities should not be multiplied beyond necessity".[1][2] It is generally understood in the sense that with competing theories or explanations, the simpler one, for example a model with fewer parameters, is to be preferred. The idea is frequently attributed to English Franciscan friar William of Ockham (c.  1287–1347), a scholastic philosopher and theologian, although he never used these words. This philosophical razor advocates that when presented with competing hypotheses about the same prediction, one should select the solution with the fewest assumptions,[3] and that this is not meant to be a way of choosing between hypotheses that make different predictions.

Question

Can the "simplicity" of a hypothesis be objectively measured?

Is "simplicity" something that can be unambiguously defined and quantified?

For example, is the claim "given our subjective experience, the existence of time is a simpler explanation than the non-existence of time" objectively true?


Related questions

4
  • 2
    Occam's Razor (OR) is hardly correctly applied as it needs an ideal condition, ie, other things being equal and it's seldom the case when carefully comparing two theories with different number of ontic entities or hypothesis. Thus to decide the existence of time per OR you need to compare two theories with exactly same effect while one's language commits to time and the other not. Btw OR is often countered by Plato's famous principle of plenitude (PP) as reflected famously in Leibniz's possible=actual and Dirac's magnetic monads claims... Sep 10, 2022 at 5:16
  • William of Occam didn't say to choose the simplest hypothesis; he said not to postulate entities that you don't need to postulate. I don't know of any philosopher who has said that the simpler explanation is more likely to be true. Sep 10, 2022 at 5:32
  • 2
    Popper attempted to formulate a quantitative measure of the simplicity of a scientific hypothesis, but his criterion is considered to be flawed. There is quite a bit of material in the SEP article on Simplicity. plato.stanford.edu/entries/simplicity
    – Bumble
    Sep 10, 2022 at 5:46
  • 2
    There is no such thing as "the simplicity", so the question is moot. It is a loose term that refers to different things in different contexts (just like "similarity"). If you take Ockham's formulation, for example, then you can "objectively" count the number of "entities" (where that makes sense) and take hypotheses with their minimum as the "simplest". That would disfavor the multiverse interpretations of QM. But their proponents would say that they achieve "conceptual simplicity" instead, by removing the collapse postulate from the theory, which is also perfectly "objective".
    – Conifold
    Sep 10, 2022 at 6:33

4 Answers 4

7

Yes, see Solomonoff's theory of inductive inference. The idea is that you start with a formal language that lets you formally describe mutually exclusive hypotheses. And then we can measure simplicity by simply the length of the hypothesis, as a number of bits. Shorter hypotheses are simpler. Solomonoff's theory then uses this concept to form a Bayesian prior over the set of hypotheses, where shorter hypotheses are initially more likely.

See also minimum description length, which again measures simplicity of a machine learning model by its length in bits, and Kolmogorov complexity, which describes the simplicity or complexity of some piece of data based on how long a program (in bits) is required to produce it.

In applying these concepts to real-world scientific hypotheses, we need to be more careful. For one thing, a simple, short theory that, while not actually wrong about what it does say, is very vague, should not be preferred to a more complicated theory that precisely accounts for everything. "The height of Mount Everest is >8,000m" fits the observations, and is shorter than "The height of Mount Everest is 8,848.86m," but we should not use Occam's razor to prefer the first over the second because the first is also more vague than the second.

To properly apply Occam's razor we need to use it to select among hypotheses that are equally specific and mutually exclusive. In other words, each hypothesis we are comparing should give a correct, specific prediction of all of a certain set of observations. Then we can say that the shortest hypothesis among those - when described in some formal, logical language - is preferred, provided it gives the correct answers for each observation.

6
  • The linked references, which elaborate on this answer's thinking, suggest strongly that the first word of this answer should have been a "no". For instance, Kolmogorov showed that one can never know if one has compressed an algorithm to minimum bits, so we can't know which is shorter. The MDL link notes that MDL is non-computable. And the Solomonoff link notes that the process is computer language dependent (IE different answers in different languages), plus the method presupposes the universe is algorithmic, when the universe is not, because it is indeterminate, and pluralistic.
    – Dcleve
    Sep 12, 2022 at 14:07
  • @Dcleve No, what is uncomputable in all three references is the task of finding the simplest hypothesis. Counting the simplicity of a particular hypothesis, represented as a computer program - if you already know the hypothesis matches the data - is trivial, you just count the number of bits. It is true that this answer is computer language dependent, but I wouldn't necessarily say that makes it "subjective."
    – causative
    Sep 12, 2022 at 17:08
  • The task of finding the simplest hypothesis is hard because the halting problem makes it hard to know when a hypothesis is actually a valid hypothesis matching the data. You don't know whether a given hypothesis/computer program, that has not yet halted, might at some future point eventually halt and give the desired output that matches all the data. Counting the complexity of the hypothesis is the easy part.
    – causative
    Sep 12, 2022 at 17:12
  • Anyway, whether a program halts or not is uncomputable, but I wouldn't say that makes it subjective. If it actually does halt in N steps, that's certainly objective, because we can just watch it for N steps and see that it does halt. If it doesn't halt, we might not know, but still whether it halts is an attribute of the program, not the person watching.
    – causative
    Sep 12, 2022 at 17:54
  • Causitive -- if the answer as to which of two reasoning processes is "simpler" is dependent on the language chosen, then yes, it is subjective. Counting the "number of bits" is also not valid, as the Kolmogorov article made clear with examples where the Kolmogorov Complexity is smaller than its number of bits. Using the Kolmogorov complexity explicitly is impossible, as K cannot be derived. Therefore it requires making assumptions that one is "close enough" to deriving a Kolmogorov number. And then Solomonoff used Bayesian statistics, which are subjective in the choice of priors.
    – Dcleve
    Sep 14, 2022 at 10:08
2

Can the simplicity of a hypothesis be objectively measured?

I will assume that this only applies to rational thinking, and that the reasoning could be fully articulated in a formal way.

If so, we can observe that the same conclusion could be obtained logically from very different sets of premises. Potentially, there is no limit to the number of premises. However, it is likely, without attempting any formal demonstration, that there is such a thing as the minimum number of premises that are sufficient to make the conclusion necessary. This number, if we could compute it, would be an objective measure of the simplicity of the reasoning, or more to the point, of the assumptions necessary to prove the conclusion.

There may be a misunderstanding here, however. Magical thinking will always achieve the absolute minimum number of premises, that is, one. For example, how can we explain life? Easy, God did it! One premise, good job! However, we of course renounce magical thinking. We want rational thinking, and rational thinking is both logical and grounded on empirical data. And, then, this becomes immediately much more difficult to prove anything.

As I understand it, we could collect all empirical data and argue from there. It presumably would work but this is not terribly convincing. What we want instead is to find the minimum set of empirical data that can explain the maximum number of empirical data. That is, in the sort of diarrhoea of empirical data we barely manage to emerge from, we want to find which ones we can use to explain all the others.

Ah, yes, this is going to be hard, much harder than God did it, but it may well work and this is what we are interested in doing anyway. Scientists have already done really good work and this shows that there is something to it, but we are not finished yet.

You could see this as a way to put some order into the apparent chaos, a sort of late coming logos, but one which is what humans do anyway, and have probably tried to do since the beginning of humanity at least 300,000 ago. Or, it's just called "rationality".

2

You might try Juliet Floyd who I think has a deep analysis of simplicity and has a lovely quote on it:

"As we shall show in what follows, in philosophy, in mathematics, and in art there has been a repeated conceptual turn, a transition from simplicity to other idealizing notions: that of systematicity, that of rigor, and from here back to that of simplicity understood as common sense and shared understanding, virtues that overcome false rigor. Simplicity conceived in this way takes communicability to be a central feature, so it has a pragmatic flavor. One might think of it as a mere fiction. Yet, in the end, being indispensable, simplicity is an ideal that remains robust, repeatedly embodied, even while remaining part of an ongoing process reflecting our needs, desires, and discussions. (emphasis mine)

I will struggle to do her work justice. "We are arguing against the idea that simplicity is a simple ontological or a simple epistemological notion, and denying that it is formalizable." Yet she says we profited greatly from Wittgeinstein and Turing on how they conceived formal systems: "Turing explicitly held that the idea of a “cast-iron” notation, formalized “reason” unguided, is not only undemocratic and unscientific, but in the end a will o’ the wisp."

And again another beautiful and scholastically informed quote:

The point is not merely that simplicity is relative to a choice of system, and not absolute. Rather, as he [Witt] stressed from hundreds of angles in the mature writings, our needs and demands for simplicity are ubiquitous and unending. For we always require a first step when we analyze or voice a thought, we always require something simple, and we must learn to acknowledge that any such starting point is always taken from a particular place, one that we can share, break off from, pass off to the next person, reject, discuss, and contest. This perfectly echoes, and philosophically deepens, Turing’s mathematical analysis of logic.

In short simplicity is democratic, robust, fluid, and necessary. Necessary to formulate a hypothesis to then test, to start mathematical inquiry, to communicate. And as fluid as our uses of language which are endless according to Wittgenstein. Yet still analyzable (but not formalizeable) and heavily pragmatic.

Floyd, J. (2017). The Fluidity of Simplicity: Philosophy, Mathematics, Art. Mathematics, Culture, and the Arts, 153–175. doi:10.1007/978-3-319-53385-8_1

2

Short Answer

No. One of the other answers posted links to the last several decades of analytic efforts to do this "objectively" through calculation bit length, and Kolmogorov demonstrated that it is theoretically impossible to demonstrate that one has identified the minimum number of bits in a particular code, or language, or steps in a logic method, to do this comparison.

Longer Answer

The failure of Kolmogorov and Solomonoff is just the latest in a century and a half of rationalists failure to evade Kant's Critique of Pure Reason by attempting to do rationalism in more and more "precise" terms.

The first of the most notable of these efforts was by Frege, whose project was eventually undone by a math error. Russell and Whitehead attempted to redo Frege's project with corrected math, but their work failed to close. One of their students, Godel, discovered why -- leading to Godel's Incompleteness theorem.

More recent and related failures, specifically in the computation of inductive inference: Popper attempted to refute Kuhn's inference that paradigm shifts in science were JUST sociological happenstance by developing a quantification measure to show science was increasingly more "true" to reality -- and Popper's methodology was itself shown to be logically invalid.

Lakatos, with his Research Programmes and their progressively and regressivity, provides the best answer yet to Quine's observation that theories are always underdetermined by evidence. But Lakatos then wanted to quantify progressivity and regressivity of Research Programmes, and HIS method to do so also was shown to be logically invalid.

Also, there have been recent concerns that the "objective" frequentist statistics used by science until this century are ad hoc and can at times be misleading or "hacked". Bayesian statistics have been promoted as a logically more valid alternative, but Bayesian statistics rely upon a JUDGEMENT call by the statistician, in the selection of a "prior" probability for a hypothesis to be true. Different priors yield different answers. So Bayesian statistics are NOT "objective", subjectivity is intrinsic to them.

This is all just a reprise, over and over again, that one cannot establish anything about a contingent world, using anything which is "necessary", which logic is presumed to be in the Analytic approach. Kant pushed analytics into a secondary role in empiricism and characterizing this world, which is intrinsically a judgment call.

Note also, that logic is pluralist. One will get different answers to these questions, depending on which logic system one is presuming. https://www.cambridge.org/core/journals/think/article/guide-to-logical-pluralism-for-nonlogicians/EDFDFA1C9EB65DB71848DABD6B12D877 Hence, the appeal to a presumes "One True Logic" is itself a logic error.

Aside on whether simplicity is useful in empiricism

Our liking for simple worldviews is well established. Humans throughout history have latched on to universalized claims about "theories of everything", them dogmatically asserted them in the face of contradicting evidence. The non-religious have noted this is a characteristic of a religious mindset. Sociologists have noted that religious dogmatism is just a subset of the broader human trait of inclination toward ideological dogmatism, of all kinds of ideologies. That we have an intrinsic attraction for simplicity, and for ideologies, are things we should be SUSPICIOUS of, rather than presume are actually determining about the nature of our world.

The very simplest model of our universe is that of self-delusion. That only ourselves exist, and all other events and actors are really just "in our heads". This model will always win out in a "Occam's Razor" contest. Note, however, it involves a key assumption: DELUSION is asserted in order to dismiss data and observations.

Science and empiricism's response has been that only models that actually fit all the observations and data, rather than dismissing them as delusion, should be considered in an Occam contest. Note however, that like Popper's and Lakatos' efforts to formalize how to do empiricism -- THIS general rule is also not universalizable. We KNOW that we are sometimes deluded, so SOMETIMES, we WILL need to dismiss data. Science has to apply pragmatic standards of judgement (IE not objective) as to when to do so. A good pragmatic standard is that a delusion claimant has a very strong burden of "proof" (empiricism cannot provide "proof", so rename as "burden of justification") for any dismissal of data.

Contrary to the simplicity inclinations we have, what we have discovered about our universe, is that it is almost UNIMAGINABLY complex. Nobody today can even understand all of one field of science -- all scientists need to highly specialize within a general field before they can do useful work. Therefore, none of us can integrate all knowledge about our universe.

Plus, it appears to be non-integrable. Science has abandoned the globally reductive model, in favor of a combination of reductionism, wholism, and pluralism. See section 5 of this SEP entry: https://plato.stanford.edu/entries/scientific-reduction/ Plus scientism, the claim that science is the only source of knowledge about our world, has been rejected by scientists themselves. Reject global reduction and scientism, and it is then IMPOSSIBLE to build a logically coherent worldview. Multiple independent reference frames, if all valid, will of necessity lead to contradictory conclusions.

Additionally, even our most reductive science, physics, is non-calculable. Quantum Mechanics is indeterministic. As is Newtonian mechanics. See this answer: Deterministic or stochastic universe?

If our world is non-reductive, and non-deterministic, it is non-calculable, and this violates a starting assumption behind Solomonoff's thinking.

Science, therefore, cannot support logical coherence or simplicity. The best simplicity can offer is a USEFUL standard for evaluating science claims. And simplicity has proven to be so subject to rationalization (I have had a multiverse advocate claim to me that multiverses are simpler than a single universe), that Popper proposed a far more useful standard: more predictively powerful, where predictive power is measured by the potential for falsifiability.

You must log in to answer this question.