7

At some level, it seems more intuitive to think that simpler theories are better than complex theories. Is this an aesthetic? Or is it logical given the fact that each assumption introduces a new possibility of a theory that could be wrong?

16
  • 5
    Newtonian gravity is simpler than general relativity, yet the latter is definitely better by allowing all the explanations and predictions the former can, plus a bunch of others. What do you mean by "intuitive" ?
    – armand
    May 21, 2023 at 12:09
  • 1
    Survey the philosophical landscape and discover fer yerself ... the truth. By the way, along the way, over the span of the last 2 millennia, our priorities have beem reshuffled many times over. Sigh. May 21, 2023 at 12:11
  • 4
    Simplicjty is a "need" of human mind; no reason to believe that is a feature of the world. May 21, 2023 at 14:08
  • 1
    @MauroALLEGRANZA Couldn't agree more. Nature does not owe us to be "simple". Not even to be comprehensible, no matter how much we wish it.
    – Frank
    May 21, 2023 at 15:55
  • 1
    You might like Juliet Floyd who talks about simplicity via scholarship with many important philosophers and mathematicians. Simplicity is a first step, fluid, and homespun. "Logic’s certainty depends upon the surveyability [Überblickbarkeit] of these symbols in all their parts (simplicity)"mdpi.com/2409-9287/8/1/6. Simplicity plays more of a "starting role to analysis" than the ones you bring up, accordingly. In my humble opinion about not so humble things, that is the common thread to simplicity, homespun first steps, not truth or aesthetic appeal
    – J Kusin
    May 21, 2023 at 21:10

7 Answers 7

13

There apparently was some aesthetic component to it. But to a degree it also neatly exemplifies the general goal of science as a compression/decompression algorithm.

The thing is complexity is difficult. It's hard to grasp, understand, conceptualize, make use of, explain and communicate and pass on. If you went outside and took a picture of the world the description of that picture in all it's glory might take you more than a lifetime, regardless of how meaningless or meaningful the subject matter actually was. If you were to describe the path, velocity, angle of incidence, energy, ... of every photon hitting the receptors of your camera you'd need loads of time and/or super computers. And no matter how detailed your description there's probably still something lacking.

So in order to have a chance to share information and to make use of them, we need to reduce the complexity to a manageable amount.

That means we can ignore information, like humans are only able to see light between 400-700 nanometers, so only that range is relevant information. Or it could mean to abstract, conceptualize and categorize information. Maybe the image shows people, structures and trees. So you could abstract that in foreground and background, living things and objects, nature and artificial structures and so billions of pixels become 1 tree and tones of trees become 1 forest.

And the latter part is the business that science is in, we like to reduce the complexity by finding patterns and creating models around them so that we can can comprehend and express something complex, while actually dealing with it as something simple.

So we want to lose as much complexity as possible while losing as little information as possible. So the ideal is a lossless compression. Take something complex, reduce it to something simple, understand and modify the simple thing and then decompress it to something complex again.

That is not about the truth, as the truth likely is complex and we're losing not just complexity, but also information with our models, but we kinda have to in order to actually make use of what we see. So our hope is that reality is actually based on simple patterns so that this quest has a chance of success in the first place, but even if it isn't it's still a useful approach as long as there is any such pattern.

Also sure the fewer variables and assumptions and the better you understand your models the easier the debugging process and if you're lucky you can keep the simplicity in some parts of the model while only being forced to locally adapt to complexity.

3
  • 1
    "You can't handle the truth!" So true.
    – Scott Rowe
    May 22, 2023 at 13:23
  • 1
    @ScottRowe Yeah kinda I wouldn't even be able to confirm it if it were right in front of me ;)
    – haxor789
    May 22, 2023 at 13:24
  • +1 The way I tend to explain this is that science is not about why something is a certain way, science is about creating tools to predict and thus affect the future. Apples don't fall because of gravity or space time curvatures, gravity and space time curvatures are tools to help describe the behavior of apples. (As a developer though I love your comparison of science with compression/decompression algorithms, got a real life chuckle out of me :) ) May 24, 2023 at 15:33
8

You need to consider the role of overfitting. You have a set of data points, and you want to understand what variables are generating their pattern. There may be fundamental reasons to expect say a linear or exponential relationship. But essentially the complexity of the curve, relates to the number of variables, and given enough variables you can generate the semblence of a pattern that isn't there, the mathematical equivalent of apophenia, like seeing faces in clouds.

A good example is the 'String Theory landscape', where the complexity of the theory allows unlimited overfitting, with no way of establishing constraints or limits, that we know of. This has been why String Theory has fallen out of favour, it can accomodate any data, yet not predict anything we don't see.

So simplicity is not just an aesthetic, it relates to finding true relationships between variables. We can compare it to consilience: if our different senses give conflicting evidence, in general we doubt the complex picture that somehow they are all correct, and jump to the reasonable assumption there is an illusion or misconception involved. Similarly, we use double-blind controlled trials, to check if people see the same results regardless of expectations or preconceptions.     

2
  • String Theory as the modern version of geocentric solar system... Seeing things that aren't there. Interesting.
    – Scott Rowe
    May 22, 2023 at 11:47
  • @ScottRowe: "Pick a rest-frame, any rest-frame.." Einstein does elaborate shuffles of his deck of coordinate systems then proffers the infinite set of them If you listen to the people that did the work, like Susskind, you get a very different persective than the pop-sci picture of String Theory.
    – CriglCragl
    May 22, 2023 at 18:33
6

I will narrow down my answer to theories in modern physics, acknowledging at the same time that there are many other places where "theories" can be found.

In the case of theories in modern physics, as has been said in other answers, simpler theories are easier to handle than more complicated ones. If you can formulate a linear differential equation, you have a much better chance at solving it than if you start with non-linear differential equation. You can always make things more complicated with various corrections if the linear model is not an accurate enough approximation. But even if the starting point is not a linear equation, physics loves to tack on corrections (for example, to calculate cross-sections in particle physics).

The point though, is that rather than simplicity, there is a notion of finding the right complexity of the model, the complexity that explains as many observations as possible.

That being said, "simplicity" is also a bit subjective and in the eye of the beholder. Is quantum mechanics "simple"? The mathematics underpinning it are anything but simple or intuitive. They involve Hilbert spaces over complex numbers - a far cry from the familiar 3D space that is the background of elementary Newtonian mechanics. But it works pretty well. It seems some sort of simplicity might be desirable in physics, but not the ultimate criterion that will decide if a theory should be retained or not: the criterion for that will be whether the theory adequately explains the observations so far.

1
  • +1 Right complexity is a measure of adequacy.
    – J D
    May 23, 2023 at 16:55
2

Consider model theory. If a theory has relatively small Kolmogorov complexity, then it will tend to have few axioms with few nuances, and thus will have lots of models. As the theory becomes more complex, the breadth of models shrinks. Note that when the theory is a logic, then the models are categories; this connection is called categorical logic.

For a worked useful example, it is recently known that the axioms of quantum mechanics are only satisfied by a finite handful of models; see Axioms for the category of Hilbert spaces for discussion and proof.

I would caution against worrying about truth. A model implements a theory; it is not true or false.

1
  • 2
    +1 Theories are adequate, not true.
    – J D
    May 23, 2023 at 16:54
2

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

-- Antoine de Saint-Exupéry, Airman's Odyssey (source, one of many)

Occam's razor is all about removing unnecessary factors from our understanding of how things work in the real world.

Consider the geocentric model of the solar system. Assuming Earth to be the center of the solar system, it's easy enough to plot the course of the moon, planets, and stars through the night sky. Having those plots, it's easy enough to predict where they'll be in the future. But, figuring out why the heavenly bodies moved as they did required adding retrograde motion to the planets: something caused the planets to orbit "backwards" for a few days every year. Planets are heavy, so changing the course of a planet would require a tremendous amount of energy (doing so that quickly without radically changing the planet's appearance would be pretty tricky, too).

Heliocentrism is computationally equivalent to geocentrism (at least, according to Wikipedia: "Ptolemy devised a system that was compatible with Aristotelian philosophy and managed to track actual observations and predict future movement mostly to within the limits of the next 1000 years of observations"): both models proved (roughly) equally competent at explaining observed phenomena and predicting future events. But, heliocentrism was simpler by viewing retrograde motion as a consequence of the relative motion of Earth and the other planets rather than the result of some force acting upon those planets.

Any hypothesis, theory, or hunch can have any number of bits bolted on to the side: special cases that require special consideration. Occam's Razor tells us that the accretion of special cases means that we've probably got the model wrong. We might not! The real world, as a comment says, doesn't owe us simplicity: it's entirely possible that there are natural laws/processes that have a whole slew of special cases. But: a more fundamental law that allows those special cases to emerge from the math is preferable to one that requires finding the exceptions experimentally.

At some level, it seems more intuitive to think that simpler theories are better than complex theories. Is this an aesthetic? Or is it logical given the fact that each assumption introduces a new possibility of a theory that could be wrong?

Remember that Occam's Razor doesn't talk about "simplicity" per-se; rather, it says:

Entities must not be multiplied beyond necessity.

That is: "when presented with competing hypotheses about the same prediction, one should prefer the one that requires the fewest assumptions" (wikipedia). Heliocentrism is preferred over geocentrism by Occam's because the former remove the assumption that there's some mysterious force that changes the direction of the planets' orbits twice a year (but doesn't affect anything else).

Which is to say "it's both, and". There is definitely an aesthetic preference for simplicity, and it's also the case that extra entities (assumptions) provide more places for the model to be wrong. But, it's also a reminder that those extra entities, assumptions, complexities, special cases - all of those things together are hinting that the model is missing something more fundamental.

It's like the old saw about how correlation doesn't imply causation, but it does waggle its eyebrows and gesture in its general direction. When entities accrete on a model, that's Occam's waggling its eyebrows and gesturing towards a better and/or more fundamental explanation for what's going on.

0

If p_n is the total probability of all the theories of complexity n , then the sum over all n of p_n is 1. Since there are an infinite number of n, for this series to converge the finite value of 1, it must eventually be decreasing. There are 2^n theories with complexity n (where complexity is measured in bits), which means that the average probability of each one is p_n/2^n, or (p_n)(2^(-n)). If p_n is decreasing, then (p_n)(2^(-n)) is decreasing even more quickly, which means that more complex theories are less likely than less complex ones. This isn't an ironclad argument, but it's a strong intuitive argument; one can still argue that a particular complex theory is likely, but it would hard to make that argument without some reason that that theory is "special" somehow.

0

The simplicity of a theory is an aesthetic, to be sure, but it comes about through the honing of reason, like a diamond is produced by the continual pressure of rock.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .