Does the want to seek determinism in physics come from a fallacy that it explains more?

Question

Many physicists even in the modern day tend to be bothered by the indeterminism within quantum mechanics. When they see the probabilistic patterns that lie at the heart of the world, they seek a deterministic cause that would get rid of those probabilities. Many physicists also seek a theory of everything, a simpler all encompassing theoretical framework that can explain everything.

Imagine that the world is in current state A. And its next state can be B, C, or D with a probability of 1/3. If the next state ends up being B but without a sufficient cause, then the probability of it arriving at B is 1/3. However, if there is some sort of deterministic cause that leads A to B, the probability of B now becomes 1. In other words, one can say that C and D never really were possible in the first place.

In this sense, determinism seems to give an explanatory advantage in that it gives us a reason as to why a certain state happens. B doesn’t now just occur as a mysterious brute fact: it is now determined.

I wonder then if this is a mistake. If there is a law that maps A to B, one can also further ask: why is there a law that maps A to B rather than A to C? Or even A to D? Presumably, there is no further reason for this. We are now again left at a brute fact as to why a certain law exists instead of another.

In other words, there doesn’t seem to be an inherent advantage in explanatory power with determinism. The only case in which we can make explanatory progress is if we see actual evidence for a cause. Without this, there seems to be no reason to think that there is determinism underlying the indeterminism that we see. Why then is the instinct still there among physicists?

EDIT: Since some are asking for clarity, I am not saying that seeking a deterministic explanation is useless since it will still bring a further “why”. I’m saying that postulating a deterministic explanation only because it would supposedly solve a problem is a fallacy, since postulating a new explanation without evidence is no better than no explanation. One needs evidence to postulate determinism behind everything that is going on and so far there hasn’t been any.

Answer 1

You seem to have it backwards with regards to what science (in particular physics) is doing. Where mathematics, logic and to some extend religion build a universe from the ground up, so where they start from a ground truth (or a whole set of them) which are then just assumed/believed (axioms) and then mixed and matched into something complex. Science is more or less taking the opposite route.

In science you look at a world that is incomprehensibly complex and first of all make observations and measurements and then try to find patterns in them and then describe those patterns. So the reason why force is equal to mass times acceleration is because that's what we observe. You observe that for example an otherwise resting object is being attracted/repelled (by e.g. gravity) and that this attraction (force) is proportional to the mass of the object and that if you look at the ratio of force and mass you end up with a constant.

So physical formulas are essentially the mathematical form of story telling. You have a bunch of facts and key events and you weave them into a coherent narrative. The beauty is that the result is short, easily memorable and comprehensible (unlike the chaos that are your real life data points). The downside is that it is wrong.

So you could call these stories an answer to the "why" question, but as you've realized yourself they technically aren't, because they don't actually describe "why" something is happening and quite frankly don't even bother to do so, they just describe a model that would produce similar results.

Also what does it mean that they are WRONG??? Well the proportionality of mass and force and the equal ratio isn't exactly like what you expect in mathematics where F_1/m_1 = F_2/m_2 = F_3/m_3 = ... F_N/m_N and so on, but in reality it's closer to idk having a set of measurments like [10.2, 9.7, 9.1 , 9.0, ...] where you end up with ~9.5 kg*m/s². So rather than dealing with inevitable certainties of calculations you more or less eyeball a curve that fits your data points. I mean nowadays you no longer literally eyeball it you could just take a simple template of a formula, idk a polynomial (a+bx+cx²+dx³+ ...) let the computer guess the parameters and measure the difference between the curve and the real world data, rinse and repeat until the error is small enough for your liking. So as a result, unless axiomatically defined, scientific constants are NEVER just a value, they ALWAYS include a margin of error (usually something like the average discrepancy between the idealized curve and the real world data).

Now after several iterations through different narratives, improvements in measurements and so on, we end up with something that is quite useful in our everyday life.

And another feature of these models is that they don't just summarize the collected data points, but due to being continuous and not limited with respect to their parameters you could also predict what the results should be for values that are NOT in our data set.

Which enables us to look into the future (timetables, weather predictions, etc) or to estimate how things will behave that we haven't tested yet. where it's often times these situations of the theory breaking (large (enough) discrepancies between prediction and reality) which reveal something interesting about the world and enables us to improve our estimates. Again with many asterisks, like it's a pretty decent approximation to say the earth is flat (if you're not traveling far). While if you traveled in all directions and find out that you're back to where you started from you might realize that there's something wrong with this idea. So they might beat random guesses what comes next, but the further you deviate from the data that the model is based on the more likely you'll find something "interesting".

Now as a result of all these things you might accidentally still embark on a quest to find the "why". Because in your attempt to build models of phenomena and maybe even meta models of the models you've build, you might produce chains of cause and effect which might already suffice the quest for knowledge of some. But they ultimately are deterministic, they give no answer for a reason or purpose, things happen because they happen and if our model could describe all the things happening that would be sufficient.

Also worth noting, that doesn't need to be a feature of reality, it's more of a feature of our models of reality, because with regards to compressing data points and making predictions, it's quite counter productive if you'd end up with something unpredictable and indeterministic. Like sure a religion might be fine with having things be the result of the agency of a god, but if you want to describe how things are happening, this would mark a brick wall that you'd crash into.

So in that regard indeterminism just isn't that nice for this method of epistemology. Like not being able to predict the outcome makes it kinda tricky to derive a formula for the outcome. Like sure on the macro scale, you can deal with that because while the outcome of an individual experiment is unpredictable, the outcome of millions of them is predictable.

Also with regards to the world formula. That's a tricky thing. Like on the one hand if you had this one effect that is underlying all the other effects and makes all the rest just a consequence of that one thing, would certainly be a major accomplishment in the quest of creating an understandable universe. The problem is, have you actually reduced the complexity of the universe or is that formula still as complex as the universe used to be? Also how good does it perform? Like it's technically not hard to create a world formula e.g. 42. Though while it might be working in some domains, quiz questions concerning the Hitchhiker's Guide to the Galaxy, it might be considerably less useful in most other domains... So it's not enough to make up a theory of everything, it also needs to be simple enough, have a low margin of error and ideally have a somewhat uniform margin of error. Like it is pretty awful when the theory of everything works for close to everyone but you and you're just discarded as "within the margin of error".

I mean that's where the debate about pseudoscience usually starts with, that you end up discarding real world evidence in favor of the theory rather than the other way around.

So TL;DR that instinct stems from the fact that physicists are not really in the business of "why" but in the business of "how".

Answer 2

Physics isn't practiced just for the sake of it- much of what gets discovered in physics ends up being put to use for all kinds of purposes. So, the question is whether the 'explanations' developed by physicists have some benefit, and I can imagine that if, for example, physicists could identify some underlying mechanisms that could better predict why certain quantum mechanical interactions happen, then that could lead to all kinds of useful applications. Ultimately, whatever laws of nature you identify, you are still left with a question of why they are as they are. And if you can answer that, you have another question- why is the answer what it is? And so on indefinitely. So yes, you are deluded if you think that clarifying some uncertainty is ever going to explain everything, but the fact that you cannot explain everything does not mean that it is pointless to explain more things.

Answer 3

I wonder then if this is a mistake. If there is a law that maps A to B, one can also further ask: why is there a law that maps A to B rather than A to C? Or even A to D? Presumably, there is no further reason for this. We are now again left at a brute fact as to why a certain law exists instead of another.

This is like the child who keeps asking their parent "Why?" questions, until the parent gives up and says something like "That's just the way it is" or "Because I said so."

People have a natural tendency to view the world in terms of cause and effect, and understanding how and why things happens allows them to organize phenomena in their minds and make more general predictions. This then allows more effective manipulation of the outside world, from which we get technology.

Many scientists are satisfied to do this without knowing why. As long as we can form useful models and mathematical formulas that allow us to make predictions, it's good enough to get things done. Newton didn't know why objects attracted each other when he determined his formula for gravitation, yet this allowed the calculation of orbits of planets. This is an engineer's view of physics -- we only need to know enough to be able to make useful things.

But scientists are people, and many of them find this unsatisfying. Merely cataloguing all the formulas that describe the universe is drudgery. Because we see things in terms of cause and effect, there's some expectation that there's an ultimate, simple explanation for everythng. This follows from the fact that over time we've been able to provide more and more detailed explanations for what we see. For instance, we discovered that objects are made of molecules, molecules of atoms, atoms of protons/neutrons/electrons, and protons and neutrons are made of quarks; we've further been able to determine the properties of these particles that explains how they work together to form the objects we see in the macroscopic world.

I'm not a philosopher, but I believe the technical term for this is "reductionism".

Yes, like the child's questions, this can go on indefinitely. But the hope is that we can get to a point where there's a description of an ultimate cause that is sufficiently obvious that it can be used to explain everything else. We hope we can end with a reasonably satisfying "That's the way it is" rather than a meaningless "Because I said so" -- the latter seems like giving up (theists may be OK with "Because God said so", but that still leaves us with an unexplained God).

Answer 4

The greatest benefit of living in deterministic Universe is not that we can explain what already happened, but that we can predict what will happen. In particular, this gives us agency by allowing us to predict the outcomes of our actions.¹

Luckily, our Universe appears to be deterministic on macro level, making it possible for us to make plans and accomplish things.

¹ Whenever we talk about freedom, it pays to remember that there are two kinds — freedom to act, and freedom to get things done. Without determinism, we would have the former, but not the latter.

Answer 5

Probabilistic events do not occur the way you describe. There are no discreet outcomes with equal probability. Instead there is a continuum of probabilities called normal distribution.

The effect for cause A is a probability distribution B. In that distribution the average value of B has the highest probability, B-1 and B+1 have a slightly lower probability, B-100 and B+100 have significantly lower probability. The probability of the occurrence of a certain value is the lower the further away from the average it is.

So, only the probability distribution is determined by the cause. The actual effect occurs randomly within that distribution. The actual effect is not a brute fact determined by an unknown cause, it is just incompletely determined by the known cause.

Determinism is the assumption that the cause completely determines the effect. This is a useful practical assumption to make in classical physics, but it does not work with quantum mechanics.

The "instinct" may simply be that they are used to work with classical physics where variance in the effects is always due to variance in the causes. They don't seem to understand that the observed random variance must happen at some point anyway, if not in the effects now, then in some causes in the past. Determinism does not help in finding the random variance in the past, as determinism simply assumes that there is no random variance at all.

Answer 6

Does the want to seek determinism in physics come from a fallacy that it explains more?

That is a question about the motivations of any physicists who happen to have that want, not about philosophy. Furthermore, the assertion in the question body that a substantial proportion of modern-day physicists have such a want is questionable and will not be addressed here.

You clarified:

I’m saying that postulating a deterministic explanation only because it would supposedly solve a problem is a fallacy, since postulating a new explanation without evidence is no better than no explanation. One needs evidence to postulate determinism behind everything that is going on and so far there hasn’t been any.

Inasmuch as you are supposedly asking a question, about philosophy, I take that question to be whether the position you present in your clarification is sound.

In the first place, postulation itself is not reasoning, so no, postulating anything is not inherently fallacious. A postulate may be flawed, arbitrary, or contrary to available evidence, but those are altogether different things, speaking to whether we should accept the postulate. Merely choosing to postulate something that I like better, for any particular reason or for no reason at all, is not fallacious.

On the other hand, rejecting a postulate on the basis that it does not provide a deterministic explanation is an exercise of reasoning. It is not fallacious per se, but it rests on the proposition that only a deterministic explanation can be correct, and that proposition is not only in question, but it is a question of metaphysics, not physics. As such, I see no valid reason to accept that as a justification for rejecting non-deterministic explanations.

I guess that's what you're getting at when you say that one needs evidence to postulate determinism, but your actual claim is incorrect. You do not need any evidence at all to postulate anything. Rather, you need evidence to evaluate a postulate for its truth / reliability / accuracy. Promoting a proposition that you cannot support with evidence runs the risk of making you look foolish, either now or in the future, but it is not inherently erroneous, and it is anyway different from forming the postulate in the first place.