How can a fundamentally random process follow a probability distribution? [closed]

Question

If a process is fundamentally random, how can it follow a probability distribution? What "keeps track" of the statistics of the random process and "ensures" that its outcomes align with the probability distribution it is supposed to "obey" over the long run? What could possibly prevent a fundamentally random process from producing outcomes that exhibit no consistent pattern at all—perhaps appearing highly regular for a time, then Gaussian, then uniform, then completely chaotic, etc.?

---

Clarification due to misundertanding

Answers making their case from examples involving dice, balls, and similar objects are out of the scope of this question because I'm interested in processes that are fundamentally random, not deterministic with the "appearance" of randomness. Dice and balls obey the laws of rigid body dynamics, which are deterministic, not random. Anything that resembles a pseudo-random number generator, which is fundamentally deterministic, not random, is off-topic as well.

---

Are there truly random events?

For those interested in that question, it has been asked before:

• Is there anything that is totally random?
• Are actually random events causeless?
• Would truly random events be strictly equivalent to events without a cause?

Answer 1

I believe that "fundamentally random" (as posed in the question) and following a probability distribution are mutually incompatible.

In order to follow a probability distribution, outcomes need to be constrained. They must: (1) be mutually exclusive; (2) be independent of each other; (3) have a well-defined probability. Also (4), the probabilities for all outcomes must sum to exactly 1.

In the examples of dice, coins, cards, etc., it's the construction and physical nature of these things that provide the constraints. Even in abstract fields such as economics, psychology, meteorology, and politics, the presumption is that something deterministic is affecting the outcome, even if we don't know enough to trace all the causes and effects. (And we construct outcome curves first, then deduce the probabilities of individual events.)

The only exception I can think of might be quantum mechanics. Even then, there is the presumption that something is constraining the universe to act the way it does. That "the book of nature is written in the language of mathematics."

Probability distributions arise as a mathematical consequence of the 4 constraints listed above. It is the physical nature of real systems that impose those constraints.

The original question was: If a process is fundamentally random, how can it follow a probability distribution? The answer is: It won't. At least not necessarily. And at least if "fundamentally random" is defined to exclude anything constrained by physical reality.

Answer 2

Nothing "keeps track" of a probability distribution other than us observers. The physical processes are whatever they are, and to us, this may manifest as observable probability distributions.

The probability distribution is our description of our observations and a predictive model of the underlying process.

Normal (Gaussian) distributions arise naturally as sums of many independent distributions. This is a consequence of the central limit theorem.

Answer 3

Good, you think about the difference between objects and attributes, what Hoffstader called figure and ground.

At the world's fair, I watched in amazement as rubber balls fell through a series of pegs and form a normal distribution pile. And I asked myself the exact same question.

The balls don't have to pay attention to the distribution; the distribution is contained in the balls themselves, in their likelihood to fall this way or that. Each one acts independently.

Every time the ball hits a peg, the symmetry breaks. It will break left twice in a row one quarter of the time.

The balls not only don't have to remember the distribution, they don't even have to remember their last choice at the preceding peg.

Answer 4

What "keeps track" of the statistics of the random process and "ensures" that its outcomes align with the probability distribution it is supposed to "obey" over the long run?

Lowri addressed this already -- probability distributions are descriptive not prescriptive.

As far as the "stability" of a distribution, there is nothing that precludes that. In the field of stochastic processes we deal with "non-stationarity" all the time -- a Gaussian Random Walk does not converge to any distribution because it’s non-stationary -- it tends to veer wildly all over the place.

---

One fun thing I like to show when I'm teaching probability is that you can apply probability even when the underlying process is not random at all.

It's interesting to look at the density of deterministic functions like sin(x) over the unit interval. The only difference is if X is smoothly increasing or is all over the interval in no particular pattern.

Answer 5

What "keeps track" of the statistics of the random process and "ensures" that its outcomes align with the probability distribution it is supposed to "obey" over the long run?

You don't need any "memory" in order to generate samples from a probability distribution. When you roll two fair dice, a sum of 7 is six times as likely as a sum of 12. This follows from the logic of the experiment; it doesn't require the dice to "remember" that they previously rolled a 12 so now they have to roll six 7s before they roll another 12. The misconception that previous samples from the same distribution constrain future samples, so as to ensure the correct distribution, is the root of the gambler's fallacy. Nothing "keeps track", and nothing needs to.

What could possibly prevent a fundamentally random process from producing outcomes that exhibit no consistent pattern at all—perhaps appearing highly regular for a time, then Gaussian, then uniform, then completely chaotic, etc.?

Nothing prevents that, but you have to understand the difference between a random process and a probability distribution. A random process can produce a sequence of outcomes which are not independent or identically distributed. For example, suppose I play a game where on the first 100 turns, I roll a six-sided die, and on the next 100 turns I roll an n-sided die where n is the number of 1s I rolled in those 100 turns, and then choose a different die in this manner every 100 turns. (Suppose these are truly fair, truly random dice.) This is a fundamentally random process, and it apparently has one probability distribution for a time and then changes to a different distribution, then to another, and so on.

But this is a random process, not a probability distribution. In fact, there are infinitely many different probability distributions here: the distribution of the number rolled on turn 1, the distribution of the number rolled on turn 2, and so on. The definition of a random process does not require that these distributions all be the same, nor that the outcomes at each turn be independent. We shouldn't expect all these distributions to be the same, not least because there is a different rule for which dice I roll on turn 1 vs. turn 101. Nonetheless, if we take repeated samples of one of these distributions (e.g. by playing the game many times and measuring the number rolled on turn 361), those repeated samples will be independent of each other.

So a random process is allowed to "keep track" of things (in this game, I have to keep track of the turn number, the number of 1s rolled since I last changed the die, and how many sides the current die has). But each of the probability distributions can be sampled without keeping track of anything in between samples. To take independent samples of the distribution for turn 361, for each sample you play a new game for 361 turns.

Notice that this new process (on each step, play a new game for 361 turns) does produce independent outcomes which don't require "keeping track" of the number of steps, or the outcomes of previous steps. Since there is no state, and the rules are the same on every step, the outcomes (i.e. samples) do define a probability distribution. Like before, we could say that there are infinitely many distributions ─ the outcome for game 1, the outcome for game 2, etc. ─ but in this case, those distributions are all the same. Mathematicians call these outcomes IID, which stands for "independent and identically distributed".

It also doesn't have to be physically possible to take independent samples from a distribution. Only that if a process does "keep track" of some state, or logically does not produce outcomes according to the same rules each time, then we don't generally expect its sequence of outcomes to follow one fixed probability distribution.

Answer 6

You answered the question yourself. You said:

What could possibly prevent a fundamentally random process from producing outcomes that exhibit no consistent pattern at all—perhaps appearing highly regular for a time, then Gaussian, then uniform, then completely chaotic, etc.?

A random process that appears highly regular is simply not random. Nor is it random when the process is uniform.

I think though, that a better answer is to just do the math. When you throw two dice, which I hope you agree with me is a totally random process, then there are 36 possible outcomes:

(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)

When you add the outcomes you get the following distribution:

( 2) ( 3) ( 4) ( 5) ( 6) ( 7)
( 3) ( 4) ( 5) ( 6) ( 7) ( 8)
( 4) ( 5) ( 6) ( 7) ( 8) ( 9)
( 5) ( 6) ( 7) ( 8) ( 9) (10)
( 6) ( 7) ( 8) ( 9) (10) (11)
( 7) ( 8) ( 9) (10) (11) (12)

You will now instantly see that there's only a 1/36 chance of obtaining 1, but a 6/36 = 1/6 chance of obtaining 7 as you can see in the diagonal. If you plot this in a graph you will see a beautiful bell shaped curve that is the probability distribution you were referring to.

Well, it is not a bell curve yet, but if you use more dice it will approach a bell curve. Anyway, I hope you agree with me that there is no magical following or obeying a rule. It is just following the math.

Answer 7

Random processes don't follow probability distributions; random processes have probability distributions. If we have a truly random process (however that might be defined), what we know is that events are independent: the occurrence of one event is not conditioned on the occurrence of any other event. In that sense the process has no memory. But if you take hundreds, thousands, or millions of these events they will necessarily show some pattern. Maybe it's uniform, with events occurring equally across the spectrum; maybe it's normal, with events clustered towards the middle; maybe it's exponential, with most events at one end and the rest tapering off towards the other.

Think of it like raindrops. Statistically it's impossible to say where a single raindrop is going to fall. But if we have a big rainstorm we can easily see if it rains more in one place than another, or if it rains equally everywhere, just by putting out bowls in different areas to see how much they fill. The distribution is a consequence of the collected random events, not the other way around.

Answer 8

As soon as you postulate that probabilities at least have to sum up to 100% (either something happened or it didn't), you immediately get the central limit theorem showing, that even if the distribution is strictly what we've observed so far, and looks like no recognisable curve (strictly speaking, that's not a curve, but a discrete sampling) ... if you start combining said curves for multiple processes, you will "magically" get a normal distribution going. And if you then claim "well, the distributions weren't really random then", then you're actually trying to constrain a process to be random, when it randomly could be non-random.

I think the simple answer here, is that the "fundamentally random" you're seeking just doesn't exist. A universe where randomness could result in antimatter bowling balls appearing in your soup with anything but minuscule probability would not have survived to evolve to the point where there's anyone to ask this question.

To be clear, nothing in our universe, as far as we know, precludes said bowling ball in your soup. It is just so unlikely, that to the best of our ability to tell, it won't happen within millions of universe-lifetimes.

Alternatively, perhaps an even unlikelier event did happen once: the Big Bang.

The answer thus is that the behaviour of randomness is as it is because otherwise you wouldn't exist to ask the question. This is, after all, Philosophy, not Physics, which has much more rigorous definitions for randomness, and can show how distributions arise.

Answer 9

I come very close to @Lowri's answer here and I would like to add that "randomness" is too an observer's perspective. But besides that, I see that your question has an underlying concern, regarding how can (even from our subjective experience and understanding) order come out of chaos! This my friend is the "million-dollar" question of all times!

Answer 10

I commented in a few places, but I think it's worth an answer.

Probability distributions come about when we apply labeling to the outcomes of a random process.

I'll provide a few examples - here, we're assuming no bias has been added to these processes.

Consider a die. Here we have a cube. We throw it, and it lands with one of its faces pointing up. Since the cube is symmetrical, it's reasonable to conclude that any face could be the upper one following a throw, and there's no reason to suppose that one face is favored over the others. Once we apply labels to each face, we will get a distribution. The faces are still unfavored, but we now aggregate the outcomes by label, and we get our uniform distribution.

For a Plinko board, we know that each time a ball hits a peg, it can go left or right. The ball has no memory, so each pin it hits goes through an identical process. Symmetry tells us that each path through the board is equally likely. Its path can be any sequence of Lefts and Rights, each with the same probability, so the overall probability of a path depends only on its length. We get our Gaussian distribution once we apply labels to the balls' destinations. Again, how we aggregate these outcomes leads to a distribution.

Finally, when we examine a radioactive isotope, we know that any nucleus in the sample may or may not decay with some probability. There's no reason to conclude that any given nucleus differs from the others, so the decay of each one is as likely as the others. Now, we apply the label of "radioactivity," which refers to the amount of radiation emitted in a period. In this case, it's not so much a "distribution" as a decay curve: the radioactivity decreases over time. This isn't because the behavior of individual nuclei has changed... just that the number of them has changed. Again, our aggregation results in the decay curve... not the behavior of the elements of the process

Answer 11

What could possibly prevent a fundamentally random process from producing outcomes that exhibit no consistent pattern at all—perhaps appearing highly regular for a time, then Gaussian, then uniform, then completely chaotic, etc.?

In maths, infinite sequences of equally distributed numbers are considered normal (proof is difficult). That means they contain any finite sequence of numbers anyone can specify. That means if you come up with any finite sequence of numbers that has gaussian distribution, or other non-equal distribution, then this sequence will surely appear somewhere in that infinite sequences. Same as any finite sequence of the same number. Like 1000 times 7. They will appear. Not necessarily in the beginning, or in the first billion trillion positions. Not necessarily anytime a fast (non-deterministic if you will) computer could calculate before our sun goes out and all life on earth would be gone. But sometime, it would get there, if one is patient.

In fact, it would contain any such finite sequence infinitely many times. "That's a lot." But the longer the series to check, the more rarely they would appear.

So, this also means that if you take the board with pegs and balls, and flip it infinitely many times, then at some point all the balls may end up in the same slot. Or all slots may have the same number of balls. But this will be so incredibly rare that likely you would never in practice observe it. But nothing is preventing it.

Answer 12

The whole point of uniformly random is that it doesn't track previous values.

What you are doing is going one step past the random events, looking at the statistics based on them, and saying, I want these statistics to be random.

So let's do that thought experiment. Take a sample of noise and interpret it as a bar chart or line chart regarding the base data. Now we see that the process which generated the base data points must be non-random.

Answer 13

I am taking "fundamentally random" to mean: For N events (where N is very very large), each event has an equal probability of occurring and the sum of the probabilities for each event = 1.

So the probability for each event is 1/N. The curve for this would be a constant flat line. The actual curve would look like noise with an average value of 1/N summed over N events.

Now, it's certainly possible for sub patterns to emerge that average 1/N over N events especially when N is very large but these patterns are not repeatable when the experiment is repeated. Here, I mean that the sub pattern may occur between two experiments but I can't predict the location of the pattern in the chain of events from experiment to experiment.

A simple example is randomly picking a number between 1 and 5. How can I determine the process is random? I can state that the average value per event over 5 events is 1+2+3+4+5/5 = 3.

If I repeat this experiment N times where N is a large number then the average value per event will still be 3 regardless of the order that I pick the numbers.

The curve looks like flatline noise with a max of 5 a min. of 1 and an average of 3.

Does this help?

Answer 14

The simple answer is that we don’t even know if it does. We don’t know if anything is fundamentally random. We only have interpretations of quantum mechanics that propose that certain things are fundamentally random but we don’t actually know this.

And contrary to what the commenters said, everything being fundamentally deterministic is hell of a lot simpler than independent processes all occurring without no cause that then magically always align together to create consistent probability distributions.

Sure, there may be no explanation for why everything is deterministic if determinism is true. But if everything is fundamentally random, you have more unexplained brute facts. This is because every quantum event becomes fundamentally unexplained without a cause in this view, unlike a deterministic universe.