# Paradox involving the principle of indifference

The principle of indifference states that:

"in the absence of any relevant evidence, agents should distribute their credence (or 'degrees of belief') equally among all the possible outcomes under consideration."

This is for determining epistemic probability. I.e., these are not repeatable events and the possibilities are not drawn out of a bag. Rather, in this scenario there is and always was a probability of 1 for the actual outcome and a probability of 0 of all others, it is our ignorance which forces us to ascribe a lower probability to a given outcome.

Suppose I have a set of N elements. The nth element is defined as a two element sub-set of the form (n, B) for all n<N, where n is simply its index and B is a Boolean, e.g. the 3rd could be (3, 0) or (3, 1). Lets suppose one of these elements is "true", and there is no available evidence to determine which is true. By the principle of indifference, each possibly outcome is equally likely. We can also say that the probability of B=1 is 50% by the same principle.

Now let's say that B=1 only when n is a multiple of 3. If we apply the principle of indifference to each possible element and summing those probabilities, we find that the likelihood of B=1 is around 1/3 if N is large. This implicitly assumes that the discrepancies between the number of elements with 1 and 0 serves as evidence for B. But this could just as easily flipped the other way around, treating B with indifference and using that as evidence that the likelihood of n being a given multiple of 3 is lower than it being a given number which is not a multiple of 3. We arrive at different probabilities for the same question.

If you are not convinced that we can flip the situation around, then this example may make the problem more obvious. I have either a cube or a sphere, and each face can take one of 3 colours. Clearly there are more possible cubes than spheres, so should I expect to have a cube rather than a sphere or is the probability of having either equally likely?

This is just a less abstract version of the previous problem (with the variable B signifying a cube or a sphere), albeit in different proportions. It seems people are more willing to treat the variable B with indifference in this more concrete example (i.e., 1/2 chance B=1), whereas with the numerical example they are more willing to treat the individual elements with indifference (i.e., ~1/3 chance B=1).

I have three questions with regards to this apparent paradox:

1. Which "level" should we treat with indifference and why?
2. If it is situation dependent, then how does context determine which we choose?
3. Why does our intuition seem to change between the two mathematically equivalent examples?
• Can you try to clarify this? Your concrete example doesn't seem to fit the abstract problem. It should be something like "you have N spheres numbered 1 to n. those numbered a multiple of 3 are black; the rest are white. Drawing a random sphere and applying the principle of indifference to the color (1/2 W, 1/2 B) implies that ... what? That 1/2 of the spheres that are not a multiple of 3 are missing? Commented Jul 25, 2022 at 14:12

A better demonstration of this principle is the Bertrand Paradox.

The problem is stated as follows: "Consider an equilateral triangle inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?"

There are several intuitive ways you can pick a chord 'uniformly' at random, but they each give different answers.

The "random endpoints" method: Choose two random points on the circumference of the circle and draw the chord joining them.

The "random radial point" method: Choose a radius of the circle, choose a point on the radius and construct the chord through this point and perpendicular to the radius.

The "random midpoint" method: Choose a point anywhere within the circle and construct a chord with the chosen point as its midpoint.

The reason for the confusion is that there are several different equally-intuitive ways to parameterise a chord, and they are non-linearly related to one another. So a uniform distribution on one set of parameters will be non-uniform on any of the others.

Edwin Jaynes discussed how you might answer it by using the 'maximum ignorance' principle. We should not use information not specified in the question. Since the question doesn't specify the position or size of the circle, the distribution shouldn't depend on it, and we ought to use a distribution invariant to translations and changes of scale. The unique distribution that does so is the "random radial point" method above.

Now let's say that B=1 only when n is a multiple of 3. If we apply the principle of indifference to each possible element and summing those probabilities

Why should you?

"in the absence of any relevant evidence, agents should distribute their credence (or 'degrees of belief') equally among all the possible outcomes under consideration."

That is not the absence of relevant evidence that is the presence of relevant evidence. You don't have to guess if you know that the index determines the truth value. But suppose you knew that this was the initial condition before someone swapped all the indexes and now all you know is that the probability is roughly 1/3 for large N.

This implicitly assumes that the discrepancies between the number of elements with 1 and 0 serves as evidence for B.

Not implicit, that is explicit. Like you know because of evidence that you're drawing a ball from a bag where 1/3 of the balls are green and 2/3 of the balls are not or at least are unknown. So your chance of drawing a green one is at least 1/3.

But this could just as easily flipped the other way around, treating B with indifference and using that as evidence that the likelihood of n being a given multiple of 3 is lower than it being a given number which is not a multiple of 3. We arrive at different probabilities for the same question.

So you're point is that if you'd assume there are only 2 options then it's 50:50 (either a multiple of 3 or not a multiple of 3), but that one has a probability of 1/3 and the other of 2/3? The problem is that you again already have evidence and prior knowledge of the setup, so you don't have to guess.

If you are not convinced that we can flip the situation around, then this example may make the problem more obvious. I have either a cube or a sphere, and each face can take one of 3 colours. Clearly there are more possible cubes than spheres, so should I expect to have a cube rather than a sphere or is the probability of having either equally likely?

I assume what you mean is that a sphere has 1 and only one face while a cube has 6 so there are 3 unique spheres and 6³ unique cubes. While there are only 2 options whether it's sphere or cube. However that is vastly different from the previous as you have a lot less information.

Like you seem to think that it is 50:50 or 1:72 but no one specified that any possible cube has actually been made or that there are no copies of the possible spheres. And without further information to narrow it down, you probably need to assume that any possible relation of cubes to spheres is valid and that therefore on average there are as many variations where there are more cubes as there are where you've got more spheres. And so you're stuck with 50:50.

That could be horribly wrong, but that's the risk with every guess.

Which "level" should we treat with indifference and why?

In the absence of any hint at a preference: the different options.

If it is situation dependent, then how does context determine which we choose?

It gives hints and evidence as to what option is more or less likely than the rest.

Why does our intuition seem to change between the two mathematically equivalent examples?

Because as exemplified, they are not actually equivalent, because in one case you've got vastly more relevant information.

• I don't see how the cubes/spheres scenario is any different. It's just rather than a label, we have a shape. You could imagine an equivalent scenario where B=1 if n<=6^3 and B=0 otherwise. What changes to make that no longer a 50/50 chance?I also want to stress: there is only one cube/sphere, and these are not being drawn at random. There is either a cube or a sphere and this is predetermined.
– 1986
Commented Jul 25, 2022 at 14:30
• This would be a vastly different scenario. Because then I'd know how huge the set is and what's the distribution of cubes/spheres within that set. Then I'd not have to guess the probabilities but I'd already know them. These two options wouldn't be indifferent but I'd know that they are different. Do you by any chance have heard of this Monty Hall Problem and are referencing that? Commented Jul 25, 2022 at 14:37
• In the cubes/sphere scenario the set is of size 6^3+3. In the numeric version, consider a set of size 6^3+3 where the first 6^3 are labelled "C" and the last 3 are marked "S". So the information here is the same. Is the probability of the element being within the first 6^3 items equal to the likelihood of it being in the last 3? If the information is the same as the cube/sphere scenario, the only difference being a visualisation, then surely the odds must be even?
– 1986
Commented Jul 25, 2022 at 14:45
• No the point is that it's different from the version where you don't know the size and makeup of the set to begin with. Or is your point that this is irrelevant information? That the knowledge of the size of the set doesn't tell you something about the outcome because the selection is not random and you don't know the criterion of selection? Well in that case you're back to 50:50. But either way you don't have the paradox, do you? Commented Jul 25, 2022 at 17:02

Bear in mind that the principle of indifference is not a law of probability theory. It is more like a rule of thumb that is useful when solving particular problems. In some cases, the principle can be justified using game theory.

The principle really only works in simple cases where there is a single property that is of interest. Assuming an equal probability for all possible outcomes in effect assumes that there is a single property or measure that all those outcomes can be assessed against. The probability is then distributed uniformly according to that measure. In practice, there may be several different measures depending upon what property is of interest to us. So, as you point out, you can get quite different results by attempting to use the principle. This issue is related to the frame problem in statistical inference and the problem of finding an uninformative prior in the objective variant of Bayesianism.

In some cases, we can address this issue by making use of the maximum entropy principle, or we can use the Kullback–Leibler divergence to find the minimum information divergence between the prior and posterior distributions. But again, this is not guaranteed to give a unique solution in all cases.

As to the question of what we should treat with indifference, the answer is that it depends on what our objective is, i.e. what decision problem we are trying to solve.

Suppose that at a factory, a machine makes cubic boxes whose edges are of magnitude between one and five units. We have no information about how the machine works or what the probability distribution is. Using just the principle of indifference, we could assume that the length of the edge of a cube is uniformly distributed between 1 and 5. Or we could assume that the area of a face of a cube is uniformly distributed between 1 and 25. Or that the volume of a cube is uniformly distributed between 1 and 125. Which of these assumptions makes the most sense will depend on some plausible guesses about how the machine works, or some considerations about what problem we are trying to solve. If our job is to stack these cubes on a truck and the truck has a weight limit, and the cubes are of uniform density, then the volume of a cube is of more interest to us than the size.