Is the principle of indifference invalid?

Question

The principle of indifference (also called principle of insufficient reason) is a rule for assigning epistemic probabilities. The principle of indifference states that in the absence of any relevant evidence, agents should distribute their credence (or 'degrees of belief') equally among all the possible outcomes under consideration.

Is this valid? Does this also technically mean that assigning a prior of 0 to everything would still technically follow the principle of indifference?

To me, it seems as if a uniform distribution is just one type of distribution, and would seem to require justification. The only way to escape justification is to assign no priors.

Secondly, it seems as if one can come up with an infinite number of theories that have no evidence to explain something. The only way for our credences to be equally distributed among an infinite number of theories is to assign them all to be 0. But this contradicts Cromwell’s rule that says not to assign a prior or 0 or 1. What should one do in this case?

Answer 1

The short answer is that sometimes it is OK to use the principle of indifference, but it is not generally applicable. Under some circumstances, it can be considered to be a special case of an uninformative prior.

The intuitive idea behind it is that in simple cases where there is a definite range of possibilities and no information whatsoever about which is correct, sometimes the best thing to do is to assume a uniform distribution across those possibilities. The emphasis here is on simple cases.

Sometimes the principle can be justified using game theory. Suppose someone shows you a box and tells you it contains a green or a red marble, but you have no information about the probability distibution, so you don't know which is more likely. However, you must offer me the opportunity to bet $10 that the marble is green, not green, red, or not red, according to odds that you set.

Simple Dutch book arguments require you to assign probabilities such that P(marble is green) + P(marble is red) = 1. But we can go further and use some game theory. If you assign any probability other than 0.5 to each option then you will be offering me longer odds on one option than on another. E.g. if for some reason you thought that P(marble is red) = 0.6 you would offer me odds of 3:2 on green and 2:3 on on red. I would respond by taking the longer odds and betting on green. Like you, I don't know which colour is more likely, but if I bet on green and win I take $15 from you, while if I lose I give you $10. By assigning any probability other than 0.5 you have allowed me what game theorists call a 'dominant strategy' by betting on the option with the longest odds.

In practice, the principle of indifference is highly problematic on a number of counts.

1. It may not be feasible to pin down the range of possibilities.

2. If your range of possibilities in infinite, you cannot have a uniform distribution across them: it is called an improper distribution. (Though in some cases, Bayesians do use improper priors if they can be shown to update to proper posteriors.)

3. It may be problematic to identify which variable we want to treat as uniformly distributed. An example, after van Fraassen, is the following. Suppose a machine makes cubes whose edges have a length between 1 and 2 units, and hence whose faces have an area between 1 and 4 units, and volumes between 1 and 8 units. Depending on whether you choose length, area or volume, you could get 1.5, 2.5 or 4.5 as indifferent values.

4. There may be a complex structure to the range of possibilities. What is the probability of there being a rat in this hole? What is the probability of there being a rabbit in this hole? What is the probability of there being a snake in this hole? What is the probability of there being a mammal in this hole? What is the probability of there being an animal in this hole? They cannot all be 0.5 because of the dependencies between them. This problem can sometimes be addressed by using the principle of maximum information entropy, which can be extended to continuous variables by minimising the Kullback-Leibler divergence between the priors and posteriors.

Answer 2

A uniform distribution IS just one of many possible distributions. And often times it actually IS the case that 1 option is correct and the others are just wrong. Just picture a coin flip and realize that the landed coin's distribution of probability is not 50:50, but that a particular side is up.

Though the point is not that the distribution of probabilities is actually 1/n, the problem is just that you have no way tell what distribution you're ACTUALLY dealing with.

So given your lack of information you're presented with n possible options which show no meaningful difference with regards to their outcome. So your coin might have different reliefs, a die might have more or less eyes per side, balls in a bag might have different colors but none of that tells you which one is more likely to be picked.

So from your point of view all these options are equally nebulous and so anything other than assuming them to be of equal value, if you can't make out any difference, amounts to guessing (without any reason related to the problem; so would be irrational).

So yes the bigger n the smaller the probability to pick a particular of those n options. Though as long as n is finite 1/n is bigger than 0.

So it's insufficient that the number of options is very big, it must be actually without limits in which case at least the SEP definition of the principle of indifference would have added the catch of:

Given n mutually exclusive and jointly exhaustive possibilities, none of which is favored over the others by the available evidence, the probability of each is 1/n.

So this would likely not be covered by it. Also more interesting than the limit of 0 which essentially just means "I know NOTHING, hence I distrust EVERY explanation of mine", would be the problem that the PoI is possibly ambiguous.

Like the probability of a horse winning a race between 3 horses could be 1/2 (either it wins or it doesn't) or 1/3 (one of the 3 is going to win) or any other fraction depending on how you phrase the problem and outline the different options.

https://plato.stanford.edu/entries/formal-epistemology/#PriInd

Though the end of the article hints at how there are attempts to use it not just as a pragmatic rule but argue that it's more reasonable to use it than not.

Answer 3

There will almost always be some way to distinguish the possibilities. At the very least, we have Occam's razor to fall back on. In probability terms, if there are two mutually exclusive possibilities, and one of them is simpler to describe, the simpler-to-describe one is more likely.

In some circumstances, a uniform distribution will be a reasonable prior. For example, for a coin toss or dice roll, we know all the sides have a roughly equal chance of coming up. But this is due to the particular design of the coin and the dice. If you are instead rolling a rock, the side of the rock that ends up facing upwards will not be uniformly random, because the rock is unevenly shaped.

For a classical example of how it can be misleading to always go with a uniform prior distribution, see the Bertrand paradox. In short, if you want to randomly select a chord on a circle, the distribution you get depends on the method of selection of the chord, even if you try to use uniform distributions throughout the selection process.

Secondly, it seems as if one can come up with an infinite number of theories that have no evidence to explain something. The only way for our credences to be equally distributed among an infinite number of theories is to assign them all to be 0.

This doesn't work because 0 + 0 + 0 + ... (an infinite sum of 0s) still sums to 0. You actually can't have a uniform distribution over a countably infinite set of theories. If you let the probability of each theory be 0, then the sum is also 0. Or if you let the probability of each theory be greater than 0, then the sum is infinite. It doesn't work.

This is in fact the reason for Occam's razor. The only probability distributions possible over an infinite sequence of theories, that sum to 1 as any probability distribution must, are those that assign higher probability to earlier theories in the sequence and lower probability to later theories in the sequence. For example, 1/2 + 1/4 + 1/8 + ... does sum to 1. The earlier theories in the sequence we may name "simpler."