In Bayesian statistics, the posterior probability of a hypothesis is composed of two parts:
- the prior, reflecting our initial belief in a certain hypothesis, and
- the likelihood, which represents how well our observations are explained by each hypothesis.
One way I interpret the prior is that it acts as a tie-breaker for hypothesis which explain the observations/data we have about the world equally well. If this interpretation is correct, is Occam's Razor an example of a generalized prior?
As a small example, consider the following setup: One afternoon I am cooking a steak. Just as I get ready to eat it, the doorbell rings. I take care of the situation, but once I return to kitchen, (i) the plate is shattered on the floor, (ii) the steak is gone, and (iii) my dog is sitting next to the shattered plate, happily wagging its tail.
What happened? Let's say I come up with (for simplicity only) two hypotheses:
When I left the room, the dog took the opportunity to steal the steak from the plate. In the process, he knocked the plate off the table. Now he feigns innoncence. Bad dog.
When I left the room, aliens teleported into my kitchen, trying to steal the steak. My dog valiantly tried to defend the homestead, knocking the plate off the table in the process. With their technological superiority, the aliens zapped my dog with an amnesia/happiness-beam, stole the steak, and teleported out of the room before I returned.
One of these hypotheses is clearly nonsense (my dog would never steal a steak... hehe), but both explain the three observations I have made. Hence, in a Bayesian sense, their likelihoods (given only these three observations) would be equivalent. The second hypothesis would probably be assigned a low prior for assuming a lot of unsubstantiated space magic, but aside from that, it's also more complex than the first hypothesis. So if I apply Occam's Razor and consider the second hypothesis inferior merely on the basis of complexity... would that constitute a prior?