What is the probability of events that don’t seem clearly defined?

Question

It makes sense to talk about the probability of a series of coin tosses but what about seeing a TV on a wall, or seeing a person riding a bicycle on the street?

If one were to compare an event such as a person riding a bicycle on the street vs. a coin landing on heads 10,000 straight times, the second very strongly and intuitively seems more improbable.

But is it, or is it just that the probability of the other event isn’t as well defined? If it isn’t well defined, how does one compare the “likelihood” or “ease” at which nature can produce an event like 10,000 heads vs. a person riding a bicycle?

The second issue I seem to have trouble grasping with is depending on how I describe the event of a person riding a bicycle, the hypothetical probability of it seems to be different. John riding a bicycle vs. John riding a bicycle at 7 PM vs. John riding a bicycle at 7 PM on MacArthur street seem to be events with decreasing “probabilities”.

Specifying details of an event seems to be adding evidence, not subtracting it away. Thus, the more specifically detailed the event is, the more evidence one includes about the event. But this suggests that the “actual” probability of the event is when it is defined with as many specific details as possible. But this also makes the event seem to have a very low probability. What is wrong or right about this, and how can one compare the “rarity” of an event like this with a series of coin tosses?

Answer 1

"Not well-defined" and "unknown" mean different things. If we want to know whether John will be on his bicycle on MacArthur street at 7pm, we need one of the following two:

• a statistically significant sample of MacArthur street at 7 pm's in which to count instances of John on a bicycle, in which case we'll get a probability with an attached P-value.

• or a stochastic model of John which includes on a bicycle in MacArthur street as a relevant domain, and a set of measured initial conditions pertinent to the model, in which case we'll get a probability with an attached measurement error.

For instance, our model of John could be "John is generally trustworthy when he makes plans to ride his bicycle, but things happen to John which make him not keep his plans about 1/50 of the time" and our measurement could be "I heard John say 'I will be riding my bicycle on MacArthur street at 7pm'", in which case we should bet 50 to 1 that a measurement of MacArthur street at 7pm will reveal an instance of John on a bicycle.

The stochastic model itself, as with the coin flip, is based on a large number of previous measurements of the sample in question, of similar samples, or of different samples which can be related to the sample by logical inference.

Answer 2

This question would get a much better answer if asked on the statistics SE rather than here. For the first question the coin event probably seems better defined because you are using a familiar frequentist definition of probability, whereas it might be a Bayesian definition for the second. To understand what the probability means, you need to understand how probabilities are defined.

There are two basic definitions of a probability: frequentist and Bayesian. I'll start with the frequentist one.

A frequentist defines a probability in terms of a long run frequency, e.g. if you flip a fair coin a large number of times, the proportion of heads will converge on 0.5 and that is an example of a frequentist probability. This sounds very simple, and indeed it is if we are talking about large samples from some population (e.g. what is the probability of an Englishman being more than 5'10" tall?). However, it means you can't attach a probability directly to specific events. For instance you can't attach a frequentist probability to to the proposition "The next Englishman I meet will be taller than 6'1" tall". This is because there isn't a population of "the next Englishman you meet", there will only be one of them. The proposition is either true or it is false, it doesn't have a long run frequency (you could perhaps attach the probabilities 0 and 1, but nothing in between).

However, frequentists will often attach probabilities to specific events, but what they usually don't tell you is that the probability isn't about the event itself, but some (often fictitious) population of events in some underlying statistical model. This causes a lot of problems, for instance a lot of people don't realise that there isn't a 95% probability of the true value lying in a particular 95% confidence interval. What it means is that if you conducted the experiment a large number of times with new samples of data each time, 95% of the intervals that you compute will contain the true value. It is possible to construct confidence intervals where you can be 100% sure from the particular sample of data you have that the true value of the statistic is definitely not in the interval calculated.

So form a frequentist perspective the probability of "seeing a person riding a bicycle on the street" is the proportion of times you would see someone riding a bicycle if you made a large number of visits to the street. What it can't be is the probability of seeing a person riding a bicycle the next time you go to the street, because that would be a specific event rather than a statement about a population of events. In this case, you can usually safely move between the two, however if you did so, you would be implicitly moving to a Bayesian framework:

The other definition is the Bayesian definition, where a probability encodes your state of knowledge about the truth of a proposition or the value of some variable. So a Bayesian can tell you the probability that you will see someone riding a bicycle the next time you go to the street, or whether the next coin flip will be a head, because the probability is a measure of the plausibility of those propositions. Obviously Bayesian probabilities can be applied to statements about populations or long run frequencies. Indeed a long run frequency is often an excellent basis for a state of knowledge regarding a proposition.

Note that Bayesian probabilities are not necessarily subjective. They can be viewed as an extension of propositional logic to allow reasoning under uncertainty. In objective Bayesianism, priors are often used only to give the prior knowledge that you know you don't know something, or that the results of the analysis should have some invariance properties, such as it doesn't matter what units you measure the variables in.

So if you want to know how to specify and event, first you need to be clear whether it is a frequentist or a Bayesian probability and how they are defined.

"But this also makes the event seem to have a very low probability."

Yes, this is a common difficulty people have with probability. Consider the chance of picking a random number between 0 and 1 and it being rational (can be expressed as the ratio of integers). There are vanishingly few rational numbers in this interval compared to the number that can't be expressed that way, so the probability is zero. If you want to know how that is dealt with, ask on the statistics SE rather than here!

Answer 3

Probability can be fully determined statistically as long as there is a sample. However, in situations or cases that are difficult to precisely define, we primarily utilize a method called nominal statistics. In such cases, the p-value can approach 1.

In the process of creating samples, as you mentioned, if there is ambiguity in defining the subjects, we often create arbitrary scales and impose order. We assign numbers to them. For instance, interval scales, ratio scales, and in psychology, Likert scales. Through this, we can arbitrarily arrange the ambiguity into an order.

Regarding the last mention, it seems that you have some confusion. If the addition of information implies the imposition of conditions, then that adds more constraints, leading to a decrease in probability. Therefore, if there is an imposition of infinite information, the probability would become 0. On the other hand, if the only information we possess is always-true propositions (like p = p), then the probability becomes 1. This aligns intuitively with flipping-a-coin case as well.

Answer 4

The second issue I seem to have trouble grasping with is depending on how I describe the event of a person riding a bicycle, the hypothetical probability of it seems to be different. John riding a bicycle vs. John riding a bicycle at 7 PM vs. John riding a bicycle at 7 PM on MacArthur street seem to be events with decreasing “probabilities”.

Well of course the probability is different. If you add new attributes to the event you are creating an "event space" that is indexed by those attributes. An event that relates to a probability will be a set of locations in that event space. If you specify a street and a time, you are specifying one cell in the event space. If you specify a streetname but not a time, it is all the cells in that row of the event space. If you specify a time but not a name, it is all the cells in a column. If you don't specify either, it is all of the events in the space.

Each cell in the event space has a probability associated with it and the probability of the event is the sum of the probabilities of all of the cells selected by the attribute values (or absence thereof). So naturally the probability goes down the more closely the event is specified - every time you specify something you are excluding cells.

Answer 5

If J = Probability of John riding a bicycle
T = Probability of John choosing 7 PM for biking
M = Probability of John choosing McArthur street

The probability that John rides his bike at 7 PM on McArthur road = J × T × M

My two cents ...

Answer 6

This is a great question, first of all.

Let's take your example of seeing a person riding a bike on a street.

Without any additional information, you can give an educated guess that the probability would be less than seeing a car pass by. But by how much? Of course, this guess is based on prior information that you have that transportation by car is more common than transportation by bicycle. Is it possible to quantify the probability?

Now let's say you have access to the following information: a) the distribution of bike ownership in the city, b) the distribution of mean popularity of the 4 available transportation methods, c) distribution of means of popularity of ridership by season, month, and time of day, d) the distribution of age brackets, e) the popularity of bike ridership by age bracket etc.

The list can go on and on. Any additional factual piece information that you add, will nest the probability in a set of conditions assumed to be true. These will increase the certainty of one of the possible outcomes occurring: a) seeing a person riding a bike on the street b) not seeing a person riding a bike on the street. That is to say, if the probability was initially 50/50, now it asymmetrically in favour of one of the propositions. Regardless of which way it tips, Shannon entropy (the mathematical measure of uncertainty) has decreased, and the certainty of one of the outcomes occurring has increased.

Why two probabilities? Because your event has two possible outcomes. If there were more possible outcomes, the distribution of 100%, usually normalized to a value of [0,1] range with 0 meaning no probability and 1 meaning certainty, will be along k partitions.

The number of k partitions is partially defined by the event and partially dependent on what one is looking to predict. You could distribute the probability across any number of arbitrary events, depending on what you're trying to predict. So to answer the question you posed to the other poster, yes the individuation of events is in some sense subjective.

Philosophically, there is ongoing debate about how events are individuated. For example, some philosophers contend that event individuation depends on natural kinds. This means that conceptual joints must relate to ontological joints that are intrinsic to mind-independent phenomena. For example, if I want to know the probability of whales feeding their young, the concept of whale corresponds to a natural kind and the predication "feeding their young" is an empirical event. Music subgenres on the other hand are not natural kinds as they depend on convention. The other view contends that event individuation can be as fine-grained as our linguistic posits allow. This seems to go hand in hand with your suggestion that the level of description of an event is in some respects arbitrary. However, there are formal constraints on how events are to be individuated. Generally speaking you should partition classes into mutually exclusive and exhaustive sets.

Probability theory is agnostic about these philosophical views since it only defines the mathematical form of probability. We can use Bayes' Theorem to nestle the probability of an event given a set of dependent conditions. The probability of an event A can be computed by conditioning it to a set of mutually exhaustive and exclusive partitions/events B(i) where i = 1, 2, 3...n and summing those probabilities: P(A) = ∑P(A|B(i))P(B(i)). Conversely and by corollary we get: P(A|B) = P(A&B)/P(B). This is the formula for conditional probability which can be used to derive Bayes' Theorem -- P(A|B) = (P(B|A)P(A))/P(B) --, wherein the likelihoods -- P(B|A) -- can be any number of classes (events with discrete or continuous outcomes) that constrain A.

For artificially constructed scenarios like games e.g. chess, the sample space is discrete and can be determined via combinatorics, depending on whether we are counting combinations or permutations and whether we're sampling with or without replacement. What about sample spaces for continuous variables? A continuous sample space is a sample space where some interval contains an infinite number of possible outcomes. For example, the position or distance of a basketball on the court. Remember that discrete sample spaces can be finite but also infinite, specifically countably infinite, where 'countably' means 'can be put on a 1 to 1 correspondence with the natural numbers' as defined in Cantors' Theorem of Transfinite numbers.

Talk about sample spaces, presupposes classical probability, wherein Bayesian probability can also be assimilated. In other words, classical and Bayesian probabilities presuppose probability distributions of variables, which means we can work with some estimation of the sample space in advance. However, there's another conception of probability called Empirical probability that eschews sample spaces and simply takes the ratio of outcomes over trials e.g. 10 tails over 25 tosses. Empirical probability does away with the a priori assumption of the principle of indifference where in binary scenarios like coin tosses for example the probability prior to trials is distributed equally between the outcomes i.e. 50/50.