Inadequately small sample fallacy

Question

I am almost sure there is a name for the fallacy whereby its exponent tries to apply a very small sample of observation, usually anecdotal evidence, as representative of the population, of which that sample is part, at large.

For example:

Burmese python cannot possibly be dangerous. My friend has two of them and they are both sweethearts.

It's nice to be reserved against hasty generalization based on (however carefully collected) statistics -- but it is laughable to argue one's pathetically minute sample of observation trumps that of, say Pew research and similar (which I think can also be considered hasty generalization but in a non discriminatory way).

What is the name of that fallacy?

Answer 1

I would say Hasty Generalization still applies, although since the speaker is trying to argue against the general claim, we can call it a Hasty Refutation

But yes, this argument has the further problem of being anecdotal evidence on the basis of personal experience, which means:

1. It is not a random sample: instead of randomly picking two cases, the speaker carefully chose two that would support their view.

2. The speaker may well construct the memory of their experience in a way to align it with the view they wish to express ... and thus come to believe. For example, the burmese pythons the speaker is familiar with may not have done anything that hurt the speaker, but that is not the same as being 'sweethearts'. Personal experience is always subject to those kinds of constructive interpretations and 'polishing', whereas this is indeed much harder to do with data from Pew research.

Answer 2

I think it's a "Faulty Generalization" (en.wikipedia.org):

A faulty generalization is a conclusion about all or many instances of a phenomenon that has been reached on the basis of just one or just a few instances of that phenomenon. It is an example of jumping to conclusions.

For example, we may generalize about all people, or all members of a group, based on what we know about just one or just a few people. If we see only white swans, we may suspect that all swans are white.

That includes "Hasty Generalization" (en.wikipedia.org):

Hasty generalization is an informal fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence—essentially making a rushed conclusion without considering all of the variables.

The fallacy is also known as:

• Illicit generalization
• Fallacy of insufficient sample
• Generalization from the particular
• Leaping to a conclusion
• Blanket statement
• Hasty induction
• Law of small numbers
• Unrepresentative sample
• Secundum quid

Answer 3

Here is Wikipedia, "Fallacy of composition".

The fallacy of composition arises when one infers that something is true of the whole from the fact that it is true of some part of the whole (or even of every proper part).

This fallacy seems to describe what you are looking for.

Answer 4

As already said, it is "Hasty generalization" as subset of the more enclosing "Faulty generalization". For the sake of the arguments below, I simply assume that the observation of data points was honestly acquired (no bias), but lack in observation size.

It must be said that it is not really a true/false fallacy. If you estimate the chances for false positives or false negatives with small sample size, you get very high chances that your conclusion is errornous. Hasty generalization over- oder underestimates the chances for being wrong, but in contrast to many fallacies it does not necessarily mean that the conclusion is actually wrong. So if several outcomes are in fact possible and your hasty generalization deduces one specific outcome, the generalization can be in fact true (!).

In fact many scientific discoveries by single persons which looked like measurement errors were in fact genuine (for example cosmic rays), but there are also important looking discoveries which turned out to be dupes (for example N-rays).

Anecdotal evidence is also not zero evidence, but weak evidence (!). Like the Sorites paradox, you cannot assign arbitrarily a 0% probability for an observation because then you never get a valid probability for a higher number of observations.

So the true problem is getting a valid estimate for the probability of the outcome. Let's say we e.g. want to explore an unknown tribe and we don't know anything about them except their existence and if they are friendly/hostile. Now if the first meeting with one tribe member ends friendly/hostile, hasty generalization would come to the conclusion "The tribe is friendly/hostile" when in fact the likelihood is 50%. If we make more contacts, the calculated hostility will very fast converge to a good prediction how friendly/hostile the tribe really is.

The problem is not a small dataset in itself; if only a small dataset exists, it is not a fallacy to say: The probability of an error in our dataset is 24%. The fallacy is over- or underestimating the probability of an error severely.

Answer 5

While not a formal fallacy per se, you could also say that it is availability bias or the availability heuristic, as per Kahneman and Tversky's work, for example Judgment Under Uncertainty: Heuristics and Biases (1982). By the same work it could also be an example of falsely employing the representativeness heuristic.

EDIT: Actually, Kahneman's Thinking, Fast and Slow is probably the more accessible and famous work.