8

I am almost sure there is a name for the fallacy whereby its exponent tries to apply a very small sample of observation, usually anecdotal evidence, as representative of the population, of which that sample is part, at large.

For example:

Burmese python cannot possibly be dangerous. My friend has two of them and they are both sweethearts.

It's nice to be reserved against hasty generalization based on (however carefully collected) statistics -- but it is laughable to argue one's pathetically minute sample of observation trumps that of, say Pew research and similar (which I think can also be considered hasty generalization but in a non discriminatory way).

What is the name of that fallacy?

  • This is not even about sampling. I believe this has more with wrong inductive logic application. – rus9384 Jun 1 '18 at 18:41
  • 1
    Wikipedia calls it overgeneralization, hasty generalization, fallacy of insufficient statistics, fallacy of insufficient sample, etc. – Conifold Jun 1 '18 at 22:21
  • 3
    Yep, there is a name for drawing conclusions from anecdote - it's called "bullshit" – Mr. Kennedy Jun 2 '18 at 16:05
  • @Mr.Kennedy Actually your claim is itself wrong, anecdotal data is not necessarily wrong. I explained below in my answer. – Thorsten S. Jun 3 '18 at 1:35
  • @ThorstenS.Try true or false instead of right and wrong. The comment does not state that anecdotal data is wrong or even necessarily false. Drawing conclusions from anecdotal evidence, however... it's right up there with commentary based on mis-read. – Mr. Kennedy Jun 3 '18 at 2:43
9

I would say Hasty Generalization still applies, although since the speaker is trying to argue against the general claim, we can call it a Hasty Refutation

But yes, this argument has the further problem of being anecdotal evidence on the basis of personal experience, which means:

  1. It is not a random sample: instead of randomly picking two cases, the speaker carefully chose two that would support their view.

  2. The speaker may well construct the memory of their experience in a way to align it with the view they wish to express ... and thus come to believe. For example, the burmese pythons the speaker is familiar with may not have done anything that hurt the speaker, but that is not the same as being 'sweethearts'. Personal experience is always subject to those kinds of constructive interpretations and 'polishing', whereas this is indeed much harder to do with data from Pew research.

  • 1
    I think this does count as a "sample", but it's a "convenience sample", rather than a "random sample". But I don't know how consistently statisticians use this terminology. – Ian D. Scott Jun 2 '18 at 15:45
  • @IanD.Scott Hmm, interesting, right: I always thought of a sample as something random, but then why would we talk about a 'random sample'? So you may well be right: a 'sample' is not necessarily random. Never heard of a 'convenience sample', but I'll look it up: thanks! – Bram28 Jun 2 '18 at 15:51
  • @IanD.Scott OK, looked up 'convenience sample' .... I see what you mean ... and yet I think that doesn't quite capture it either .... it's isn't really out of 'convenience' that the speaker picked the two cases ... I think the two python cases jumped out to the speaker exactly because they went against the general claim made. – Bram28 Jun 2 '18 at 16:14
  • @IanD.Scott And it also wasn't because the two cases just kind of 'accidentally' landed in the speaker's lap, or that they were the 'only ones available' to the speaker, which is how I see some sources talk about the 'convenience sample'; no, they were the speaker's own experience, which is fraught with its particular set of problems due to psychological interpretation, constructive memory, etc. as I explained in my post. Maybe there should be a particular word to describe this kind of 'sample' ... a 'personal sample' maybe? – Bram28 Jun 2 '18 at 16:16
4

I think it's a "Faulty Generalization" (en.wikipedia.org):

A faulty generalization is a conclusion about all or many instances of a phenomenon that has been reached on the basis of just one or just a few instances of that phenomenon. It is an example of jumping to conclusions.

For example, we may generalize about all people, or all members of a group, based on what we know about just one or just a few people. If we see only white swans, we may suspect that all swans are white.

That includes "Hasty Generalization" (en.wikipedia.org):

Hasty generalization is an informal fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence—essentially making a rushed conclusion without considering all of the variables.

The fallacy is also known as:

  • Illicit generalization
  • Fallacy of insufficient sample
  • Generalization from the particular
  • Leaping to a conclusion
  • Blanket statement
  • Hasty induction
  • Law of small numbers
  • Unrepresentative sample
  • Secundum quid
3

Here is Wikipedia, "Fallacy of composition".

The fallacy of composition arises when one infers that something is true of the whole from the fact that it is true of some part of the whole (or even of every proper part).

This fallacy seems to describe what you are looking for.

  • 2
    Not a fan ... if the parts are the two sweetheart pythons, then whole would be the collection of all pythons as a whole, but the claim is not about that whole collection, but rather about every python within that collection. So, while I agree it's related, I would not see this as a part-whole mistake. – Bram28 Jun 1 '18 at 21:53
0

As already said, it is "Hasty generalization" as subset of the more enclosing "Faulty generalization". For the sake of the arguments below, I simply assume that the observation of data points was honestly acquired (no bias), but lack in observation size.

It must be said that it is not really a true/false fallacy. If you estimate the chances for false positives or false negatives with small sample size, you get very high chances that your conclusion is errornous. Hasty generalization over- oder underestimates the chances for being wrong, but in contrast to many fallacies it does not necessarily mean that the conclusion is actually wrong. So if several outcomes are in fact possible and your hasty generalization deduces one specific outcome, the generalization can be in fact true (!).

In fact many scientific discoveries by single persons which looked like measurement errors were in fact genuine (for example cosmic rays), but there are also important looking discoveries which turned out to be dupes (for example N-rays).

Anecdotal evidence is also not zero evidence, but weak evidence (!). Like the Sorites paradox, you cannot assign arbitrarily a 0% probability for an observation because then you never get a valid probability for a higher number of observations.

So the true problem is getting a valid estimate for the probability of the outcome. Let's say we e.g. want to explore an unknown tribe and we don't know anything about them except their existence and if they are friendly/hostile. Now if the first meeting with one tribe member ends friendly/hostile, hasty generalization would come to the conclusion "The tribe is friendly/hostile" when in fact the likelihood is 50%. If we make more contacts, the calculated hostility will very fast converge to a good prediction how friendly/hostile the tribe really is.

The problem is not a small dataset in itself; if only a small dataset exists, it is not a fallacy to say: The probability of an error in our dataset is 24%. The fallacy is over- or underestimating the probability of an error severely.

0

While not a formal fallacy per se, you could also say that it is availability bias or the availability heuristic, as per Kahneman and Tversky's work, for example Judgment Under Uncertainty: Heuristics and Biases (1982). By the same work it could also be an example of falsely employing the representativeness heuristic.

EDIT: Actually, Kahneman's Thinking, Fast and Slow is probably the more accessible and famous work.

  • Is there a title for Kahneman and Tversky's work? – Frank Hubeny Jun 3 '18 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.