8

Intuitively speaking, multiple independent eyewitness accounts of a single event are more convincing than a single eyewitness account. For example, multiple independent eyewitness accounts of a loud explosion in a remote area (e.g. from different locations and viewpoints) are more convincing/reliable than a single account (e.g. maybe the single witness hallucinated the explosion).

A bit more formally, if we define X as some truth claim about some event, process or phenomenon in the real world, we could say that:

P(X is true | multiple eyewitness accounts) > P(X is true | a single eyewitness account)

However, what happens if we keep the number of eyewitness accounts constant and only change the number of events?

For example, let X = "alien abductions are real", and let Ei be a concrete example of an (alleged) alien abduction. X is a general claim, Ei is a claim about a very specific instance of X. It is clear that Ei entails X. Thus, which of the following probabilities is greater than the others?

  • P(E1 is true | N eyewitness accounts for E1)
  • P(E1 is true or E2 is true or ... or EN is true | one eyewitness account for Ei, for each i in {1, ..., N})
  • P(E1 is true or E2 is true or ... or EN/2 is true | two eyewitness accounts for Ei, for each i in {1, ..., N/2})
  • P(E1 is true or E2 is true or ... or EN/3 is true | three eyewitness accounts for Ei, for each i in {1, ..., N/3})
  • Etc.

In other words, given N eyewitness accounts, what is the optimal distribution of eyewitnesses over specific alleged instances of X that maximizes the probability of X being true? What should be more convincing, 1000 eyewitness accounts for E1, 500 eyewitness accounts for E1 + 500 eyewitness accounts for E2, etc.?

Note: I used eyewitness reports of abductions by aliens as an illustrative example, but the reasoning can be extended to other rare events, such as reports of miracles, paranormal phenomena, angelic encounters, Bigfoot sightings, testimonials from whistleblowers (conspiracy theories), etc.


Related questions

14
  • 2
    You might introduce the concept of "theoretical witnesses": how many people would have observed this event had it occurred, and take the ratio of actual witnesses to theoretical witnesses. I think this would help with your "number of events" issue Aug 16 at 16:27
  • 1
    Well, you'd have to come up with an estimate of how many people could have theoretically been abducted, which I admit is difficult for any "rare" event. I still think it's an important overall statistic though. Aug 16 at 17:19
  • 2
    Consilience or convergence of evidence is helpful. Misinformation can be infectious, like in the Mandela Effect. We look to different modes, contexts, types of source, to converge, eg one sense checking another. The details of alien abduction accounts strongly follow templates from specific films, eg particular descriptions of aliens. But strange encounters, such as with angels, predate scifi.
    – CriglCragl
    Aug 16 at 19:21
  • 2
    Interesting question, but probably would get better answers on math.stackexchange.com or statistics.stackexchange.com ! If a witness claims they have seen event E_1, they might be lying or hallucinating, or someone else might have staged E_1. So you have to account for P(E_1 | eyewitness account), but also for P(X | E_1), which is the probability that aliens exist knowing that a possibly-staged alien abduction event happened.
    – Stef
    Aug 17 at 13:28
  • 2
    You may get better answers on Cross Validated. Under some simplifying assumptions (like equal credibility of reports, independent events, etc.), this is essentially a mathematical question about calculating Bayesian posteriors. The formula for multiple reports of a single event is standard, but I am not sure how they are calculated for derivative statements implied by multiple reported events. You may want to use a less provocative example than alien abductions there, so it doesn't become a distraction.
    – Conifold
    Aug 18 at 12:02

5 Answers 5

2

I will defend the following 4 contentions. I do not claim that they follow from natural law or mathematical proof--they are merely designed to represent actual human behavior:

  1. All else equal, multiple witnesses reporting one event each is more compelling than one witness reporting multiple events
  2. Multiple independent witnesses are more compelling than the same number of dependent witnesses
  3. Increasing the number of events/witnesses will eventually experience diminishing returns
  4. In the real world, "all else" is not equal, and one credible witness is more compelling than multiple less-credible witnesses

--

1. One witness or many: Whistleblowers

The OP cited whistleblowers, which are an effective example to defend contention one. It is common practice in a corporate environment for there to be a small subset of employees who are angry, loud, and dissatisfied. Corporate surveys tend to discount these voices if they are numerically overwhelmed by different perspectives.

In this case, one disgruntled employee who voices numerous concerns is going to be taken much less seriously than multiple employees voicing similar concerns.

Evidence of this can be found in examples such as:

  • Organizational Health Indices, which aggregate large data sets and use averages to compare company performance with peers. In the mathematical summaries of these surveys, one very angry, long response with many examples will be weighted no differently than one very short, satisfied response. One dissatisfied voice will do little to drag down the company's score. Multiple dissatisfied voices--especially if they are dissatisfied for similar reasons--will make a bigger splash.
  • Voting: most political organizations with democratic norms acknowledge that a better result is achieved if each person votes once than if some people vote more than once. So strongly do most nations believe this, that in most jurisdictions, voting more than once is a crime that comes with hefty penalties. Lots of people voting once rather than one person voting many times is an example of placing trust in multiple attestation.

As a result, lone whistleblowers often take significant blowback, even if they are reporting serious violations in good faith--this is why whistleblower protection laws exist.

--

2. How multiple attestation works for historians

Most events in human history are not preserved in any extant records. And most events that are preserved survive in only one source. For this reason, especially in the study of ancient history, having two sources for an event is a very high, very important standard, which historians refer to as the criterion of multiple attestation.

However, if we have two sources and source 2 is quoting source 1, this does not count as multiple attestation. We could just as well write another work today quoting source 1 and claim we now have triple attestation. Source 2 is only effective in corroborating source 1 if there's reason to believe source 2 isn't simply regurgitating what source 1 said.

The most prominent usage of the criterion of multiple attestation is seen in New Testament studies, and particularly in relation to the Synoptic Problem (disclaimer, the linked source is my own work on the Synoptic Problem). If one Gospel appears to be quoting another, this is single attestation. If two Gospels have the same event recorded independently, this is multiple attestation (what fits in which category depends very much on how one solves the Synoptic Problem).

To give one, concrete example from Roman history: Dio Cassius, Orosius, and the Acts of the Apostles all tell us that Emperor Claudius made a decree expelling the Jews from Rome (circa AD 49). Each provides distinct details that could not have been obtained from the other two. From the standpoint of a historian of ancient history, this is the gold standard (we'll call finding an inscription the "platinum standard" =)) - this is a very solid piece of history and the likelihood is much, much greater that the decree happened than that it didn't.

--

3. Diminishing returns: We've heard this story before

Alien abduction is a good example here. There have been so many stories of alien abductions told that many are numb to it. Hearing one more alien abduction story is very unlikely to persuade someone who has already heard 100 of them and didn't believe any of them.

Additionally, as pointed out in other answers, because so many alien abduction stories have been told, people can be influenced by each other's stories, diluting the value of an incremental alien abduction story.

An illustrative example is the JFK assassination (arguably the grand-daddy of all conspiracy theories). For those who believe the lone gunman theory, hearing yet another alleged eyewitness account that proves a conspiracy, or another complex cover-up scheme described, can be exasperating--it's not going to move the epistemological needle. Ironically, even if many of the witnesses & theorists are wrong, it does not necessarily mean all of them are wrong. In this sense, having too many witnesses can be damaging: if 10 people claim they saw a man on the grassy knoll shoot JFK, and 9 of them were mistaken...JFK was still shot by a man on the grassy knoll--but the erroneous witnesses make it the claim less credible, not more.

--

4. Credible Witnesses

In both the ancient history and the JFK examples, the credibility of a witness can overpower (positively or negatively) the quantity of witnesses.

If most of recorded history is preserved by a single source, do we discard most of what we think we know about history? No. We evaluate the credibility of the historian by comparison to other sources of information. If the historian gets it right when we can fact check the claim, we have a greater level of confidence that the historian is correct when we cannot fact check the claim.

In practice, one highly credible witness can provide a compelling basis to believe something even if it is opposed by numerous, less-credible witnesses. Because people tend to consider themselves to be credible witnesses, we have sayings such as "I saw it with my own eyes", where despite receiving criticism for doing so, people believe something others disbelieve, because it is consistent with their own experience (since all evidence arises from experiential evidence, this practice is not wholly irrational).

I'll offer one specific example here as well: what was the last song played by the band of the Titanic? (spoiler alert, we're not going to solve that vexed question here).

This question is interesting and relevant, because most Titanic survivors who spoke on the matter said the band played Nearer My God to Thee, and for many years this was uncritically accepted as correct. However, more recent historians have challenged this claim, largely on the basis of a single witness: Harold Bride.

Bride was the junior wireless operator on board, his profession was one which required attention to detail and a sharp memory, and he was closer to the Titanic in its final moments than most other survivors were (he survived aboard the overturned collapsible B lifeboat (ibid)).

Bride said nothing about the band playing hymns, but spoke of their playing lively tunes.

The Titanic musical debate rages on with no end in sight, but it is a useful example of my 4th contention. Many are prepared to take the word of one, particularly strong witness over and against the majority of witnesses who said something different.

--

Conclusion

Eyewitness testimony cannot be ruled out entirely, as it is the ultimate basis for all data that we have to work with. We look for aggregation and repeatability in order to establish credibility.

On a matter as subjective as judging the accuracy of an eyewitness, we should not expect to find a formula for weighing one claim against another. But in actual practice, all else equal people tend to take more seriously something they hear from multiple sources versus than something they hear repeatedly from one source, but credibility of sources can readily outweigh quantity of sources.

I submit that the most secure epistemological claims are those that come not from quantity but quality of sources.



Which sources are high quality? This requires epistemological evaluation as well...

...loop...

...crash! =)

3

The experience of science is that eyewitness accounts are very low value items in general. That is to say, there is no such optimal distribution as you suggest. Events reported purely through eyewitnesses are going to remain entirely suspect no matter how many times they are repeated or what distribution of persons viewed them each time. It's "The Easter Bunny" all the way down.

Consider, for example, witnesses to E2 who have previously become aware of reports of E1. This could well convince us that their reports are far less credible. Reports of aliens are in the news and popular entertainment. If somebody stumbles in from a corn field reporting little gray people did things to his buttocks, we naturally attempt to perform some type of field sobriety test. And we will do this no matter how many times such occcurs.

What is required is careful eyewitness accounts, careful vetting of the witnesses, and some form of corroboration. Here are just a few.

  • A witness who has passed that sobriety test is more credible than one not so tested.
  • A witness who has some other type of evidence besides only his claims is more credible.
  • A witness who is unaware of, unknown to, and unrelated to a previous events or witnesses is more credible. Though this lack of contact is naturally difficult to be sure of.
  • Predictions of future occurences that are then confirmed are more credible than tales of aliens "who got away." Particularly if the predicted event is observed by "hostile" observers who are attempting to disprove the hypothesis.
  • Physical evidence of some sort can be interogated in ways that a witness cannot. This can come in a large number of forms from photographs, to impact sites, to material, to an actual member of the Asgard Fleet (Stargate reference) coming down to say hello.

Each of these is made more credible by being examined for evidence of self-delusion or fraud and being found innocent. There are a large number of such examination methods from reproducibility to double-blind to looking at the ID info on a photograph, etc.

6
  • 3
    Events reported purely through eyewitnesses are going to remain entirely suspect no matter how many times they are repeated or what distribution of persons viewed them each time. - Does this apply to most of history? Is most of history entirely suspect? Aug 17 at 0:59
  • 1
    @SpiritRealmInvestigator. "Most of history" is not necessarily rendered "entirely suspect" if it is true that eyewitness testimony is unreliable. There are other methods by which conclusions about history are reached. These methods sometimes corroborate eyewitness accounts, sometimes disprove/discredit eyewitness accounts, or sometimes fail altogether. In the absence of any supporting evidence, eyewitness accounts of course need to be treated skeptically (like all material deemed as potential evidence). Aug 17 at 6:44
  • 3
    "Eyewitness accounts are very low value items in general. That is to say, there is no such optimal distribution"??? Existence or not of optimal distributions is orthogonal to how high or low the optimal values might be. Whether multiple reports of a single event produce higher confidence than single reports of multiple events is a well-posed question (in a Bayesian framework, say) whatever the event type or the credibility level of the reports might be. Including "entirely suspect" ones. The "witnesses" may just as well be technicians reporting readings on detection devices.
    – Conifold
    Aug 17 at 7:38
  • "A witness who is unaware of, unknown to, and unrelated to a previous events or witnesses is more credible. Though this lack of contact is naturally difficult to be sure of." This can be made even better if some details of the event are not disclosed to the general public. For instance, the newspaper announce that aliens from E_1 are said to have spikes on their backs, but the exact number of spikes on an alien's back reported by witnesses of E_1 is kept secret by the FBI. If witnesses of E_2 also claim the correct number of spikes on an alien's back, their claim becomes more credible.
    – Stef
    Aug 17 at 13:32
  • @SpiritRealmInvestigator The important word there is purely. History often has quite serious amounts of evidence beyond eyewitness reports. Just one example, we have a variety of physical evidence of the existence of George Washington, including his home, his signature, many of his possessions, and his wooden teeth. We also have corroboration in evidence about other persons and many events.
    – BillOnne
    Aug 17 at 13:51
3

Epistemic value of multiple eyewitness accounts

I don’t know if this is an answer, but here is an example from real life: Babe Ruth’s called shot from the 1932 World Series. Look it up at Wikipedia > Babe Ruth’s called shot.

It is a great story, and relevant here. In the fifth inning of game three of the series, Babe Ruth was behind the pitcher, Charlie Root, with two strikes against him. Ruth then made some pointing gesture which is still debated today. Ruth then hit the next pitch for a home run near the center field flag pole. Did the mighty Babe call the home run before the ball left Root’s hand?

Then followed 80 years of controversy. The eye witnesses, all 30,000 of them, say that Ruth was calling the next pitch. Ruth himself said different things. Apparently, surviving newsreels of the event do not resolve the issue.

The relevance here is that the event and its controversy show the difficulty of the question. Do the called shot supporters prevail just because there were 30,000 of them?

1
  • This is probably a good example of one way you shouldn't interpret eyewitness accounts. 30000 people saw Ruth make a hand gesture and then hit a home run (which are not disputed facts). None of them saw Ruth "calling the next pitch", but rather that's a conclusion they reached based on what they saw. If you're interpreting eyewitness accounts, you should attempt to separate what they actually saw from what they inferred. I've also heard this distinction being discussed in the context of courts of law.
    – NotThatGuy
    Aug 19 at 18:48
3

Epistemology doesn't use concrete mathematics, so you're unlikely to find a distribution that maximizes the probability of X being true (and if you do, I wouldn't be quick to trust it).

Generally speaking, you want at least enough eyewitness accounts for at least one event so you can say that event is true independent of the other events. How many eyewitnesses there are also isn't the only/most important factor, but rather the sum of those eyewitness accounts should be reliable and consistent (with one another, with reality as we know it and with any other evidence we may have for the event), so that the event being true becomes the most likely explanation for those eyewitness accounts.

Beyond that, it doesn't seem like it would make a big difference whether additional eyewitness accounts are for the same event or for other events.

Multiple events that can all be concluded to be true independently is possibly slightly better than a single event, as this is more difficult to explain with e.g. a mass hallucination. Although multiple events spread over a long period of time may be very susceptible to eyewitness accounts being influenced by one another (one person claims to have been abducted by aliens, other people hear about that and that makes them more likely to attribute some experiences they have to abduction), which reduces their reliability greatly.

Alternatively, even the extreme of one eyewitness per event can be trusted if you can compare and corroborate the eyewitness accounts with one another in a way that can't reasonably be explained through other means (e.g. if eyewitnesses give detailed descriptions of the aliens that abducted them, but these match the appearance of aliens in popular media, that would make their accounts unreliable).


Note: there is an important distinction between what people experienced (as in the raw signals their senses send to their conscious brain) and how they interpret that. We could generally say that what people claim to have experienced is accurate (except in cases of (a) misremembering, which is quite common, and (b) intentional lies, which is less common), while we reject their interpretation of that experience. If someone claims to have seen Bigfoot, they may in fact just have vaguely seen something moving in the woods (which we may accept as true), and attributed this to Bigfoot (which we reject). It may not always be easy nor possible to distinguish the "raw" experience with the interpretation, and the line may be blurry, but we should try our best to distinguish them.

We should take their interpretation into account, but we should focus much more on their "raw" experience, because interpretations can trivially be wrong, while "raw" experiences tend to be more trustworthy (although they are far from fully trustworthy, as people can hallucinate, dream, misremember, etc.). Depending on the "raw" experience, we can try to determine how likely their interpretation is to be correct (compared to other possible interpretations).

So it's not strictly a question of "we have N witnesses confirming event A", but rather "we have N witnesses claiming to have experienced B, which they attribute to event A", and then we evaluate whether event A is the best / simplest explanation for those eyewitness accounts (while also considering the possibility of lies or false memories).

16
  • Epistemology doesn't use concrete mathematics - then how can we measure "how sure" we are about some claim being true? Gut feeling? Aug 17 at 18:08
  • @SpiritRealmInvestigator We can be sure things are true through logical reasoning based on the evidence.
    – NotThatGuy
    Aug 17 at 21:08
  • By logical reasoning do you mean deductive reasoning, inductive reasoning or abductive reasoning? Of these 3, only deductive reasoning can give you the logical certainty that you seem to be talking about. But a deductive argument can only be sound provided that it is valid and that its premises are true. So, again, even with deductive reasoning you still need to prove that the premises are true, and so you will have to resort to either inductive, deductive or abductive arguments to defend the premises. And then I can repeat my question: how sure are you that the premises are true? Aug 17 at 22:02
  • How can you quantify how sure you are that the premises of your deductive argument are true? Aug 17 at 22:04
  • 1
    @SpiritRealmInvestigator You don't quantify your certainty of a specific claim, you compare it to competing claims based on the evidence available and how well each matches reality as we know it (and you compare it to similar claims that have more evidence available). This also provides some rough level of confidence. If someone claims they've been abducted by aliens, you compare "they were abducted by aliens" with "they hallucinated", "they're lying", etc. and see which seems the most plausible. It's a judgement call based on reasoning and experience, not a "gut feeling".
    – NotThatGuy
    Aug 18 at 17:29
0

Ok first of all your N is limited by having a finite number of people who are physically able to observe the event, yet you may have more "eyewitnesses" coming out of the woodwork to give their account. Which raises the question of "How credible are eye witnesses". And the answer is not very much. Like they give a biased account that is on it's own not reproducible and that might not even stay consistent if prompted for multiple times.

Also how do you define "independent"? Like as soon as a group of strangers is close enough to each other to be aware of the other strangers existence they are somewhat no longer independent as they'd have more confidence in their account because they weren't the only one who saw it. Not to mention if they had time to talk to each other and sync their accounts. Or hear other people's testimonies before their own.

Which severely limits the possible N, before an increase in N does not result in an increase in credibility. Because of EVERYBODY KNOWS then you'll have "self-correcting" effects on the accounts, rendering them useless despite the raw number.

Also again, those accounts are biased so just because you have a large number of events or a large number of witnesses per event or ideally even both, that does not mean that they've actually seen what they say they've seen. Like they've probably seen something but their interpretation might be way off from the truth. Like take a stage magician performing an illusion and ask the non-initiated audience what happened and despite having lots of eye witnesses the accounts might be completely useless in figuring out the trick even if they align. Because unless they are likely falling for the same distractions and then describe it as magic.

So what you get is not actually P(Ei|...) but just P(E|...), so just the probability of an event and E1, E2 and so on can be independent and just linked to each other because of a bias to expect a sequence.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .