Is there such a thing as weak evidence?

Question

I recently came upon a video by a YouTuber named Alex O’ Connor who talks about how you can have evidence that increases your credence in a claim but not enough to “push it” over 50%. For anyone who’s interested, here’s the video: https://youtu.be/dqEt1D2ciGA

This got me wondering: can you have weak evidence for a claim? What is an example of a claim of which we have weak evidence for? If weak evidence is taken to mean something that supports a claim but isn’t enough to make you believe in the claim, then in what sense is it evidence in the first place, especially in the case of a claim that can only be true or false?

Answer 1

I fully agree with some of the prior answers including causative's excellent answer. But that seems to leave you unconvinced, so perhaps I can offer a different way of viewing this that will help.

Not everything is in fact binary

One of your comments suggests that you think belief should be binary. In certain clear cut areas such as pure first-order logic or most types of formal mathematics (not all types of formal mathematics fit this either as we know thanks to Goedel, but that gets painfully technical immediately), this is true. Each statement is either true or false.

But in many areas of the real world, we are often forced to deal with things that are "fuzzy". For instance, there is the heap problem sometimes more formally referred to as sorites paradox.

In over simplified terms, a heap is not a well defined term and the boundary between heap and not a heap is "fuzzy", but it is useful enough that we use it and similarly fuzzy terms all the time. You can have one person look at a bunch of sand and say it is a heap and another look at it and say it is not. In some cases one would clearly be right, but at the boundary it would hard to be say that either of them is clearly wrong.

Another example is the species problem. It is very hard to come up with a definition of species that is remotely satisfactory. The grade school version that basically says that members of the same species can breed runs into problems almost immediately with things like mules and the fact that no, not all mules are sterile. (It runs into a lot of other problems). When asking if two specimens are from the same species, there are edge cases where you have to first start by specifying which definition of species you are using and even then experts can disagree and they will bring forth evidence on both sides.

For that matter, life is hard to define. Are viruses alive? That is a serious question that does not have a generally accepted answer and will probably depend on your exact definition of life. Is fire? Almost everyone would say no to that, but it is very hard to come up with a definition of life that excludes fire while including everything we think is clearly alive.

You might of course point out correctly that most of my examples arguably come down to trying to come up with better definitions. But that doesn't change the fact that many things sit on a continuum, rather than a binary, and when you are talking about a continuum it becomes very hard to draw firm lines and many statements that seem to be clear will actually be very vague.

The law recognizes degrees of evidence

armand touches on this. But under the law there are various degrees of conviction of belief about evidence. In the USA, most civil cases are judged under a "preponderance of the evidence". That means more likely than not. In other words, the judge needs to be convinced a position is true to act, but it acknowledges that there can and often will be significant doubt. Certain things, such as fraud, require "clear and convincing evidence". This is not well defined other than more than preponderance but less than beyond a reasonable doubt. Most criminal matters are addressed on a "beyond a reasonable doubt" standard, which is also not defined but it is more than "clear and convincing". (Side note: "Beyond a shadow of a doubt" is not a recognized legal standard in the USA. It gets used rhetorically sometimes and shows up on TV often, but is not a recognized standard.)

This is a deeply flawed system as legal commenters recognize, but it works reasonably well for practical matters in court systems. There are often cases where we don't really know the truth. We think that one side is more likely than not, while acknowledging that we do not know for certain, and we work with that. Or sometimes we think that one side has overwhelmingly more evidence than the other while still acknowledging some possibility of doubt.

And in the real world, that is often the best we can do. We will never know for certain. We are just convinced enough to act or not, while still acknowledging that some doubt is possible.

And within that system there is definitely evidence that is stronger than other evidence and the system is generally built on the idea of cumulative evidence rather than finding one smoking gun. Many cases are decided in the real world exactly on who has the stronger evidence cumulatively.

Imagine you have a murder victim that was shot. You find someone two city blocks over 15 minutes later that has gunpowder residue. This is evidence. It increases the likelihood that you found the perpetrator. But it is fairly weak. At least in the USA, there are a lot of reasons you might have gunpowder on you that are completely innocent (and some that are not so innocent but don't make you guilty of that crime). So it is evidence, but its weak evidence. To cross a threshold, you need more.

Or think of if you have a murder victim and you find someone with a clear motive. That is definitely evidence, but it is still fairly weak. You cannot convict someone based on motive alone.

Also, the strength of evidence might change depending on what else you find. If you have a beloved saint and find that only one person in the entire world has a motive to kill him, then the fact that person has a motive becomes stronger evidence (though still not enough to convict by itself). But if you have a widely hated person, there might be dozens with a motive, making it very weak indeed.

Now you might say, and your comments suggest, that your belief might be binary. But the law at least says your degree of belief matters. If you say you believe because you crossed a 51% threshold, that might be a cogent position, but the law wants you to be much more certain than that before you vote to convict on a criminal jury. Your degree of belief explicitly matters in this case. You might in your own mind hold a binary "yes" or "no", but if you are on a jury, the law wants an answer to the follow up question of "Ok, but how strong is that belief?"

Answer 2

You seem committed to a point of view where everything is either believed or disbelieved, with nothing in between, and either there is evidence for a proposition, in which case you believe it, or there isn't evidence, in which case you don't believe it.

When given situations where exact probabilities can be calculated, and are intermediate between 0 and 1, you dismiss this as only being about probabilities and not beliefs.

I think I have a scenario that you won't find so easy to dismiss. Imagine there is a box filled with thousands of balls of initially unknown colors. You may press a button which will cause a machine to shake and stir the balls to thoroughly randomize them, and then select one at random, which the machine will show to you and then drop back into the box.

You discover, after pressing the button a few times, that all the balls shown to you are red.

At issue is the question, "are most of the balls in the box red?"

How many times will you press the button, always getting a red ball, before you decide the answer to that question is "yes"? If you press the button a million times and always get a red ball, would you by that point determine that yes, you believe most of the balls are red?

A single drawing of a red ball would not convince you, but some number N of drawings would pass the threshold to convince you. So each drawing must provide some small amount of evidence in favor of most of the balls being red, which eventually adds up until you decide yes, that's enough evidence.

Note that there is never perfect certainty - it is still possible, even after a million drawings of red balls, that the red balls might be a minority that have only been drawn by chance, and that green balls or yellow balls make up the majority. But at a certain threshold of evidence you will believe red balls are the majority.

Answer 3

There is absolutely "weak" evidence. Or at least the notion that some evidence is more conclusive than others.

See these references to the Evidence Based Medicine notion of "hierarchy of evidence":

• https://libguides.mssm.edu/ebm/hierarchy
• https://libguides.winona.edu/ebptoolkit/Levels-Evidence

The articles are a little technical and if you're not already familiar with study design, they may not make that much sense. But the idea is that some scientific studies are more "rigorous" than others. ("Rigor" is the amount of control for "bias," where "bias" is any systematic skewing of results, not just the human prejudice often equated with "bias.")

Why not always use the most rigorous kind of study? Many reasons, but expense and ethics are probably the two biggest reasons. Sometimes it wouldn't be ethical to subject humans (and some would say sentient animals) to the kind of prospective, interventional studies that are near the top of the rigor hierarchy.

All the above has to do with medical studies. Studies in the physical sciences can also be ranked on rigor, even if the types of studies used don't match the EBM hierarchies in the articles above.

But even in law it's recognized that some evidence is better than others. There are Standards of Proof used in different kinds of cases such as:

1. "Beyond a reasonable doubt": (used in criminal cases)
2. "The preponderance of the evidence" (used in civil cases)
3. "Clear and convincing evidence" (used as a stronger standard in some civil cases)

The field of epistemology deals with quality of evidence and "How do you know something?" or "What does it mean to know something?" I am not trained in epistemology, but I am involved in medicine, was familiar with the concept of Evidence Based Medicine and the "Hierarchy of Evidence," and wanted to present that as a concrete of example of strong vs. weak evidence (with shades of gray in between).

Answer 4

I thought that clip was pretty good.

There's a simplistic picture that science is about finding clear evidence for each positive assertion, but as Popper noted the actual task is discriminating between the limited hypotheses that can account for all the data, and that can come down to a very small fraction of relevant accessible data predicted by it. The perihelion of Mercury alone almost, was enough to overturn Newtonian gravity and the expectation of an ether, even though Newton's theory had made many succesful predictions, and is still used within certain parameters of speed and gravitational field strength.

Compound inference, is actually very powerful. I'd suggest it crucially impacts our considerations of the issues of Solpsism, and Other Minds. Like compound interest, our intuitions about likelihoods and profit start to fail pretty quickly, because we rarely have to deal in everyday life with true exponentials.

But. It's going to be very hard to recover a specifically Christian god, who responds to prayers. Deism, sure. Or Pierre Teilhard de Chardin propsed the Omega Point, that infinite calculations being done in finite time means an unlimited simulation us possible. Or Penrose's Conformal Cyclic Cosmology suggests the final distribution of matter could effect the next iteration of the universe, opening up the scope for the minds and cultures that survive and coalesce to have a kind of feedback loop we could equate to an imminent future collective intelligence, absent in our era.

There I'd say Durkheim's sociology of religion, provides a better account than approaching speculative theology as a science question, like to solve the Fine Tuning Problem. Durkheim described religion as the binding together of communities by enactment, of shared attitudes to sacred things. That isn't just about epistemology, it's about community. And we see it in how binding to certain values about publishing work, peer-review, double blind control trials, and replication, bind together the scientific community - challenge the enacted values, challenge the coherence of the community. Shape the individuals views through adherence to a creed, and you shape the scope of what the community is capable of.

Making sacrifices, was probably the most widely shared religious practice the world has known. Then relatively suddenly, it seems to have ended. I'd look to it as about the binding power of spectacle, of feasting, being replaced by the power of books, described as the emergence of the Axial Age. The complexity of a unit, an individual, was fundamentally shifted by literacy, and so the mechanism and capacity of religious systems also shifted.

In Judaism they account for this, describing the end of the era of prophecy, the Nevuah, as part of the maturing and increasing autonomy of humanity. For Adam, Noah, & Moses, there were binding rules that we can understand as forming cohering and binding the community that enacted them, and so as shaping the capacities of those communities. It is this view, of a religious community as a living cultivated organic whole, that I think is necessary for understanding the importance of a tradition, and the community that enacts it, over theology as a kind of speculative cosmology. We can picture the fiats with the deity, as a precursor to rights-discourse, and the idea of universal laws, which have enabled specific social capacities.

When epistemological statements, like faith in a deity, are considered to completely overlap with cultural enactments like rituals and festivals, then a change in scientific data can only result in an end to enacting the practices. But, atheists still love Christmas. The basis of a succesful cultural festival or ritual or observance, is not in whether the theology it's based on is held to correctly reflect reality, but on it's social impact, it's power to bind and cohere us, and reinforce our sense that we want to enact common values even when they don't benefit us personally, but which adhering to increases the capacities of a society that values them.

So there is the compound value of the epistemologically unlikely. A lived community, enacting positive and useful values, can succeed better even if the basis for determining the need to observe them is held scientifically not to be true. To live well together ultimately isn't about what's true, but what works, and sometimes that can mean undertaking a ritual we don't need to believe in, because it enacts a continuity which changes the scope of what our community is capable of.

Answer 5

There's certainly such a thing as weak evidence.

Suppose our friend Natalie has two 120-sided dice. (Yes, there are such things!) One of the dice is a fair die; the other die is a loaded die which can't land on the number 120, but has an equal chance of landing on any of the other 119 faces.

Natalie chooses one of the two dice at random (equal probability) and hands it to us. We don't know which die she gave us, but we're going to roll it once. Let U denote the event that this die is the unfair die, and let S denote the event that when we roll the die, the result is something other than 120.

First, we calculate

P(S) = 1/2 * (119/120) + 1/2 * 1 = 119/240 + 1/2 = 239/240.

Bayes' law tells us that

P(U | S) / P(U) = P(S | U) / P(S) = 1 / (239/240) = 240/239.

Since P(U | S) is greater than P(U), S is evidence for U. However, since the ratio between the two is only 240/239, S is only weak evidence for U.

Answer 6

You can find plenty of easy to grasp exemples with criminal cases. Both defense and prosecution come forward with evidence, say a fingerprint for the prosecution and an alibi for the defense, but in the end a verdict of guilty or non-guilty will be issued. That clearly means the evidence for one side was weaker than that of the other side.

The reason is evidence comes in bundled with assumptions (see Dune-Quine thesis) that provides them relevance, and each of those assumptions are a weak point from which this evidence can attacked and discarded.

Let's say a witness says "I saw Armand drawing graffitis on the street corner", this is evidence that i did the deed, as it could convince people that it is the case. Yet, this testimony comes with a bunch of assumptions:

• The witness could be lying.
• The witness could have bad eyesight or it was at night and confused the culprit with my likeness.

Some of those assumptions can be tested ("was it at night?" "was the corner lit?" "was the culprit covering their face?" "Don't you have this personal grudge with Armand that could motivate you to lie?"), and then become as many point of attack. Some can't be, typically the motivation to lie, and therefore we need to assign it some form of probability.

What we see here is, the most assumptions there are linked to a piece of evidence, and the more difficult to assess those assumptions are, the weaker is the whole piece of evidence.

An exemple given in the video is revelation, the feeling a believer has that God is speaking to them, guiding their moral feelings and actions. This comes with the assumption that this feeling could not come from anything else than God. This is typically the kind of assumption that can't be tested, so we are limited to evaluate its probability a priori. For the already believer the assumption is solid, since they already believe. But to the non believer it is very weak.

Answer 7

The other answers appear to leave you unconvinced. Here’s a totally different approach (that veers more into math and science, but that doesn’t mean it’s irrelevant).

Regarding causative’s answer, you appear to believe that a few samples supporting a hypothesis give no confidence, while a certain number (300 in that case, though I understand you don’t mean exactly that number specifically) gives you total confidence.

Now, let us assume you and a statistician are each given $1000, and you’re betting (against the bank/house) on the hypothesis that all the balls are red. If 2 balls are drawn and are red, the mathematician will bet some amount of money based on their calculations, while you will bet none, since you have no confidence in the hypothesis based on the evidence. If this is done many times, the statistician will make a net profit, because they know how to quantify strength of evidence (here is not the place for the gory details). They’ll beat you.

Likewise if 300 balls are drawn and are red. You’ll bet everything (since you feel certain) and may eventually lose it all, while the statistician will always keep some money. Likewise for any amount in between.

This doesn’t have to be just a thought experiment. You can try it yourself with a statistician, fake money, and an urn of balls.

Update: belief is not simply based on what one would bet in a repeatable experiment. At a very layman (and probably slightly wrong) level, frequentist statistics deals with a class of events (sort of long-run frequency), while Bayesian statistics deals with belief in the single next event.

I recommend you read The Signal and the Noise, by Nate Silver. It deals exactly with these questions, from betting money on single sporting events that will not occur again, to things like protecting against earthquakes stronger than have ever occurred. Again, the point is that those who accept a spectrum of evidence evidence strength and act accordingly will outperform those who do not, which has been demonstrated in practice.

Answer 8

Cutting to the chase:

If weak evidence is taken to mean something that supports a claim but isn’t enough to make you believe in the claim, then in what sense is it evidence in the first place, especially in the case of a claim that can only be true or false?

Something that tends to support a claim, or a body of such things, is the very definition of "evidence". It's the consensus of English speakers on the meaning of the word. Being sufficient to convince anyone is not a required characteristic for evidence (as the consensus understands the term), nor am I aware of any specific communities that use the word in a substantially different sense. So to answer the question directly: in every sense.

We recognize that some evidence is more persuasive and some less. And we recognize that some evidence has little persuasive power on its own, but contributes to the persuasive power of a larger collection of evidence. We use the concept of "strength" to describe these related dimensions of the persuasive power of evidence. This is, again, a question of the consensus meaning ascribed to words.

The meaning you seem to want to apply to "evidence" seems closer to that of "proof".

I recently came upon a video by a YouTuber named Alex O’ Connor who talks about how you can have evidence that increases your credence in a claim but not enough to “push it” over 50%.

I think the idea of pushing it over 50% is a bit fraught. It's a lot easier to judge the relative strength of evidence than to categorize evidence as "weak" or "strong" in an absolute sense, and it's not clear to me that the amount of evidence required to flip me from "more likely false" to "more likely true" is a good criterion. In fact, that flip could occur for very weak evidence. Its more a question of confidence, in either direction, and I would not consider evidence that inspires only 50% confidence in a given result to be very strong.

This got me wondering: can you have weak evidence for a claim?

Yes, of course. And we deal with weak evidence all the time.

What is an example of a claim of which we have weak evidence for?

Pretty much every scientific theory that we now accept as a law (which is not the same thing as "true") is one that at one time we had only weak evidence for. For example, Newton's law of universal gravitation, which says that any two objects with mass attract each other with a force that is proportional to the product of their masses and inversely proportional to the square of the distance between them. We now have made quite a lot of measurements that are consistent with that, within the precision of those measurements, but at one time there was only weak evidence.

And, of course, we now think that Newton was only approximately correct, based on newer, more general theory and additional observations. That was controversial at one time, which goes to show you that evidence is about what we believe, not about what is true.

Answer 9

If you want to call something "weak evidence", I would say that means that its probability of proving a claim is lower than your threshold of certainty; That is, the probability for which any claim with a higher probability of being true is something you are willing to believe, and anything with a lower probability of being true is something you are not willing to believe. Many of the answers so far have hypotheticals, mine is no exception: I see a rock above a cliff one day, and come back the next day to see a similar looking rock at the foot of a cliff, and claim that the rock fell to where it is now. I conclude I should not build a house underneath rocks on cliff tops. In this you should consider:

• Is the observation correct? Maybe the rock I observed on the first day is not the same as the one I observed on the second day, maybe I mistook something else for the rock atop the cliff, maybe the cliff I saw on the first day is an entirely different cliff from the one I went to on the second, or perhaps the cliff and rock are merely a philosophical contrivance and never really existed. At a certain level though, we would need to trust at least some of our senses to come to any conclusion, or we would end up in Decartes' "what is true?" dimension.
• What is the likelihood that the claim follows from the observation? In this case, perhaps the stone at the foot of the cliff is a totally different one, and the other stone remains at the top of the cliff. This contradicts the current observations, so it is unlikely. Perhaps a gnome teleported the stone from the top of the cliff to the foot of it. This is consistent with the current observations, but it is not a previously observed mechanisms for stones to travel between locations, so this is also unlikely. Perhaps the stone was moved slightly due to erosion, and then gravity pulled the stone to its current location at the foot of the cliff. This is consistent with the current observations, and the claim adheres to previous evidence that exceeds our threshold of certainty, assuming we've thrown stones and seen them fall back to the ground before, so we can deem the claim likely to follow from the evidence.
• What are the consequences of our evidence being accepted but incorrect, or being rejected but correct? In this case, I may choose to reject the claim and associated evidence that stones above cliffs may fall, but if I reject it and the claim was right, my house would be damaged and I myself might be harmed. Conversely, if I choose to accept the evidence but it is wrong, there is not much resultant harm; My house wouldn't be under a cliff with rocks, but that is unlikely to be a hindrance to me. In this case it would be wiser to shift my threshold of certainty lower than it might otherwise be. You'll see this historically in some journals, that set a certain "confidence level" to serve as the bar for what is statistically significant, in some cases 95% or higher. 95% might be fine for a journal where the consequences of incorrect conclusions is low, but for say, the defect rate of aircraft parts, you'd want a confidence closer to 99.999% (1/10000 chance of a defective part), because the stakes are a lot higher.
• How skeptical or gullible am I? Where your threshold of certainty sits before taking consequences into account will largely depend on how skeptical/gullible of a person I am, and can range quite a bit between people.

So if (chance of observation being correct) x (chance of claim being the cause of the observation) +/- (severity of being wrong) > (default threshold of certainty), you've got strong evidence. Naively it would then follow that everything else is weak evidence, but we also have a notion that evidence can be so weak as to imply no connection at all, so there seems to be for most people a "threshold of maybe", usually around 50%, but as subject to personal scrutiny as threshold of certainty, that signifies something is conceivable but further observations or explanations are necessary to move it beyond the threshold of certainty. I'd say this is where weak evidence lies: between one's threshold of certainty and threshold of maybe. Weak evidence is not worth believing in outright, but it's not worth discounting entirely either. Keep in mind also that "evidence" and "what is true" are necessarily separate; If we know a piece of evidence to be 100% true in supporting a claim, then the notion of evidence as something that convinces us of a claim breaks down, and such evidence is almost non-existent in our current universe.

Answer 10

Let's say you have a normal deck of card, and I try to draw the Ace of Spades. I draw one card at random. Do you think I have the Ace of Spades ? Probably not, you think there is a 1/52 chance that I have it.

Now you are allowed to peek at one of the remaining cards at random, and you discover the King of hearts. So you learned that I don't have the King of Hearts. Does it change anything, is it enough to convince you that I have the Ace of Spades ? No, but now you think that there is a 1/51 chance that I have it.

If you ever discover the Ace of Spades this way, it is absolute evidence that I don't have it. But every time you discover another card, it increases your belief that I have it (1/50, 1/49, 1/48 etc...), and we call this weak evidence.

On a side note, a good way of measuring your belief is by imagining a betting situation, but not just an equal, one to one betting : if you are willing to bet (that I don't have the Ace of spade) 1$ in hope of gaining 1$, it means that you believe (that I don't have the Ace of Spades) with more than 50% confidence, and you should absolutely do that at the start. If you are willing to bet 999$ in hope of winning 1$, that means that you believe (that I don't have the Ace of Spades) with more than 99.9% confidence, and it would be a mistake to do that. By asking these questions (assuming we ignore aversion to risk), you can determine how confident you are in something, and you can see the effect of weak evidence in the scenario above.

Answer 11

When we have incomplete information, we often employ the notion of 'evidence' in order to assess what might be true. With more complete information, we might realise that what we once considered to be evidence was not in fact evidence at all. Our habit is often to conclude that a piece of information is 'weak' or 'strong', when in fact we are describing our ability to determine whether a given piece of evidence constitutes evidence at all.

If we take a simple definition of evidence as something like:

An available fact or body of facts or information indicating whether a belief or proposition is true or valid...

....then 'weak evidence' equates to something like 'weak facts', which doesn't make much sense.

Fingerprints of the accused upon a drinking glass at the scene of the crime might be admitted as 'evidence' by a court of law during a trial. It is a mistake though to consider this as 'evidence of the accused's guilt' (weak or strong), because it might be later determined that the glass with the fingerprints was placed there by somebody else, and that the accused was actually at a different location at the time of the crime.

So, you are right. The fingerprints were never actually evidence of the accused's guilt. The only evidence that they provided was that at some point in time, the accused's prints had been left upon a glass which was found at the scene. This goes some way to explaining why even substantial amounts of circumstantial 'evidence' is sometimes deemed insufficient to prove guilt. None of it provides the final 'nail in the coffin'; it doesn't pass the threshold to move a case beyond 'reasonable doubt'.

You make a good point, that evidence is either evidence or not. The task is to properly identify the precise nature or evidence which is provided by a set of facts. We must ask ourselves, when presented with claims:

What is this 'information' actually evidence of? What does it really tell us?

Answer 12

Yes. There's a lot offered in responses, but I think the best way to make the point is to appeal to the fact that evidence is that which allows justification of a belief to promote it to knowledge.

Some evidence necessitates a conclusion. Other evidence merely suffices to reach a conclusion. Thus, one simple way to consider evidence weak is that it merely suffices and does not necessitate.

Take for example that you leave your cookies on a plate in your den, and when you return the cookies have been eaten. If in that short moment when you were gone, there was only your trusted Great Pyrenees in the den in your absence, missing cookies necessitates drawing the conclusion your dog has eaten your cookies, particularly because you two are the only living, breathing animals in the house.

Yet, imagine instead you have three dogs in the den, the Great Pyrenees, a Golden Retriever, and an Irish Wolfhound. When you return, the evidence that the Great Pyrenees has eaten the cookies is weak, but it is suffices to draw the conclusion; having multiple dogs means it is no longer necessary. You reach the conclusion one of the three dogs did it, and assign a probability of 1/3 to the event. So, you go to your home security camera and review the footage to find that your child passed through the den in your absence, and grabbed the cookies and ran out of the room. Now, you are confronted with a defeater. Missing cookies does not suffice to claim that any of your dogs ate the cookies.

Thus, recognizing that some evidence leads to knowledge without doubt, and some evidence suggests and might be defeated means that evidence comes in at least two classes (one could cook up a taxonomy like in law) to extend this thinking.