On the determination of scientific consensus

Question

The question:

Are there any philosophers or sociologists who have written about the process in which the scientific community decides that a body of evidence is enough to consider a particular theory true (or at least highly likely, almost indubitable)?

Some background to my question (I'm afraid it got a bit long):

I am a researcher in psychiatry and I've been thinking about how a certain theory or hypothesis comes to be considered as true or proven among scientists. I have read Thomas Kuhn's Structure of Scientific Revolutions and his view seems to be that what hypothesis is considered the truth is not entirely based on empirical data, and it is to a considerable degree a social process in the community of scientists.

This is a good basis which aligns with my experiences, but I'd like to read some works that focus on this particular question. From my own readings of the scientific literature, psychiatry is riddled with hypotheses that the community of psychiatric researchers more or less accept as true (however, there are always some dissidents) even though the actual data do not give a black and white answer. Some degree of interpretation is thus necessary, but researchers seldom seem to be aware of this. Some such questions in psychiatry are: do SSRI's (a class of antidepressant drugs) actually alleviate depressive symptoms (mainstream view: yes)? Does cannabis cause psychosis (mainstream view: yes)? Does treatment with antipsychotic drugs reduce the progression of the psychotic illness over long time (mainstream view: yes)? Does the environment in the home, i.e. parental behaviors etc, have any role in the development of psychiatric illness (mainstream view: no)?

In all of these cases, the empirical data alone are in my opinion insufficient to give an answer to the question. We have to rely on assumptions, inferences and interpretations to arrive at an answer. There are of course many more such questions in psychiatry, and I suspect that this is common in most other scientific areas as well. I will give a concrete example below, though not from psychiatry.

Do homeopathic treatments work? ("work" in this context means work better than placebo). The scientific community at large is confident in saying no, which is a reasonable position. There are in fact a number of randomized controlled studies showing effects of homeopathic treatments in various conditions, though the more well-designed the studies are, the less likely they are to find an effect (this indicates possible bias). Furthermore, there is no plausible way in which the homeopathic treatments could work, given what is currently known about physics, chemistry and biology. This means that if we are to accept the positive RCT's, we also have to accept that there is a yet to be discovered type of energy transmission or something similar, and this seems less likely. The reasonable conclusion is that the positive RCT's are probably due to various biases (including fraud) and that homeopathic treatments don't work.

However, this is clearly not based on a Popperian notion of falsification. That some RCT's fail to find an effect does not falsify the hypothesis, or else we would have to accept that, for instance, SSRI's don't work because there are many trials with negative results. And just like for homeopathic drugs, the incentive for publication bias and some kinds of fraud is enormous with such drugs, sold by pharmaceutical companies and marketed as effective for conditions that affect large numbers of people (depression, anxiety), there is no real difference in the possibility of bias. And the history of medicine is full of discoveries of treatments and preventive measures that have been found to work even though there is no known mechanism that could explain how they work, so that is not an absolute argument against a hypothesis.

So this seems like a scenario that Thomas Kuhn described; the data is insufficient to determine what is true, and instead the community of researchers, one way or another, agrees upon what we should consider as true. In this particular case, I think it's correct to conclude that homeopathic treatments don't work and that the positive RCT's are a result of bias or fraud (and thus that their use shouldn't be supported by state-funded health care, which is an example of why it matters), but I cannot say that this conclusion is the only possible interpretation of the data.

I suppose that the moral of all of this is that we cannot be absolutely certain about anything in science, a view that I think I support, but to avoid total relativism and to use science constructively, we nevertheless need to agree upon what counts as the leading scientific hypothesis and how strongly we should believe it. In my mind, this is what actually happens all the time, but how does this process work in detail? Are there any philosophers, or perhaps sociologists, that have studied this process in greater detail than Kuhn, who mostly hinted about this process.

Answer 1

Philosophy of science has grown exponentially (literally) since Kuhn wrote Structure more than 50 years ago. Today there are substantial literatures on RCTs (and evidence-based medicine more broadly), the formation and appropriate roles for cognitive diversity and consensus in science, the relationship between features of scientific communities and objectivity, conflicts of interest and industry funding in pharmaceutical research (and other fields), and many of your other points. Philosophy of medicine is beginning to emerge as its own subfield, distinct from both philosophy of science and bioethics, and there's specific work on psychiatry (for example, on the DSM) within philosophy of medicine.

I recommend you start by reading the Stanford Encyclopedia entries for philosophy of medicine, philosophy of psychiatry, social epistemology (perhaps just sections 3 and 5), and feminist social epistemology (perhaps just sections 3 and 4). Reading these entries will help you find better conceptual frameworks (compared to Kuhn and Popper) to articulate your concerns and interests, and the references will point you towards major recent books and articles.

Answer 2

From Wikipedia:

In science, a fact is a repeatable careful observation or measurement (by experimentation or other means), also called empirical evidence.

The statement "homeopathic medicines do not work" is not a scientific fact in the strict sense of the word. A scientific fact would be along the lines of "In a study on MMDDYYY, 10 students selected at random for a double blind trial of {insert particular homeopathic treatment} using {insert methodology} resulted in the following observations: {observations}." From these facts, the scientists may conclude that there is no evidence that the homeopathic treatment worked better than random chance/placebo.

Where this gets tricky is that when sufficient evidence piles up we start to use the phrasing "It is a fact that homeopathy doesn't work." This uses the same word, "fact," but uses it in a different way than the formal concept of a "scientific fact."

At this point, it can get a bit murky, because now we have to describe what it means to be "empirical evidence" and such. That's a common issue in philosophy, and can be researched on its own. If I may inject my own personal opinion, I find defining "objective empirical evidence" as "any subjective empirical evidence which one believes it is impossible for any other rational agent to observe anything contradictory" yields interesting effects. For one, it explains why people start referring to "homeopathy doesn't work" as a fact.

Answer 3

The thing is, science does not ordinarily proceed on the basis of proving hypothesis. But of testing them. Here is a cartoon example.

https://xkcd.com/882/

This specific cartoon is talking about probabilistic tests, and the idea that if you do enough tests with a 5% chance of a false result, you are very likely to get a false result. This comes under the heading of "What should you expect when you are expecting to win the lottery?"

There's more to it than that. Kuhn lists these features of a good scientific theory: Accurate, Consistent, Broad Scope, Simple, and Fruitful. But he missed one. It has to be specific. That is, it should not contain vague terms that could have unspecified wide range of meanings.

So statements like "homeopathic treatment works better than placebo" are hugely vague. Random works better than random SOME of the time. Indeed, half of random tests will be better than the other half.

So it's a bit of a mug's game to try to test these things. If you, for example, swallow an entire bottle of "homeopathic sleeping pills" at the beginning of a lecture, then give the entire lecture with no ill effects, as James Randi does a the start of this lecture

https://www.youtube.com/watch?v=c0Z7KeNCi7g

have you disproved homeopathy? Well, no. Because it only says "better than placebo." And it does not specify how much better. But what Randi has shown is it is not such that 30 pills prevents you from giving an animated lecture. Even at his advanced age.

In order for an idea to be called a scientific hypothesis it has to be specific. It has to say "this procedure will produce results in this range with this probability." It has to be specific enough such that you can design a test that will distinguish it from not operating.

So, if the hypothesis were "if you take 30 homeopathic sleeping pills all at once you will die" then Randi disproved it quite handily. If it were "the 50 percent lethal dose for homeopathic sleeping pills is 5 all at once" then Randi would need a few volunteers to do what he did. And if 10 of them, say, all took 30 pills at once, and none showed any effects, then that hypothesis is squashed.

But if it's only "better than placebo" then you can't test this. Because no matter what you do, you can't be sure there isn't a tiny little effect at some lower resolution.

And an un-testable hypothesis isn't a scientific hypothesis.

Answer 4

Scientists have a human perspective on things, including the field in which they are experts. But other things being equal, one’s best bet—particularly when the stakes are high—is to go with the expert consensus, and if a consensus is lacking, you’re better off going with the opinion of the majority of experts. Keeping in mind, of course, that they might, just might, be entirely wrong.

The question of the source of the success of science has been at the core of philosophy since the beginning of modern science.

If viewed as a matter of epistemology more generally, the scientific method is a part of the entire history of philosophy.

Over that time, science and whatever methods its practitioners may employ have changed dramatically.

A better understanding of how science actually works puts us in the position of the sophisticated skeptic, who is neither a person who rejects science as a matter of anti-intellectual attitude nor a person who accepts the pronouncements of scientists at face value, as if they were modern oracles whose opinions should never be questioned.

Too often public debates about the sort of science that affects us all (climate change, vaccines and autism, and so on) are framed in terms of alleged conspiracies on the part of the scientific community on one side and of expert opinion beyond the reach of most people on the other side.< Pigliucci, a philosopher of science at CUNY-Lehman College>

Today, many philosophers have taken up the banners of pluralism or of practice to focus on what are, in effect, fine-grained and contextually limited examinations of the scientific method.

One such perspective has been offered recently by Hoyningen-Huene (2008, 2013), who argues from the history of philosophy of science that after three lengthy phases of characterizing science by its method, we are now in a phase where the belief in the existence of a positive scientific method has eroded and what has been left to characterize science is only its fallibility.

First was a phase from Plato and Aristotle up until the 17th century where the specificity of scientific knowledge was seen in its absolute certainty established by proof from evident axioms;

Next was a phase up to the mid-19th century in which the means to establish the certainty of scientific knowledge had been generalized to include inductive procedures as well.

In the third phase, which lasted until the last decades of the 20th century, it was recognized that empirical knowledge was fallible, but it was still granted a special status due to its distinctive mode of production.

But now in the fourth phase, according to Hoyningen-Huene, historical and philosophical studies have shown how “scientific methods with the characteristics as posited in the second and third phase do not exist” (2008: 168) and there is no longer any consensus among philosophers and historians of science about the nature of science.

For Hoyningen-Huene, this is too negative a stance, and he, therefore, urges the question about the nature of science anew.

His own answer to this question is that “scientific knowledge differs from other kinds of knowledge, especially everyday knowledge, primarily by being more systematic” (Hoyningen-Huene 2013: 14).

Systematicity can have several different dimensions: among them are more systematic descriptions, explanations, predictions, defense of knowledge claims, epistemic connectedness, an ideal of completeness, knowledge generation, representation of knowledge and critical discourse.

Hence, what characterizes science is the greater care in excluding possible alternative explanations, the more detailed elaboration with respect to data on which predictions are based, the greater care in detecting and eliminating sources of error, the more articulate connections to other pieces of knowledge, etc.

In this position, what characterizes science is not that the methods employed are unique to science, but that the methods are more carefully employed.

Another, similar approach has been offered by Haack (2003).

She sets off, similar to Hoyningen-Huene, from a dissatisfaction with the recent clash between what she calls Old Deferentialism and New Cynicism.

The Old Deferentialist position is that science progressed inductively by accumulating true theories confirmed by empirical evidence or deductively by testing conjectures against basic statements;

while the New Cynics position is that science has no epistemic authority and no uniquely rational method and is merely just politics.

Haack insists that contrary to the views of the New Cynics, there are objective epistemic standards, and there is something epistemologically special about science, even though the Old Deferentialists pictured this in a wrong way.

Instead, she offers a new Critical Commonsensist account on which standards of good, strong, supportive evidence and well-conducted, honest, thorough and imaginative inquiry are not exclusive to the sciences, but the standards by which we judge all inquirers.

In this sense, science does not differ in kind from other kinds of inquiry, but it may differ in the degree to which it requires broad and detailed background knowledge and a familiarity with a technical vocabulary that only specialists may possess. Ref.

https://plato.stanford.edu/entries/scientific-method/#DevPra https://thebentangle.wordpress.com/2013/08/03/a-philosopher-of-science-explains-the-importance-of-scientific-consensus/