Can one speak unambiguously of "the" scientific method?

Question

When people in general discuss science, they talk about the scientific method as if it were a fixed and universally agreed-upon principle. In a show I saw recently by Neil DeGrasse Tyson, he explicitly states that it was invented by Ibn Al Haytham in the 10th~11th century. The colloquial definition of the scientific method seems to be along the lines of:

• Emit a hypothesis.
• Run some experiments.
• Accept or reject the hypothesis based on the results of the experiments.
• Improve or come up with a new hypothesis.
• Repeat the hypothesis-test cycle until the phenomenon in question has been properly understood.

This seems to be an informal description of verificationism (possibly interpreted to include falsificationism).

In philosophy of science, on the other hand, the demarcation problem (how to tell science from pseudoscience) is still an open question. Although there is a general consensus among scientists that fields like the intelligent design idea or astrology are pseudoscience, there is no accepted demarcation criteria and those that have been proposed are all either too strong or too weak. String theory notoriously fails certain demarcation criteria. There are also borderline cases with practical relevance, such as in alternative medicine (acupuncture, herbal treatments, etc.), or psychoanalysis and psychotherapy.

The problem of underdetermination of scientific theories also still stands, as well as the problem of induction, as exemplified by Hempel's paradox (I understand that demarcation, underdetermination and induction are related to each other).

Given these open issues in philosophy of science, is it legitimate to speak of the scientific method (or methods), as if it were an established result that has been fixed once and for all?

Are laymen simply abusing language when they speak of the scientific method, simply because they are unaware of the complexities that arise in philosophy of science?

What exactly is the relationship between the scientific method and the demarcation problem? Did Feyerabend (and maybe Kuhn) question the existence of a fixed scientific method, given the unresolved issues with demarcation?

Answer 1

I think one can talk about "the" scientific method as long as it is understood that it is a broad outline rather than anything like a clear prescription. Specific sciences in specific periods provide something more along the lines of the latter, but such standards and best practices are explicitly understood as field specific and revisable, at least ideally. Perhaps, "the scientific approach" would be a better name than "the scientific method".

Several bullet points in your general outline of the method are highly indeterminate. How does one emit or come up with a new hypothesis? Doing it by exhaustive search or at whim is practically hopeless in most cases, in fact discovery is the most non-trivial part of doing science. There are many written and unwritten heuristic guidelines, but none of them are a permanent part of the method. Accepting/rejecting a hypothesis is also non-trivial. Popper's falsification criterion strictly speaking applies to theories, not individual hypotheses, and which part of the theory is to be rejected in response to a falsifying test is not always clear. Even if applied to hypotheses it does not straightforwardly translate into statistical hypothesis testing. The traditional Fisher and Neyman-Pearson significance testing has come under heavy criticism in soft sciences recently on the grounds that "statistical significance" is often a poor guide to scientific significance, see Hypothesis testing: Fisher vs. Popper vs. Bayes.

There is plenty of vagueness in what science is and is not, but we know from the Sorites paradox, a.k.a. the line drawing fallacy, that no bright line of demarcation does not imply no difference between science and pseudo-science. We will know it when we see it, but there will be borderline cases. Popper's falsification criterion proved too restrictive because multiple parts of a theory can be revised in response to a falsifying test, not just a single hypothesis. But unrestricted holism makes everything retainable "come what may", and is too broad. But if drawing a line between a few grains and a heap in Sorites is still debated how can it be easier with science.

Lakatos's more holistic revision of Popper (replace degenerative research programme with a progressive one) explicitly gives no prescription on when a degenerative programme ought to be replaced. According to Hacking, Lakatos not so much prescribes a method to scientists as gives advice to historians on how to rationally reconstruct scientific developments of the past. Rationality requisite for science can only be judged in retrospect, after the benefit of hindsight made the methodology clear. And this is quite in tune with Quine's naturalized epistemology, which subjects all methodological rules to the same re-examination as the science they regulate. Another interesting approach that grew out of polemic with Kuhn is the structuralist programme, see Architectonic for Science by Balzer, Moulines and Sneed.

String theory is in a unique position with respect to hypothesis-test cycle because it can not run it through no fault of its own: probing energy levels where the Standard Model and General Relativity fail will likely remain practically unfeasible for decades (as predicted by the Standard Model itself, not by string theory). As is often said, modern physics is the victim of its own success. Johansson and Matsubara give detailed methodological analysis of string theory from various empiricist perspectives (positivists, Popper, Kuhn, Lakatos, Quine) in String Theory and General Methodology.

Answer 2

No, "the" scientific method does not really exist. Feyarabend has argued that historical case studies do not support the idea of a unique "scientific" method and, further, that such an idea is 'pernicious'.

The idea that science can, and should, be run according to fixed and universal rules, is both unrealistic and pernicious. It is unrealistic, for it takes too simple a view of the talents of man and of the circumstances which encourage, or cause, their development. And it is pernicious, for the attempt to enforce the rules is bound to increase our professional qualifications at the expense of our humanity...All methodologies have their limitations and the only 'rule' that survives is 'anything goes'. Against Method

Perhaps Feyerabend's view can be understood in analogy with the idea of a single Logic: today we readily accept that there are many logics and no apriori criterion which form should be specifically used. Most people however are still convinced that they reason and act according to logic, assuming it is the same one known since Aristotle, even if most of their questions do no get a yes/no answer (but rather a 'dunno').

Answer 3

From my questions I've asked on the topic, The Scientific Method is understood to be the process you describe, perhaps omitting the last step. However, not all results acquired via The Scientific Method are accepted as legitimate by The Scientific Community. Attached to The Scientific Method is a prologue: what sorts of assumptions you made before starting the experiment. Also attached to The Scientific Method is an epilog: how you aggregate the piles of results produced using The Scientific Method into usable knowledge about the world. I find the demarcation problem is mostly in that prolog and epilog. Science, as a whole, includes both this prolog and epilog, not just The Scientific Method.

One thing to remember is that the results of The Scientific Method can never prove a hypothesis to be true. They can merely demonstrate that an existing hypothesis is more statistically false than this new one. The abductive step of converting from this process to knowledge about how the world works is not part of The Scientific Method, though it is clearly part of the greater concept of science itself.

If you try to define The Scientific Method to include these steps, then I agree that it ceases to be unambiguous. The difference in opinions of Kuhn and Popper should be more than sufficient to demonstrate just how much ambiguity there is.

Answer 4

The point of the scientific method is to distinguish theories that match reality from theories that do not match reality. To that end, a properly methodological approach will work. But with any methodology, shortcuts can be taken, which sometimes make no difference and other times will invalidate the method. Weaknesses in the scientific method all stem from which shortcuts are being taken.

Hempel's paradox is a very useful tool in considering how to do science correctly. The basic idea of the scientific method, as you outlined above, is to start with a single hypothesis, and test to see whether it matches reality. The problem is that if you pick the wrong experiments, you can confirm that a poor theory looks good. A more rigorous approach would be to take multiple theories and try to compare them. This leads to an approach where Hempel's paradox is no longer a problem.

Hempel's issue is that the theory "All ravens are black" is the same as "All non-black objects are not ravens". The two statements have identical truth values. But the obvious approach to test the two theories differ. For the former, you would find ravens and look at their color. For the latter, you would look at non-black objects and see whether they are ravens.

But what if you wished to compare multiple theories. "All ravens are black" is the same as "100% of ravens are black." That theory could be compared to "0% of ravens are black." Looking at a single raven would distinguish between the two. But since most everyone has seen a black raven, a more interesting comparison might be with the theory "90% of ravens are black". An experiment where you look at a single raven would obviously not be useful to distinguish between those two theories, since the most likely result (a single black raven) would be consistent with either theory. But if you examined 200 ravens, there are no results that would be consistent with both theories (to 1 part in 1 billion).

Likewise, everyone has seen lots of non-black objects that are not ravens. If you wished to compare "100% of non-black objects are not ravens" with "99.9% of non-black objects are not ravens", no one would care, because we already know that ravens of any color make up far less than 0.1% of objects that exist.

So, in summary, if you start with multiple theories, write them down ahead of time, and compare their predictions, then you can determine which experiments can usefully distinguish between the different theories. This approach leads straightforwardly to determining which experiments are most likely to yield useful information and also other details, like how large a sample size would be needed.

I would say this approach is the "true" scientific method. People will often take shortcuts (e.g. not identifying ahead of time multiple theories they wish to compare), and when they do, those shortcuts sometimes invalidate the results.

But the simple fact is, if you take the approach above, at the end of the day, you will have a definite answer "Theory A matches reality better than Theory B" (or vice-versa). This is the goal of the scientific method. The fact that it will work is almost a tautology.

Answer 5

In philosophy of science on the other hand, the demarcation problem (how to tell science from pseudo science) is still an open question. Although there is a general consensus among the scientific community that topics like Intelligent Design or Astrology are definitely pseudo-science, there is no accepted demarcation criteria and those that have been proposed are all either too strong or too weak. String theory for example notoriously fails certain demarcation criteria.

Demarcation between science and non-science is greatly overvalued because people have an irrational idea that science is an authority. If something they call "science" says X, then you shouldn't question X, and if you do you're a heretic and a moron.

There is no scientific method per se, as pointed out most persuasively by Popper, who pointed out why there couldn't be such a method. The problem is that how you came up with an idea is irrelevant to assessing it. You could get high and stare at the ceiling, you could be inspired by a dream, you could have the idea while knitting. What does matter is how you assess the idea. You should look for ways it might be contradicted by experiment. You should consider whether it contradicts current explanations and if so look for ways of solving that problem. You might call an idea science if it can be tested and is not ruled out by explanatory problems. See, for example, "The Fabric of Reality" by David Deutsch, Chapters 1,3,7.

My understanding is that string theory is not currently testable. You could say it is unscientific if you want. But then you should ask whether this makes any practical difference. Just about any theory of quantum gravity will be difficult or impossible to test. That doesn't mean people should stop working on that problem.

There are also border cases with practical relevance, such as alternative medicines (Acupuncture, Herbal treatments,...), or psychoanalysis and psychotherapy.

Some alternative medicine claims could have good explanations and be testable for all I know, but I doubt it. Psychoanalysis and psychotherapy are at best moral instruction and have nothing at all to do with science, see "The Myth of Psychotherapy" by Szasz.

The underdetermination problem is only a problem if you expect scientific theories to be determined by data. No scientist worth his salt would adopt that position because any scientific theory refers to unobserved and unobservable events as part of the explanation for what he does see. Both the underdetermination problem and Hempel's paradox are solved by rejecting the idea of induction root and branch. There are many other reasons for doing that as pointed out by Popper in "Objective Knowledge", Chapter 1, "On the sources of knowledge and of ignorance" and other material in "Conjectures and Refutations", "The Logic of Scientific Discovery" and "Realism and the Aim of Science".

Feyerabend and Kuhn are worthless. If you want a refutation of Kuhn see the title essay in Popper's book "The Myth of the Framework" and the introduction to "Realism and the Aim of Science". If you want a refutation of Feyerabend read his stuff and try to take it seriously as an explanation of how the world works.

Answer 6

Science is what scientists do. At root.

It is weakly emergent, and subject to contention around edge cases. This could be compared to uses of the word game. That language is use does not mean there aren't very widely used heuristics about how to use it, but applies to edge cases, and to understanding how language derived progressively from non-language. Similarly with science. And this is why narrative descriptions of the emergence of science are key to describing it.

Where Aristotle was wrong, and how proving that helped raise up 'the book of nature' over the authority of other books. How Isaac Newton's devotion to alchemy, and declaration that his work on it was more important than his physics, didn't make it science - it was already disreputable and partially illegal, and he never provided compelling results to sway the scientific community like he had on physics. His prestige was not enough, even as arguably the greatest ever scientist. One of Darwin's greatest, and underrated contributions, was to be among the most prolific letter-writers in science. This helped develop and cement the idea of collected letters, which became journals. Ignaz Semmelweis testing of handwashing between morgue & labor ward, and John Snow's ending a cholera outbreak by removing a pump handle, and the failure for decades to act on them, showed how scientific values had not spread enough to counteract other values - indeed this is always a risk & source of scandals,and to be watched for and guarded against by oversight & accountability structures (strong institutions really help good science, just as they do good government).

It's interesting to look at contention in action. Soviet agricultural science & Nazi physics were communities divided from the international norms like peer review & falsifiability. Claims of superluminal particle speeds & net-energy creating devices are met with ever increasing amounts of scorn, enforcing an 'extraordinary evidence' bar that helps prevent time-wasting. Work on human germ-line cells in China and Korea, has been linked both to fraud, and violation of ethical norms about the standards of safety expected in animal models first, and participation in international ethics discussions before proceeding, which led to some scientists being cut-off from the international community and in some cases sacked. It may be that a scientific community separate from the international one emerges in this field in the future with it's own norms & regulatory regime.

Power & economic success has repeatedly backed up the norms of what has become science, chief of which is counteracting cognitive biases' by settling disagreements with evidence. Following the Needham Question about why the technologies that defined the emergence of the modern era (gunpowder, magnetic compass, canals) were invented in China but the modern era did not begin there, we can see cannon, mortar & rifle supremacy moved around politically divided Europe, where edicts like those by Japanese (Sakoku edict) & Chinese governments (sinking of the Treasure Ships) to limit foreign influence & technological development were not possible. Retreat from science, or doing it badly, meant becoming political & economic backwaters. The US & Soviet nuclear & space programmes are another clear example, of international military power being directly tied to being able to do good science. Greater internationalism and more integrated networks and open communication, in science as in economics and trade, has been a key enforcer of shared norms. The bigger the community the more powerful, in a very non-linear wsy.

Durkheim's understanding of religion, broadened to work beyond the Abrahamic template, involves the idea of collectively holding values to be sacred, beyond question, as binding together moral communities. This is useful in understanding how different sciences have different cultures, founded in exactly what they hold to be core unquestionable ideas, yet can still share deep sympathies because of a shared core, despite huge differences in practice. And in seeing how communities that don't accept core values, get pushed out or cut off from the wider scientific community ('Tobacco science' is another example of a split).

'Science is what scientists do' can be misunderstood (eg this slightly bonkers reaction to Sabine Hossenfelder saying that). But seen as a kind of language, who's development is defined by it's speakers like any language. And as binding a moral community, connected into a shared endeavour by those values. Given that understanding, that is the scientific method: it's what scientists do.

Answer 7

The scientific method exists as a general concept, but it is subject to change. Popper and Kuhn have made great contributions to the philosophy of science. There is another interesting demarcation between philosophy and science. The takeaway is that science can be falsified and the paradigm can be changed.

Answer 8

The scientific method is very rigourous. It begins in my view with an expectational proof ("no matter in nature can either be created or destroyed") and then proceeds to prove said hypothesis using carefully calibrated instruments and techniques the absolute truth of the hypothesis. In order to secure validity it must be repeatable as well.

It is the aposite of "statisical theory" which while critical in an engineering faculty is a dangerous foil when working with items of tremendous volatility or uncertain properties. Controls not just to prove the hypothesis are critical but also to keep the "scientists" from being harmed. For a textbook example read or Google "tickling the Dragon's Tail."

Answer 9

• Emit a hypothesis
• Run some experiments
• Accept or reject the hypothesis based on the results of the experiments
• Improve or come up with a new hypothesis
• Repeat the hypothesis-test cycle until the phenomenon in question has been properly understood.

I think the list above is missing a couple of important steps:

1. Why we do it. It's the very first step, before emitting a hypothesis, and the reason we are doing the rest in the list, the reason for science.¹ It all starts with a question, a "why?..". A distinctly different kind of curiosity, that of a human, not a cat.

   It is not enough for a human to know what is. We need to know why it is so. And, by that measure, how it might have been, and, sometimes, how it ought to be.

2. Knowledge sharing. Often as soon as we formulate the hypothesis (but definitely when we think the experiments confirm it) we share it with others. And not just the premise itself -- by simply saying "so and so is true", you share your personal opinion, but not knowledge.

   When you share knowledge, that truth is not your own. It is not you who is talking to us, it is one of the ways the objective reality itself conveys what it "wants" us to know.^{2 and 3} And that's why no one should take your word for it. No, seriously, no one should -- not your word, nor anyone else's for that matter.

   When the message comes from the objective reality, it must be presented with a certificate of authenticity -- an explanation, through logic and reason, of why do you think is true. And it is everyone's personal responsibility to verify that certificate before accepting the message -- no one else can or should be doing it for them.⁴ That's why every one of us has, in their prefrontal cortex, a piece of the most advanced computational hardware ever conceived -- their rational mind.⁵

   And that is the knowledge sharing in a nutshell -- of the hypothesis, as in the explanation of why something is, but also of the steps anyone can retrace to verify that the explanation is valid.

I hope this makes the scientific method much less of a mystery.

 

¹ Interestingly enough it's the same step that connects is and ought, something that Hume was literally blind to, seeing a "gap" in its place.

² "Listening not to me but to the logos it is wise to agree that all things are one." -- Heraclitus, circa 450 BC

  "The words I say to you I do not speak on my own authority. Rather, it is the Father, living in me, who is doing his work." -- John 14:10
  ... and for the record, "Though I have been speaking figuratively, a time is coming when I will no longer use this kind of language but will tell you plainly about my Father." -- John 16:25

³ We don't have to listen, of course, but the reality has other means of communicating the same message, and those we might find much harder to ignore. Or, if you want:
  "I am not seeking glory for myself; but there is One who seeks it, and he is the judge." -- John 8:50

⁴ ... skipping over the dense math if you have to, we only go as deep as we are comfortable -- the point is that we see the big picture with reasonably good resolution. No, we don't need math to understand Quantum Mechanics -- you won't be able to explain all the "why's", but that's no excuse for not trying to.

⁵ ... also, in theory, their conscious Self, and in charge -- for real! And sure, that is not what the reality is, of course. But, just as sure, that is how it ought to be.