10

When people in general discuss science, they talk about the scientific method as if it was a fixed and universally agreed upon principle. In a show I saw recently by Neil DeGrasse Tyson, he explicitly states that it was invented by Ibn Al Haytham in the 10th~11th century. The colloquial definition of the scientific method seems to be something along the lines of:

  • Emit a hypothesis
  • Run some experiments
  • Accept or reject the hypothesis based on the results of the experiments
  • Improve or come up with a new hypothesis
  • Repeat the hypothesis-test cycle until the phenomenon in question has been properly understood.

This seems to be an informal description of verificationism (or might be interpreted to include falsificationism as well).

In philosophy of science on the other hand, the demarcation problem (how to tell science from pseudo science) is still an open question. Although there is a general consensus among the scientific community that topics like Intelligent Design or Astrology are definitely pseudo-science, there is no accepted demarcation criteria and those that have been proposed are all either too strong or too weak. String theory for example notoriously fails certain demarcation criteria. There are also border cases with practical relevance, such as alternative medicines (Acupuncture, Herbal treatments,...), or psychoanalysis and psychotherapy.

Aside from the demarcation question, the problem of underdetermination of scientific theories also still stands, as well as the problem of induction, as exemplified by Hempel's paradox (I understand that demarcation, underdetermination and induction are all related to each other).

Given these open issues in philosophy of science, is it legitimate to speak of the scientific method (or methods) as if it was an established result that has been fixed once and for all?

Are laymen simply abusing language when thy speak of a the scientific method, simply because they are unaware of the complexities that arise in philosophy of science?

What is exactly the relationship between the scientific method and the demarcation problem? My understanding is that Feyerabend (and maybe Kuhn) questioned the existence of a fixed scientific method at all given the unresolved issues with demarcation?

9

I think one can talk about "the" scientific method as long as it is understood that it is a broad outline rather than anything like a clear prescription. Specific sciences in specific periods provide something more along the lines of the latter, but such standards and best practices are explicitly understood as field specific and revisable, at least ideally. Perhaps, "the scientific approach" would be a better name than "the scientific method".

Several bullet points in your general outline of the method are highly indeterminate. How does one emit or come up with a new hypothesis? Doing it by exhaustive search or at whim is practically hopeless in most cases, in fact discovery is the most non-trivial part of doing science. There are many written and unwritten heuristic guidelines, but none of them are a permanent part of the method. Accepting/rejecting a hypothesis is also non-trivial. Popper's falsification criterion strictly speaking applies to theories, not individual hypotheses, and which part of the theory is to be rejected in response to a falsifying test is not always clear. Even if applied to hypotheses it does not straightforwardly translate into statistical hypothesis testing. The traditional Fisher and Neyman-Pearson significance testing has come under heavy criticism in soft sciences recently on the grounds that "statistical significance" is often a poor guide to scientific significance, see Hypothesis testing: Fisher vs. Popper vs. Bayes.

There is plenty of vagueness in what science is and is not, but we know from the Sorites paradox, a.k.a. the line drawing fallacy, that no bright line of demarcation does not imply no difference between science and pseudo-science. We will know it when we see it, but there will be borderline cases. Popper's falsification criterion proved too restrictive because multiple parts of a theory can be revised in response to a falsifying test, not just a single hypothesis. But unrestricted holism makes everything retainable "come what may", and is too broad. But if drawing a line between a few grains and a heap in Sorites is still debated how can it be easier with science.

Lakatos's more holistic revision of Popper (replace degenerative research programme with a progressive one) explicitly gives no prescription on when a degenerative programme ought to be replaced. According to Hacking, Lakatos not so much prescribes a method to scientists as gives advice to historians on how to rationally reconstruct scientific developments of the past. Rationality requisite for science can only be judged in retrospect, after the benefit of hindsight made the methodology clear. And this is quite in tune with Quine's naturalized epistemology, which subjects all methodological rules to the same re-examination as the science they regulate. Another interesting approach that grew out of polemic with Kuhn is the structuralist programme, see Architectonic for Science by Balzer, Moulines and Sneed.

String theory is in a unique position with respect to hypothesis-test cycle because it can not run it through no fault of its own: probing energy levels where the Standard Model and General Relativity fail will likely remain practically unfeasible for decades (as predicted by the Standard Model itself, not by string theory). As is often said, modern physics is the victim of its own success. Johansson and Matsubara give detailed methodological analysis of string theory from various empiricist perspectives (positivists, Popper, Kuhn, Lakatos, Quine) in String Theory and General Methodology.

  • Partly for reasons you mention, Lakatos is an odd choice to discuss here, as is Quine, no? when there are so many 20th century philosophers who actually explored scientific method. – ChristopherE Feb 6 '16 at 23:58
  • @ChristopherE Positivists, Quine, Popper, Kuhn and Lakatos seem to be the standard list of modern scientific methodologies, with multiple variations on the themes of course, see e.g. Johansson and Matsubara. – Conifold Feb 8 '16 at 22:08
  • They're standard reading for philosophy of science courses, yea, but not for methodology or reasoning courses, no. – ChristopherE Feb 9 '16 at 0:59
  • @ChristopherE I'll bite, who do you recommend? – Conifold Feb 9 '16 at 23:07
7

No, "the" scientific method does not really exist. Feyarabend has argued that historical case studies do not support the idea of a unique "scientific" method and, further, that such an idea is 'pernicious'.

The idea that science can, and should, be run according to fixed and universal rules, is both unrealistic and pernicious. It is unrealistic, for it takes too simple a view of the talents of man and of the circumstances which encourage, or cause, their development. And it is pernicious, for the attempt to enforce the rules is bound to increase our professional qualifications at the expense of our humanity...All methodologies have their limitations and the only 'rule' that survives is 'anything goes'. Against Method

Perhaps Feyerabend's view can be understood in analogy with the idea of a single Logic: today we readily accept that there are many logics and no apriori criterion which form should be specifically used. Most people however are still convinced that they reason and act according to logic, assuming it is the same one known since Aristotle, even if most of their questions do no get a yes/no answer (but rather a 'dunno').

3

From my questions I've asked on the topic, The Scientific Method is understood to be the process you describe, perhaps omitting the last step. However, not all results acquired via The Scientific Method are accepted as legitimate by The Scientific Community. Attached to The Scientific Method is a prologue: what sorts of assumptions you made before starting the experiment. Also attached to The Scientific Method is an epilog: how you aggregate the piles of results produced using The Scientific Method into usable knowledge about the world. I find the demarcation problem is mostly in that prolog and epilog. Science, as a whole, includes both this prolog and epilog, not just The Scientific Method.

One thing to remember is that the results of The Scientific Method can never prove a hypothesis to be true. They can merely demonstrate that an existing hypothesis is more statistically false than this new one. The abductive step of converting from this process to knowledge about how the world works is not part of The Scientific Method, though it is clearly part of the greater concept of science itself.

If you try to define The Scientific Method to include these steps, then I agree that it ceases to be unambiguous. The difference in opinions of Kuhn and Popper should be more than sufficient to demonstrate just how much ambiguity there is.

  • You bring up Kuhn, but it seems obvious that the process described is only the puzzle-solving work of "normal science". The work of integrating the results into the broader paradigm (which is the other half of normal science), and that of adjusting the paradigm itself (which is outside "normal science") definitely fall outside this loop. A collection of independently tested hypotheses that constituted no cohesive and convincing "whole" would not be science, it would be chaos. – user9166 Feb 8 '16 at 23:08
2

The point of the scientific method is to distinguish theories that match reality from theories that do not match reality. To that end, a properly methodological approach will work. But with any methodology, shortcuts can be taken, which sometimes make no difference and other times will invalidate the method. Weaknesses in the scientific method all stem from which shortcuts are being taken.

Hempel's paradox is a very useful tool in considering how to do science correctly. The basic idea of the scientific method, as you outlined above, is to start with a single hypothesis, and test to see whether it matches reality. The problem is that if you pick the wrong experiments, you can confirm that a poor theory looks good. A more rigorous approach would be to take multiple theories and try to compare them. This leads to an approach where Hempel's paradox is no longer a problem.

Hempel's issue is that the theory "All ravens are black" is the same as "All non-black objects are not ravens". The two statements have identical truth values. But the obvious approach to test the two theories differ. For the former, you would find ravens and look at their color. For the latter, you would look at non-black objects and see whether they are ravens.

But what if you wished to compare multiple theories. "All ravens are black" is the same as "100% of ravens are black." That theory could be compared to "0% of ravens are black." Looking at a single raven would distinguish between the two. But since most everyone has seen a black raven, a more interesting comparison might be with the theory "90% of ravens are black". An experiment where you look at a single raven would obviously not be useful to distinguish between those two theories, since the most likely result (a single black raven) would be consistent with either theory. But if you examined 200 ravens, there are no results that would be consistent with both theories (to 1 part in 1 billion).

Likewise, everyone has seen lots of non-black objects that are not ravens. If you wished to compare "100% of non-black objects are not ravens" with "99.9% of non-black objects are not ravens", no one would care, because we already know that ravens of any color make up far less than 0.1% of objects that exist.

So, in summary, if you start with multiple theories, write them down ahead of time, and compare their predictions, then you can determine which experiments can usefully distinguish between the different theories. This approach leads straightforwardly to determining which experiments are most likely to yield useful information and also other details, like how large a sample size would be needed.

I would say this approach is the "true" scientific method. People will often take shortcuts (e.g. not identifying ahead of time multiple theories they wish to compare), and when they do, those shortcuts sometimes invalidate the results.

But the simple fact is, if you take the approach above, at the end of the day, you will have a definite answer "Theory A matches reality better than Theory B" (or vice-versa). This is the goal of the scientific method. The fact that it will work is almost a tautology.

1

In philosophy of science on the other hand, the demarcation problem (how to tell science from pseudo science) is still an open question. Although there is a general consensus among the scientific community that topics like Intelligent Design or Astrology are definitely pseudo-science, there is no accepted demarcation criteria and those that have been proposed are all either too strong or too weak. String theory for example notoriously fails certain demarcation criteria.

Demarcation between science and non-science is greatly overvalued because people have an irrational idea that science is an authority. If something they call "science" says X, then you shouldn't question X, and if you do you're a heretic and a moron.

There is no scientific method per se, as pointed out most persuasively by Popper, who pointed out why there couldn't be such a method. The problem is that how you came up with an idea is irrelevant to assessing it. You could get high and stare at the ceiling, you could be inspired by a dream, you could have the idea while knitting. What does matter is how you assess the idea. You should look for ways it might be contradicted by experiment. You should consider whether it contradicts current explanations and if so look for ways of solving that problem. You might call an idea science if it can be tested and is not ruled out by explanatory problems. See, for example, "The Fabric of Reality" by David Deutsch, Chapters 1,3,7.

My understanding is that string theory is not currently testable. You could say it is unscientific if you want. But then you should ask whether this makes any practical difference. Just about any theory of quantum gravity will be difficult or impossible to test. That doesn't mean people should stop working on that problem.

There are also border cases with practical relevance, such as alternative medicines (Acupuncture, Herbal treatments,...), or psychoanalysis and psychotherapy.

Some alternative medicine claims could have good explanations and be testable for all I know, but I doubt it. Psychoanalysis and psychotherapy are at best moral instruction and have nothing at all to do with science, see "The Myth of Psychotherapy" by Szasz.

The underdetermination problem is only a problem if you expect scientific theories to be determined by data. No scientist worth his salt would adopt that position because any scientific theory refers to unobserved and unobservable events as part of the explanation for what he does see. Both the underdetermination problem and Hempel's paradox are solved by rejecting the idea of induction root and branch. There are many other reasons for doing that as pointed out by Popper in "Objective Knowledge", Chapter 1, "On the sources of knowledge and of ignorance" and other material in "Conjectures and Refutations", "The Logic of Scientific Discovery" and "Realism and the Aim of Science".

Feyerabend and Kuhn are worthless. If you want a refutation of Kuhn see the title essay in Popper's book "The Myth of the Framework" and the introduction to "Realism and the Aim of Science". If you want a refutation of Feyerabend read his stuff and try to take it seriously as an explanation of how the world works.

-1

The scientific method is very rigourous. It begins in my view with an expectational proof ("no matter in nature can either be created or destroyed") and then proceeds to prove said hypothesis using carefully calibrated instruments and techniques the absolute truth of the hypothesis. In order to secure validity it must be repeatable as well.

It is the aposite of "statisical theory" which while critical in an engineering faculty is a dangerous foil when working with items of tremendous volatility or uncertain properties. Controls not just to prove the hypothesis are critical but also to keep the "scientists" from being harmed. For a textbook example read or Google "tickling the Dragon's Tail."

  • 1
    Proof is a very strong word. I don't think "proof" is a word that can be applied to scientific hypotheses. Experiment and observation confirm an hypothesis, but they do not prove it. – Nick May 23 '16 at 4:10
  • "the absolute truth of the hypothesis" Newton's laws were the absolute truth for centuries, then they weren't. I recommend you read up on the history of science and the philosophy of science before you state with absolute certainty that "The scientific method is very rigourous." – Alexander S King May 23 '16 at 4:24
  • Do the Pyramids of Giza still not stand? – Doctor Zhivago May 23 '16 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.