From the controversy in this question, I would like to know what differentiates "the scientific method" from other method. I'm asking for differentiation, rather than a definition, because definitions call for an all encompassing answer, which is often hard to find. If someone asks "what is an alternative to the scientific method," I would like to be able to start by being able to reliably say whether a particular method is the scientific method or not.

Of course, being a linguistic issue, the definitions of words themselves are flexible. For instance, it is not uncommon to hear one in science talk of "truth" in a sense which is incompatable with the mathematical meaning of "truth." Accordingly, I would like to stick to the following ideals in wording whenever possible:

  • Prefer mathematically accurate terminology. Ideally the word "proof" should be used in the form found in proof theory, and "true" and "false" should be used in the form of Boolean algebra.
    • No method should claim a hypothesis which predicts the state of the future is "true" unless they claim soothsaying abilities (most scientific theories include a random variable to avoid needing to claim soothsaying abilities... this is legitimate)
  • Words which imply direction should be clarified carefully. "Good" and "bad" are notoriously difficult, but phrases such as "advancing society" can also be very tricky as well.
  • The physical world and the world of information theory should be treated differently. In particular, the concept of "testing a hypothesis" will likely need to be broken down into testing and observing the physical world, taking measurements (which should have errors associated with them), and only then is the hypothesis compared against the results of the measurements.
  • Mathematically extending arguments into infinity (such as claiming some specific cellular level learning process qualifies as the scientific method) are valid, but may rapidly make it harder to distinguish non-scientific methods from scientific ones. Such claims should be made carefully, but they are not inherently invalid.

If these ideals are too strict, you may bend them, but you should qualify such choices with an explanation of why my recommended wording choices are too strict. Their goal is to put different methodologies on the levelest playing fields that I can manage in this medium.

Being stack exchange, bonus points for references to existing theories, though you may need to paraphrase their results to fit with the ideals I'm recommending for these answers.

Answers may be tested two ways:

  • Positive: If we think something is the scientific method, does it meet the requirements set forth?
  • Negative: If we think something is not the scientific method, does the answer explain why it is not?

It seems to me that what you are getting is one specific definition, and not general differentiations. So far we have a lot of Popper, but this covers only one scientific method.

I trust a workman to judge his trade. So to me, what differentiates science, must apply equally to everything that the great scientists of any era recognized as science in their own day.

That includes Alchemy (pursued by Newton) and Platonic cosmology as concretized by Ptolemy (with which Tycho Brahe tried desperately to find a Copernican compromise), even Marx and Freud (which many people at the time of their origin considered scientific, even if they disagreed with them.) Continuing to believe in such things once they are superseded by better explanations is counter-scientific, but in their day, these were science by the standards of those who should know.

So Popper doesn't cut it. His is a theory of why science works, not a definition of science, and he is only partly correct. There is not just one scientific method, there are many. And while falsification with calculated risk may provably be the most statistically sound approach overall, it does not explain what a lot of major names in science actually did with their time, or why they were productive. Here are some variations:

  • Many modern physicists spend most of their time determining which sets of hypotheses can be harmonized with one another and which cannot. Experiments in important areas have become large undertakings that need to be somewhat rationed. Where does that fit into Popper? Is it all wasted time? Their mathematics is not falsifiable, only the theories which are being compared and contrasted by it are. And we cannot seem to produce an experiment that would test them. The big experiment at CERN might have, but it came out with a number that does not actually remove any of them.

  • What Darwin did was science, and he did not follow Popper's criterion. He worked entirely inductively from a huge mass of data in almost everything he did. Until we had genetics, his work remained unfalsifiable.

  • Biologists have always drawn conclusions from amassed data and woven them into large, complex systems, relying on the weight and completeness of the system to validate their assumptions are not bad. The entire Linnean system on which our current taxonomies are built with no experiments. Linnaeus was an Aristotelian, and did not really value experimentation. This, although a very useful tool, is too general to be overall falsifiable.

  • Chemistry has at its root the periodic table, first roughed out by Mendeleev, who did it without reference to original experimentation. He obsessively categorized and recategorized the physical chemistry of the time until the sets of things in related categories had the fewest differences. Until we discovered orbitals, this was not falsifiable.

All of these are scientific methods. Some are better than others, and some only work well for people with a particular temperament, but they have produced real science that we value.

To me, the things that these systems do have in common seem to be these:

  1. Data matters. We can see Aristotle's 'physics' as physics, but the refusal to accept contradictory observation, and just drop his biases, keeps him from being a physicist.

  2. Challenges are to be recognized and met. Some may see reality more easily or find precision with less effort, but reality is reality and consistent failure to agree on observations, even those of the commoner or oddball, is to be taken seriously and diagnosed. (Galileo's Salviati owed both Sagredo and Simplicio answers.)

  3. Nature is presumed consistent. The rules are not random, so when things differ there should be a diagnosis. That does not mean that something like Brownian motion or quantum indeterminacy is forbidden. But it has a meta-layer at some level where the randomness leads to observable consistency, and the randomness itself explains that consistency.

  4. Parsimony and elegance accompany truth. The simplest explanation for a phenomenon gets the first crack at acceptance. Clutter in your theory is a sign you are failing.

  5. Different sciences harmonize. We seek a single, overall truth. So there is no split between disciplines that allows chemistry not to apply to biology, or physics not to apply to psychology, as there is are splits that allow, for instance, religion not to apply to philology, or anthropology not to apply to math.

  • Look up 'qualitative methods' some time. People in one of my fields (psychology, yes I have degrees in sciences) accept those as real science. You can disagree, but then you need to tell me why half of a profession is wrong, instead of just being a jerk. Is most of anthropology just not a science? Take a reasoned position and do something other than stick to your guns and tell folks they are wrong without introducing any data. – jobermark Apr 20 '15 at 20:18
  • Also, most major names in the philosophy of science after Popper have basically disagreed with him, from his contemporary Kuhn, to Lakatos and Feyerabend in the next generation down to Elliott Sober today. Folks teach Popper because it makes science seem simple. But it is not realistic. – jobermark Apr 20 '15 at 20:38
  • I'm coming back now that the topic has cooled down to close the question. Your answer is currently winning in terms of number of votes, and seems to contain useful information. However, I am not certain if it answers the question "what differentiates the scientific method from other methods?" It seems to suggest perhaps there is no clear differentiation to be had, but does not not mention any other non-scientific methods outright. Is that a reasonable summary of your stance when phrased in terms of the original question, or did you seek a different final answer? – Cort Ammon Apr 30 '15 at 17:19
  • It doesn't, it is the second half of a very long answer, below. That has -1 votes, having been trashed by those who think anything less specific than Popper is evasive. But I felt the need to defend the idea that one needs to look more broadly than Popper with some data, and that answer is already too long. – jobermark Apr 30 '15 at 17:24
  • I will combine them if it would be convenient for you. But then we are at zero votes all around. Too evenly divided and too emphatic in our pro- or anti-Popper partisanships. – jobermark Apr 30 '15 at 17:27

I would suggest (or assert) the Popperian rule that ultimately what makes a theory scientific is that it's falsifiable. My interpretation of that meaning is that the theory makes a difference, whether it may be "true" (or "factual" or be a closer approximation to whatever unknown "truth" is the reality) or not.

A theory can have a provisional status of "scientific" for a finite time if it is not yet known how it might be falsifiable. But if a hypothesis is inherently not falsifiable, that means there is no way that it makes any difference. If it doesn't make a difference to us, if this difference cannot be tested one way or another and, within the theory, can never be tested one way or 'nother, in my opinion, I don't see the difference between the hypothesis and a philosophical belief.

Between 1915 and 1919, General Relativity had a provisional status because humans couldn't see a difference in observed physical behavior between GR and Newtonian gravitation. But GR was not inherently unfalsifiable. During the solar eclipse that Eddington observed, the GR theory made a difference from what Newtonian physics would say. And the placement of stars observed close to the Sun appeared to be in positions that agreed more with GR than with Newton. That moved GR from provisional science into science. If the position of the stars were more in keeping with Newton, the GR would have been disproved as it was formulated and Alfred Einstead might have to go back to the drawing board. But that's not what happened.

So GR submitted itself to falsification and it withstood the test. So far. But there are problems (however these problems might not be the fault of GR). Because the masses of elementary particles are soooo much smaller than the Planck Mass but the charge on charged particles is approximately the Planck Charge, we commonly think that "gravitation is soooo much weaker than electromagnetic and the other interactions". So the Standard Model was developed without any attention to effects of gravity between particles because it would make no difference in outcomes.

But, as Lee Smolin has written:

Ignoring gravity was a step backward, to the understanding of space and time before Einstein's general theory of relativity. ... The chief lesson of general relativity is that there is no fixed-background geometry for space and time, ignoring this meant that you could simply choose the background. This sent us back toward a Newtonian point of view, in which particles and fields inhabit a fixed background of space and time -- a background whose properties are fixed externally. Thus, the theories that developed from ignoring gravity are background-dependent.

As in aether?? Like was a falsifiable theory and which was falsified in the Michaelson-Morley experiment (unless you might make a goofy theory that somehow the aether background follows the planet Earth as it revolves around the Sun)?

String theory and M-theory were proposed as a means of reconciling GR and the Standard Model. But the hypotheses require the existence of many more dimensions of space and time that simply cannot be confirmed. We cannot tell if these other dimensions are there or not. And, just because string/M-theory proposes such a means of reconciliation does not mean that it's reality.

To be falsifiable, there needs to be something or some hope of something that humans (or some beings) can observe, that is more congruent to what the hypothesis says than what existed before the hypothesis was made. Otherwise it doesn't make any difference. I.e., if all GR could say was that it was consistent with our existing Newtonian understanding of gravity, then all GR would be is a mathematical philosophy. But GR said something that was different than what Newton said and we could hope to eventually measure the difference and within 4 years we did.

To me, that is the "demarcation" that Popper is talking about. That is what you need to do to move a hypothesis, within the discipline, to science.

I can formulate a hypothesis of reality that includes the existence of God which is consistent with our observation of reality. Many, many theists, who also work in the sciences, do that. But it's not science. The material observations of reality would be no different without the existence of God. The existence of a transcendent God, who does not submit Him/Herself to material test, is not falsifiable and not science.

In my opinion, the multiverse hypotheses (and there are several different multiverse theories, some more outlandish than others) suffer the same lacking regarding falsifiability. Other universes outside our observable Universe cannot be measured any more than God can be measured. To use the multiverse concept along with the weak Anthropic Principle (the latter is a tautology, so it's gotta be true) that implies selection bias (among all them zillions of universes) as an explanation of the remarkable fine-tuning of the Universe is no more a falsifiable cosmology than is a belief that God created the Universe. This, of course, is a position disputed by strict materialists who, in my opinion, would rather subscribe to the "Multiverse-of-the-gaps" theory.

  • What methods does science use to falsify a theory? I know of several ways of thinking that are traditionally not considered scientific which claim their theories are falsifiable, but science claims they are not. How should i discern the two? – Cort Ammon Apr 18 '15 at 22:37
  • 1
    Does that suggest that all statistical experiments fail to provide Popper's requirement of falsifiability if they have a gaussian error term attached to them? After all, a gaussian can take on any value, and it would take an infinite number of tests to prove you aren't simply on a string of bad luck. – Cort Ammon Apr 18 '15 at 22:49
  • 1
    like any quantitative measurement, we have measurement error. even if the measurement is binary (something is either there or it's not, like a two-slit thingie) there is inherent measurement error. we deal with that with repeatability. if there is something consistent to pop out of the measurements, that component should team up proportionately to the number of repeat times and the statistical errors should team up proportionally to the square root of repeat times. in the average, you divide by the number of repeat times and the error gets smaller and smaller, if there is correlation. – robert bristow-johnson Apr 18 '15 at 22:53
  • 3
    @CortAmmon Popper does not care about "false" in his notion of falsifiability, he cares about risk, which is an implicitly statistical phenomenon. His long debunking of the notion of justification applies to any attempt to conflate science with truth. Faith in the actual falseness of a proposition is a justification of its opposite. From the point of view of 'falisiability', all that science can purport to do is predict well both the past and the future, giving good explanations and reliable application to engineering and open problems. – jobermark Apr 19 '15 at 1:41
  • 1
    @Dave I'm looking at what science claims qualifies as "falsification." If they want to claim statistical methods are valid for falsification, that's fine (I believe that is actually the most common definition). However, statistics is both an enormous strength and an enormous weakness. I've had discussions with those who insist there are non-statistical versions of falsification in science, but nobody has been able to produce one. This matters because, if science is purely based in statistics, it has to step aside when the sample size is 1, until it gets a chance to collect more data. – Cort Ammon Apr 20 '15 at 1:22

Popper presented the idea of falsifiability as an (the?) important consideration for the practice of science: when people are doing science they are formulating ideas (often "presenting solutions to problems" in his language) in such a way that that they can be shown to be false. In science, proving an idea false is usually a matter of comparing the idea to empricial observations (though in principle showing an intrinsic self-contradiction in an idea would also serve to falsify it).

Although some of the aspects of Popper's work have been refined (or had holes poked in them) the central idea that science involves the process of putting forth ideas that can be proven wrong, is pretty widely accepted in the philosophy of science.

  • quite the coincidence, Dave, that our answers happened simultaneously and both referred to Popper and falsifiability as the demarcation of what is science and what is not. up-arrow from me. – robert bristow-johnson Apr 18 '15 at 21:31
  • What methods does science use to falsify a theory? I know of several ways of thinking that are traditionally not considered scientific which claim their theories are falsifiable, but science claims they are not. How should i discern the two? In particular, I rarely have seen evidence where there is a "self contradiction." It always seems to be more of a statistical thing – Cort Ammon Apr 18 '15 at 22:38
  • @CortAmmon To be falsifiable, a theory needs to predict the future, not just explain the past. No theory that just fits the data, and is flexible enough to thwart any attempt to contradict its applicability to any specific future event is falsifiable, because it is insulated from risk. Hegelian logic was Popper's main target. E.g. Freud explains observed facts of development, but it is only refined to cover more and more cases. When it meets a case where its predictions are not met, it creates a fold-back that incorporates the failure. It is a dialectical method, not a statistical one. – jobermark Apr 19 '15 at 1:51
  • @CortAmmon I'm not familiar with anything where Popper defines/prescribes methods for falsifiability; I presume that he felt that the concept seems obvious enough: predictions about the state of the world are made, if observations are inconsistent with those predictions we should discard the theory. Even if you get into a discussion of "what constitutes falsifiability" you are still engaged with that concept as part of the demarcation of science. – Dave Apr 19 '15 at 15:23
  • @CortAmmon I do agree with a theme in the comments on this question, that Popper's approach does a poor job of addressing the day-to-day statistical aspects of science. As far as I can tell, he only spent significant effort on the probabilistic aspects of QM and ignored the importance/role of statistics in science in general. – Dave Apr 19 '15 at 15:33

Apologies if I've missed the mark and for so heavily relying on content that already appears on this very page. My goal was to offer a succinct view of the ideal application of the SM and the limitations encountered when not adhering to its criteria.

In the original question, Paul's answer has one very important key word: useful.

Especially (if not, only) useful for experiments in which a control group is tested against an experimental group, the scientific method produces RESULTS that can be verified by a third party (replicable and peer-reviewed), deemed valid by consensus.

Volcano! is not a science experiment, it's an observation1. However, Does water have the same effect on baking soda as vinegar? leads to hypotheses that are (non-Socraticily2) provable by observing the results of a controlled experiment (conducted of course, by means of a procedure following the SM; thereby intrinsically 'provable' by science).

Popper insists on testing hypotheses, not just drawing conclusions from amassed data. -@jobermark, discussion in chat

In drawing a valid conclusion, one must vet the results of an experiment. Observational data can only lead to further theories that then need experiments to scientifically prove them.

All that science can purport to do is predict well both the past and the future, giving good explanations and reliable application to engineering and open problems. -jobermark

Equally important is how you interpret those results and what an acceptable margin of error is; see also, sample size: (summed up nicely in @robert's response to comments on his answer, in case you missed them in the wall of text; non-italic emphases, mine)

Like any quantitative measurement, we have measurement error. Even if the measurement is binary (something is either there or it's not, like a two-slit thingie) there is inherent measurement error. We deal with that with repeatability.

If there is something consistent to pop out of the measurements, that component should team up proportionately to the number of repeat times and the statistical errors should team up proportionally to the square root of repeat times.

In the average, you divide by the number of repeat times and the error gets smaller and smaller, if there is correlation.

1. Do not mistake observations made in an uncontrolled environment to be results (data set vs. outcome; correlation is not causation).

2. Accepting the fact of knowing that you know nothing, isn't going to get you very far.

  • Maz, i did not write: "All that science can purport to do is predict well both the past and the future, giving good explanations and reliable application to engineering and open problems." someone else might have. – robert bristow-johnson Apr 20 '15 at 3:27
  • @robertbristow-johnson, fixed. Was a comment on your answer, sorry. – Mazura Apr 20 '15 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.