What is vague about the definition of the scientific method?

Question

I have asked a few questions and a statement ubiquitous throughout the answers and comments was There is no THE Scientific method, but only scientific methods.

But I don't understand. There is THE Scientific method whose definition I got by just typing define scientific method on Google:

A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.

Where is the vagueness in this definition?

Answer 1

The question is what is vague about the following definition of scientific method:

A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.

The scientific method should provide a way to solve the demarcation problem that CriglCragle pointed out in his comment. That is, it should be a way to distinguish what a real scientist does from a “pseudoscientist” or someone not doing science.

Consider what is listed in the definition obtained from the search. We have that a scientist does the following:

1. Makes systematic observations
2. Measures
3. Experiments
4. And formulates, tests and modifies hypotheses

Don’t non-scientists do the same thing? Drivers of a car systematically observe road conditions and other vehicles. They measure distances with their eyes or using a GPS. If they want to change lanes they experiment in the process of changing lanes. If another car honks or they see another car move into the same lane at the same time they do, they modify their hypothesis that they are safe to move into that lane.

This is someone driving a car, not someone doing what a scientist, as a scientist, is intuitively seen as doing. So this description of the scientific method does not distinguish what a scientist does from what people may typically do during the day. It is vague.

This leads to at least two lines of thought:

First, is there any other description of the scientific method that will allow us to rationally demarcate what is science from what is not science? If we can find this we should be able to say whether psychology, for example, is pseudoscience or not. Popper has proposed falsifiability as an example of a better method to focus on, but using the car example I provided earlier isn’t the driver who sees another car enter the same lane also falsifying an hypothesis that it is safe to move into the lane? One could say that drivers of cars are also scientists, but it does not help solve the demarcation problem if everyone is a scientist.

Second, perhaps our motivations to rationalize a formulation of the scientific method should be questioned. Why do we want to know this? Is there any way to make this demarcation that does not degenerate into a political mechanism that helps the current orthodoxy isolate heretics restricting funding resources from going to the alleged “pseudoscientific” research programs? But since funds are limited having some selection criteria even if faulty may be better than nothing at all.

Answer 2

This is a great question. The challenge with this definition is that it is merely descriptive. It is a very useful definition in every day life for talking about science, but some questions are rather difficult to answer if one is using this definition:

• Am I following the scientific method?
• Assuming I have a specific goal in mind, does the scientific method help me achieve it?
• Is science good?
• I have been introduced to some concept (like faith healing), and they make a claim that their theories are backed by science. Are they speaking accurately?

These are important questions, but they are hard to answer with a 2-line definition like Google provides. They require digging deeper into what the scientific method means, and that is where we start to see the vagueness.

A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.

When I talk with people about science, the testing and experimentation sides of the discipline are what everyone tends to focus on. There's plenty of systematic observant methodologies out there, but we tend to think of the testing an experimentation process as the defining aspects of science. But that process has shifted over time. The idea of what it means to do an experiment and test hypotheses has changed since the 17th century.

Early science was dominated by inducitivism. Inductivism was primarily built around using induction to go from a small number of observations to a universal law. It had its process of affirming and denying these inductive leaps, but it was more "free range" than we typically think of science today. Positivism restricted it a bit, focusing on models which are predictive, but it had a ways to go to reach what we consider science today.

One of the more important shifts came Karl Popper in the early 20th century. Popper promoted the idea of falsification. Under Popper's thinking, you could make any claim you wish in science, so long as it was falsifiable -- there had to be a way to disprove one's statements. If I claim "all swans are white," that statement is falsifiable by observing a single black swan.

In my opinion, Popper's science is what most people think about, but he wasn't the last philosopher to touch it. One of the problems with Popper's approach is that scientific observations increasingly became statistical in nature. And, in statistics, it is virtually impossible to truly falsify anything. That left Popperian science in a bit of a limbo.

Thomas Kuhn provided the most successful follow-on to Popper's approach to science. Kuhn's definition of science included the scientific community itself, not just the experiment. He argued that science works in waves. There's periods of order where we're simply beating down statistical errors getting closer and closer to Popperian falsification, and then there's tumultuous periods of excitement where a bunch of theories get disproven and new theories (with less evidence behind them) come up in their wake.

All of these approaches to science are substantially different, and that's where the vagueness comes from. And, if one wishes to push the concept of science further back than the 17th century, we find we have to have even wider definitions of the scientific method which start bordering on guess and check. We start seeing definitions that accept more and more of what we now call pseudo-science. And that's okay! That's the fun of working with definitions!

Answer 3

Consider particle physics, psychology, climate modeling, and cultural anthropology.

On paper, particle physics is very theory-driven: physical theory is used to make very precise predictions about what will happen under very tightly specified conditions. By realizing these conditions experimentally, particle physicists can test their predictions. But realizing the very tightly specified conditions requires extraordinarily complicated feats of engineering — the Large Hadron Collider is run by 10,000 engineers and scientists.

Psychology is also an experimental field. But here the theory can't make precise quantitative predictions, and it's basically impossible to achieve anything like the physicists' level of control over experimental conditions. For this reason, psychologists generally don't have highly mathematical basic theories; but they often use more mathematically complex methods of statistical analysis than physics.

Climate modelers construct computer simulations based on systems of differential equations describing interactions between heat, the chemistry and movement of the atmosphere and hydrosphere, the presence and reflectivity of land, and in recent years even the biosphere. The scale of these models, in both time and space, means that climate models can't work with direct observations. Instead, historical climate conditions are estimated using indirect methods — tree rings, the air trapped in glaciers — and climate models are fit or "tuned" to approximate those historical estimates. Instead of experiments, climate modelers generally rely on simulated counterfactuals and robustness arguments. For example, to establish that human greenhouse gas [GHG] emissions are largely responsible for warming over the last several decades, they'll run versions of their models with the GHG emissions turned off. And they'll use the outputs of several different models to make predictions about what the climate will look like decades from now. Both kinds of arguments assume the models are more-or-less accurate once they've been tuned to past climate.

Unlike the other three examples, cultural anthropology is primarily qualitative, not quantitative. Psychologists collect data using standardized forms or instruments (to measure reaction time in an experiment, for example). Cultural anthropologists watch people's behavior — how a certain group of people interact during meetings, or what people do as they move through a public place — and take extensive and detailed notes, generally over a period of months or even years. Typically the closest they'll get to an experiment is a semi-structured interview, which uses a small list of open-ended questions rather than a highly structured survey. Cultural anthropology research often isn't designed to test a precisely-formulated hypothesis. Instead, more typically, cultural anthropologists are trying to understand a social practice as the people involved understand it. These research goals, and the use of qualitative data, lead to very different ideas about rigor and proper research design. For example, saturation, which is identified ex post, takes the place of ex ante power analysis.

Given the radical differences between these fields, it's hard to see a single underlying "scientific method." Even concepts such as "systematic observation" and "testing hypotheses" mean different things.

Answer 4

A method of procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses.

All of these are vague: It doesn't tell you what to observe and nor how to measure it; or what experiments to make and especially not how to come up with hypotheses.

For these one can argue that you need specialist knowledge of a discipline, its history and its methods; this constrains the above more tightly, nevertheless they are still left open to a degree.

(It's probably also worthwhile to point out that they need to be vague in order for scientific progress to be possible).

Answer 5

The definition seems to imply that THE SM has to have all of the listed attributes. Otherwise it is just defining A SM, a collective set of methods which are scientific, i.e., those methods which include some of the listed attributes and none of their negations. For example, someone who just observes and reports on the behavior of snails could be a scientist. And a mathematician, who just formulates hypotheses and tries to prove them could also be a scientist. They would be scientists if they do what they are doing methodically, taking into account prior efforts.

I question whether any scientific method can or should be used to give either a necessary or a sufficient condition to determine if something is "science."