While re-reading Shimon Malin’s “Nature Loves to Hide” I was trying to get a better grasp of the distinction between Ernst Mach’s philosophy that theory comes from observable magnitudes alone and what Einstein told Heisenberg: “It is the theory that decides what we can observe.” This turns Mach’s view upside-down.
My question is: Where do theories comes from? I am especially interested if one takes Einstein’s position that the theory tells us what we can observe.
Einstein's quote is reported by Heisenberg in Theory, Criticism and a Philosophy. It is clear from the context that Heisenberg subscribed to a kind of positivist Machian philosophy according to which observations produce independent "sense data" that serves as the ground of theorizing. Einstein's rebuttal anticipated Quine's rejection of the positivist theory/observation distinction (in its logicized form advanced by Carnap) in Two Dogmas of Empiricism.
However, unlike the later proponents of the theory-ladednnes of observations thesis, Einstein apparently believed that it is compatible with maintaining what Putnam called "metaphysical realism" in physics. In other words, Einstein subscribed to a moderate version of the ladenness thesis, where theories may present different pictures of how observations proceed, and hence affect their interpretation, but there is still an independent reality that underwrites all of them, it is just described in different but intertranslatable ways. This was challenged by the Quine-Duhem thesis of radical underdetermination of theory by observations that left little room for Einstein's optimistic realism. In Epistemology Naturalized Quine describes how it leads to an even more radical version of the ladenness thesis advanced by Hanson and Kuhn in 1950s:
"The dislodging of epistemology from its old status of first philosophy loosed a wave, we saw, of epistemological nihilism. This mood is reflected somewhat in the tendency of Polanyi, Kuhn, and the late Russell Hanson to belittle the role of evidence and to accentuate cultural relativism. Hanson ventured even to discredit the idea of observation, arguing that so-called observations vary from observer to observer with the amount of knowledge that the observers bring with them. The veteran physicist looks at some apparatus and sees an x-ray tube. The neophyte, looking at the same place, observes rather "a glass and metal instrument replete with wires, reflectors, screws, lamps, and push buttons." One man's observation is another man's closed book or flight of fancy. The notion of observation as the impartial and objective source of evidence for science is bankrupt."
Quine did not go this far, but even he rejected metaphysical realism and acknowledged that "we have stopped dreaming of deducing science from observations". Quine advocates a version of realism that combines observation sentences with pragmatic considerations in drawing pictures of reality, for a recent attempt to defend a more robust realism against the underdetermination challenge see Esfeld's The Rehabilitation of a Metaphysics of Nature.
But like most analytic philosophers Quine does not analyze the initial stage of creating scientific hypotheses, Popper is even bolder, declaring:"The initial stage, the act of conceiving or inventing a theory, seems to me neither to call for logical analysis nor to be susceptible of it". In other words, it is all just psychology, lucky guesses and bursts of genius. Indeed, philosophical reflection on creative processes in science has been scarce until very recently. Hanson was an early promoter of more serious research into it in his Is there a logic of scientific discovery?, and he only had Aristotle and C.S. Peirce to build on.
Peirce founded a theory of abductive reasoning that identifies classically fallacious but fruitful forms of inference and the "instinct for guessing right" as the scaffolding for constructing scientific hypotheses. For a more mature current form of it, model-based reasoning, complete with history of science case studies, see e.g. Nersessian's Mental Modeling in Conceptual Change. The basis of abductive/model-based reasoning are not observations alone, but a much more broad and diffused substrate of common sense, life experience, artistic analogies and metaphors, cultural gestalts, pragmatic assumptions, etc., but, contra Popper, it is broadly rational and susceptible to analysis.
Here is the full context of Einstein's quote as reported by Heisenberg:
"Einstein asked me to come to his flat and discuss the matters with him. The first thing he asked me was: "What was the philosophy underlying your kind of very strange theory? The theory looks quite nice, but what did you mean by only observable quantities." I told him that I did not believe any more in electronic orbits, in spite of the tracks in a cloud chamber. I felt that one should go back to those quantities which really can be observed and I also felt that this was just the kind of philosophy which he had used in relativity; because he also had abandoned absolute time and introduced only the time of the special coordinate system and so on. Well, he laughed at me and then he said, "But you must realize that it is completely wrong." I answered: "But why, is it not true that you have used this philosophy?" "Oh yes," he said, "I may have used it, but still it is nonsense!"
Einstein explained to me that it was really the other way around. He said, "Whether you can observe a thing or not depends on the theory which you use. It is the theory which decides what can be observed." His argument was like this: "Observation means that we construct some connection between a phenomenon and our realization of the phenomenon. There is something happening in the atom, the light is emitted, the light hits the photographic plate, we see the photographic plate and so on and so on. In this whole course of events between the atom and your eye and your consciousness you must assume that everything works as in the old physics. If you would change the theory concerning this sequence of events then of course the observation would be altered." So he insisted upon that it was the theory which decides about what can be observed. This remark of Einstein was very important for me later on when Bohr and I tried to discuss the interpretation of quantum theory..." [emphasis mine]
As Karl Popper pointed out in "The Logic of Scientific Discovery", the idea that theory comes from observation can't be true. Theories don't follow from observations. A theory is an explanation - an account of what is happening to bring about events. An observation is a record of events that happened in some particular region of space and time. A record that some event happened doesn't imply anything about why or how that event happened since many of the causes will not be observed in any particular case and some may not be observable at all, e.g. - the core of the sun can't currently be observed and perhaps it will never be observed.
Any theory places limits on what physical systems can be constructed and so on what observations can be performed. For example, no experiment apparatus that requires sending information faster than the speed of light can be constructed, so no observations that require sending information faster than the speed of light can be performed.
In addition, without a theory of what's going on, you don't know what observations will give you useful information. As already noted, theories don't follow from observations, so observations don't give rise to theories. Observations don't support theories either. Part of the problem is that observations can't imply theories. But in any case, observations involve theories about what is happening at a particular place and time, e.g. - theories about wires in the experimental apparatus responding to magnetic fields etc. Since those theories about the observation may be wrong, trying to found a theory on observations doesn't make the theory more secure. The ideas you have about the observation are either right or wrong and you don't know which, so the same is true of any theory that allegedly follows from the observations. But if your theory sez X should happen and Y happens instead then your theory has a problem and should be discarded. Sometimes a variant of your theory that's only slightly different will account for Y, but on other occasions you will have to throw the theory out and go for a completely different idea.
Kuhn was badly wrong about these issues as Popper pointed out on several occasions, e.g. - the title essay in his book "The Myth of the Framework" and his introduction to "Realism and the Aim of Science". In particular, Kuhn thought there was no objective distinction between good and bad ways of creating knowledge, see his essays in "Criticism and the Growth of Knowledge" edited by Lakatos and Musgrave. So his philosophy is intellectual poison that will prevent progress if you take it seriously.
For more Popper references, see
http://fallibleideas.com/books#popper.
"The Fabric of Reality" and "The Beginning of Infinity" by David Deutsch are also good on this topic.
We don't know the details of where the guesses used to create knowledge come from, but it must be a process that involves generating variants on previous knowledge and selecting among those variations since no other way to create knowledge is known. See "The Beginning of Infinity" by David Deutsch, chapters 1,2,4,10,15,16.
I think perhaps the contradiction in views here is only partial.
It seems to me that perhaps what Mach meant is not that theory directly "comes from" observations (in the sense that observations can themselves "give" you the "right" theory), but rather that a "good" theory (regardless of how you come up with it) should stick as close to direct observations as possible.
As for Einstein's quote, it's risky to interpret without more context (is there any?); it does seem interpretable in the spirit of Kuhn & Quine, in that theories emerge from the whole historical and subjective process of doing science, and then these theories themselves inevitably shape the way in which we interpret observations. Let's consider it in this sense for the moment (although i think the comment by John Forkosh offers a very reasonable alternative).
I think in both views, theories are seen as being generated from whatever is the existing practice of doing science -- but without any claim that a given theory can represent some sort of objective reality. Both would be in opposition to scientific realism, which, to various degrees, claims that science can indeed yield knowledge of what the world really is.
The difference here may be more in terms of the the attitude towards observations and the criteria for what is a "good" theory.
For Mach, the view may be that observations can be taken as objective data, and the important criterion for the goodness of a theory is how "economical" it is relative to such observations (i.e. the fewer "unobservable" concepts, the better).
For Kuhn/Quine, the view may be that observations are necessarily subjective, affected by existing theories, conventions, scientists' practices, etc; and so the criteria for the goodness of a theory cannot be based on a common set of objective observations, but rather the goodness of a theory must be evaluated in terms of internal consistency, etc, relative to its own "scientific heritage".
As far as the general question of where theories come from, I think at this point most philosophers of science might agree that the generation of theories is the "art of science" rather than some rigorous formal mechanism (even if certain basic principles are part of the common toolbox). One might describe it as a kind of creative data-fitting practice with high standards (of consistency, clarity, economy, etc). Theories become "real science" if/when they get some support from new data; until then, they are hypotheses whose "scientific value" is determined by the prevailing views of the contemporary scientific community.
