Einstein's quote is reported by Heisenberg in Theory, Criticism and a Philosophy. It is clear from the context that Heisenberg subscribed to a kind of positivist Machian philosophy according to which observations produce independent "sense data" that serves as the ground of theorizing. Einstein's rebuttal anticipated Quine's rejection of the positivist theory/observation distinction (in its logicized form advanced by Carnap) in Two Dogmas of Empiricism.
However, unlike the later proponents of the theory-ladednnes of observations thesis, Einstein apparently believed that it is compatible with maintaining what Putnam called "metaphysical realism" in physics. In other words, Einstein subscribed to a moderate version of the ladenness thesis, where theories may present different pictures of how observations proceed, and hence affect their interpretation, but there is still an independent reality that underwrites all of them, it is just described in different but intertranslatable ways. This was challenged by the Quine-Duhem thesis of radical underdetermination of theory by observations that left little room for Einstein's optimistic realism. In Epistemology Naturalized Quine describes how it leads to an even more radical version of the ladenness thesis advanced by Hanson and Kuhn in 1950s:
"The dislodging of epistemology from its old status of first philosophy loosed a wave, we saw, of epistemological nihilism. This mood is reflected somewhat in the tendency of Polanyi, Kuhn, and the late Russell Hanson to belittle the role of evidence and to accentuate cultural relativism. Hanson ventured even to discredit the idea of observation, arguing that so-called observations vary from observer to observer with the amount of knowledge that the observers bring with them. The veteran physicist looks at some apparatus and sees an x-ray tube. The
neophyte, looking at the same place, observes rather "a glass and metal instrument replete with wires, reflectors, screws, lamps, and push buttons." One man's observation is another man's closed book or flight of fancy. The notion of observation as the impartial and objective source of evidence for science is bankrupt."
Quine did not go this far, but even he rejected metaphysical realism and acknowledged that "we have stopped dreaming of deducing science from observations". Quine advocates a version of realism that combines observation sentences with pragmatic considerations in drawing pictures of reality, for a recent attempt to defend a more robust realism against the underdetermination challenge see Esfeld's The Rehabilitation of a Metaphysics of Nature.
But like most analytic philosophers Quine does not analyze the initial stage of creating scientific hypotheses, Popper is even bolder, declaring:"The initial stage, the act of conceiving or inventing a theory, seems to me neither to call for logical analysis nor to be susceptible of it". In other words, it is all just psychology, lucky guesses and bursts of genius. Indeed, philosophical reflection on creative processes in science has been scarce until very recently. Hanson was an early promoter of more serious research into it in his Is there a logic of scientific discovery?, and he only had Aristotle and C.S. Peirce to build on.
Peirce founded a theory of abductive reasoning that identifies classically fallacious but fruitful forms of inference and the "instinct for guessing right" as the scaffolding for constructing scientific hypotheses. For a more mature current form of it, model-based reasoning, complete with history of science case studies, see e.g. Nersessian's Mental Modeling in Conceptual Change. The basis of abductive/model-based reasoning are not observations alone, but a much more broad and diffused substrate of common sense, life experience, artistic analogies and metaphors, cultural gestalts, pragmatic assumptions, etc., but, contra Popper, it is broadly rational and susceptible to analysis.
Here is the full context of Einstein's quote as reported by Heisenberg:
"Einstein asked me to come to his flat and discuss the matters with him. The first thing he asked me was: "What was the philosophy underlying your kind of very strange theory? The theory looks quite nice, but what did you mean by only observable quantities." I told him that I did not believe any more in electronic orbits, in spite of the tracks in a cloud chamber. I felt that one should go back to those quantities which really can be observed and I also felt that this was just the kind of philosophy which he had used in relativity; because he also
had abandoned absolute time and introduced only the time of the special coordinate system and so on. Well, he laughed at me and then he said, "But you must realize that it is completely wrong." I answered: "But why, is it not true that you have used this philosophy?" "Oh yes," he said, "I may have used it, but still it is nonsense!"
Einstein explained to me that it was really the other way around. He said, "Whether you can observe a thing or not depends on the theory which you use. It is the theory which decides what can be observed." His argument was like this: "Observation means that we construct some connection between a phenomenon and our realization of the phenomenon. There is something happening in the atom, the light is emitted, the light hits the photographic plate, we see the photographic plate and so on and so on. In this whole course of events between the atom and your eye and your consciousness you must assume that everything works as in the old physics. If you would change the theory concerning this sequence of events then of course the observation would be altered." So he insisted upon that it was the theory which decides about what can be observed. This remark of Einstein was very important for me later on when Bohr and I tried to discuss the interpretation of quantum theory..." [emphasis mine]