John Stuart Mill provides a group of methods which characterize an empirical approach to scientific discovery which he describes in his book "A System of Logic", see the SEP article on him, and references therein. Mill's approach is empiricist, meaning that for him we should start with the raw data, and then use various rules of inference, what he calls "“Methods of Experimental Inference” (System of Logic, Bk. 2, ch. 9), to discover laws and patterns. See also Mills methods.
An opposing view comes from William Whewell, who saw that science has an a priori nature: One starts with a fundamental idea, and then reconciles it experimental data. From his work "The History of Scientific Ideas":
[Fundamental Ideas are] “not a consequence of experience, but a result of the particular constitution and activity of the mind, which is independent of all experience in its origin, though constantly combined with experience in its exercise”
Whewell was also notable for looking at the history of science to understand the scientific method.
This same concept, that the history of science is intimately tied with the way science is and should be practiced, was taken up later by Thomas Kuhn and most radically by Paul Feyerabend. The idea is that philosophers shouldn't just sit in their armchairs philosophizing about what the best way to do science is. Instead they should look at the history of science and examine how the greatest scientific discoveries and theories were achieved to understand how science should be conducted.
In particular, Feyerabend uses examples from both the history of science and contemporary science (especially the historical example of Galileo) to show in his book "Against Method", that there is no proper method for scientific discovery. His opening statement in the introduction of the book is the following:
"Science is essentially an anarchic enterprise: theoretical anarchism is more humanitarian and more likely to encourage progress than its law-and-order alternatives."
And
"This is shown both by an examination of historical episodes and by an abstract analysis of the relation between idea and action. The only principle that does not inhibit progress is: anything goes." (Final emphasis his)