How has [or will] the prevalence of “big data” – the exploding plethora of information and computing power to correlate it – impact[ed] (i) the scientific method’s theory/hypothesis formation, (ii) the underdetermination thesis, and (iii) the realist/relativist/ constructivist debate? The question might profitably be bifurcated in terms of the social sciences and the physical sciences.

closed as too broad by Dave, James Kingsbery, Swami Vishwananda, Dennis, Joseph Weissman Dec 31 '15 at 2:05

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • You might want to split this up into separate questions to avoid a "too broad" closure. Additionally, you might want to flesh out your question a bit more. It's a bit hard to tell exactly what you're asking beyond "is there any connection between these 3 rather different things and big data?" One thing that might be problematic, even when your question is fleshed out, is the relative youth of the field of "philosophy of data science". Most of the work I've seen deals with the ethical implications (e.g., privacy concerns). – Dennis Dec 28 '15 at 5:31
  • Dennis: Don't know quite how to break them up. I'll give it some thought. If you think about it, though,the issues are intimately (inextricably) related; when one considers that any [folk or scientific] knowledge claim is underdetermined by the evidence/data cited in support thereof, and its epistemic implications. – gonzo Jan 14 '16 at 6:38

This is a partial answer only to (i), but its more than I can put in a comment:

I have seen work done on generating hypothesis from data mining the texts of large databases of biomedical papers in the bio-medical filed. The idea is pretty straight forward: Establish correlations between medical conditions on one hand and certain substances/enzymes/proteins etc... on the other hand, based on connections found in the literature that a human researcher wouldn't have time to discover systematically and could only stumble upon by accident. This allows researchers to search for previously unknown potential causes and/or cures for these conditions.

This doesn't challenge any epistemological principle per se, but the systemization of hypothesis formation ("That x molecule has an effect on y condition") does have interesting implications for the way science is conducted and the role of the scientist in the process.

There are some caveats: When I spoke to the researchers working on this in 2009, it was still a work in progress - no actual cures or solutions had been discovered using this method yet.

I asked can it be applied to other sciences besides the biomedical domain and was told that the format of biomedical publications was much more amenable to such procedures than other fields. Presumably at some point text mining technology will be advanced enough to be able to do this for domains such as physics or mathematics, etc.....

More recently, researchers at MIT, created a Data Science machine which "replaces" human intuition in the process of creating predictive models, again by mining correlations between the data in a systematic way. Their machine was able to compete against human data scientists, and beat most (but not all of them).

On a similar note, I've always wondered why mathematics and theoretical physics departments didn't hire a bunch of developers to set up large clusters churning through potential theorems (suitably encoded) and trying to prove or disprove them with genetic algorithms, simulated annealing or survey propagation. Whenever they stumbled upon a positive (i.e. a theorem that is true), they would bring in their human mathematicians to verify and then edit and publish the result if it is relevant.

  • Great answer to (i). This is precisely what one would expect. But consider the potential implications of your establishment of “… correlations…based on connections that a human researcher wouldn't have time to discover systematically and could only stumble upon by accident,” to Quines indeterminacy thesis, and the support that thesis provides to skeptics/constructivists. Just imagine the plethora of rhetorical fodder -- the marriage and full employment act for data miners and rhetoricians. – gonzo Dec 9 '15 at 21:05
  • Not to mention, sticking to Quine, imagine how the increased number of these “correlations” might entail a correspondingly increased number of “ontological commitments”, and reified objects. – gonzo Dec 9 '15 at 21:05
  • @Gonzo your comment reminded me of another relevant datum. See edit. – Alexander S King Dec 9 '15 at 21:08
  • Consider the effect of social media and endless cable channels on the general public's world views and ideologies. Big data + automated theorem proving will probably have a similar effect, everyone will just endlessly spend time trying to confirm their own theories. I suppose that at some point the necessity and difficulty of conducting lab experiments will put a break on the effect, but its still a scary thought. – Alexander S King Dec 9 '15 at 21:37
  • Precisely. If you got the time -- or money to have someone else do it for you -- with all the data out there to be mined, and different ways to correlate it, practically any claim will be justifiable/establishable. The very notion of statistics may evolve in ways not presently predictable. Thank you so so much for the "Data Science Machine" article, by the way. Very Cool. – gonzo Dec 10 '15 at 3:59

This is a very interesting question that would seem to have both obvious answers and deeper ones. Since I know little about it, I can only pass along what amounts to hearsay.

Obviously, massive data crunching impacts the sorts of questions that can be asked and potentially falsified, more or less across the board. One interesting case is practical or "experimental" mathematics. Paul Erdos, it is said, refused to believe the "logically correct" answer to the "Monty Hall Problem" until the results of massively repeated trials were computed by brute force. And this seems epistemologically a very peculiar sort of development, a confirmation of a prediction not by "nature" but by a machine replication of the logic that proposed it.

Science is partly pattern recognition and partially observation of "objects." Like the telescope or microscope, data crunching "reveals" things that could not be seen before. But what are these things? The ontological status of a "correlation" that is purely statistical rather than linear and causal is now mainstream science, since Darwin or Boltzmann, but was still an unsettling development for many, such as Einstein.

As @AlexanderKing astutely points out, "correlations" are infinite and may have some potential to subvert the sort of "enabling constraint" offered by experiment. There was originally some question for his contemporaries as to whether Galileo's telescope was purely transmitting or actually constructing the "evidence" it presented. Since very little science can now be done without the "sensory apparatus" of massive data crunching, we seem to be slipping unavoidably into a more "constructivist" circle of confirming-predicting. Sorry, these are more musings than answers. I hope more are forthcoming.

  • 1
    Don't apologize. This is very good. Both of your answers have picked up the scent of the kind of contemplation I sought to sow by the set of questions. What might Quine's idea of the underdetermination of theories by the supporting evidence look like in a hundred years. Would that be as a result of [epistemological?] movement/evolution in the direction of metaphysical realism or constructivism? And how will those two positions be distinguished or unified (using the instrumental concept of "ontological commitment"? etc. But no one seems very interested in taking a stab at it. – gonzo Dec 11 '15 at 4:04
  • Well, you might refocus and ask in some different way. One problem is the "obvious" or "shallow" answer that massive data crunching and statistical approaches affect all science across the board, what can be asked and answered. I'm not sure there is any particular implication for the epistemological principles you cite, as opposed to others. The query can easily slide into: "how do computers affect society?" So maybe a more specific case. There are lots of "big data" books out now, but I haven't read any and don't know which tackle more philosophical issues. – Nelson Alexander Dec 11 '15 at 15:01
  • No. Had the question been asked in a different tone (more technically, differently focused), it may not have elicited your and King’s so very salient responses. You likened the issue to the issue of “…whether Galileo's telescope was purely transmitting or actually constructing the "evidence," noting that “….very little science can now be done without the "sensory apparatus" of massive data crunching…” [ergo?] “…we seem to be slipping unavoidably into [constructivism], a “circle of confirming-predicting.” – gonzo Dec 13 '15 at 3:00
  • 1
    Who (maybe Wittgenstein) would have thought, say, 30 years ago, that Sellars’ [Kantian/Wittgensteinian] notion of the “myth of the given” would become so ideologically entrenched that computational/cognitive notions [ie thinking, reasoning] such as “massive data crunching”, would be likened to “sensory apparatus”, such as a telescopes. Or consider that this confluence would entail an “unavoidable” slide into constructiveism. – gonzo Dec 13 '15 at 3:01
  • Can't really answer. As a complete amateur who has only read in "continental" lineage, Sellars' "myth of the given" seems axiomatic since Kant, at least. That science inherits much of philosophy and eventually sees itself as "statistical" or "probabilistic" is, as I said, both obvious and deep, but by "deep" I only mean awaiting the best reframing. – Nelson Alexander Dec 13 '15 at 3:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.