The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

From an article by Chris Anderson in Wired, issue 16.07

Sensors everywhere. Infinite storage. Clouds of processors. Our ability to capture, warehouse, and understand massive amounts of data is changing science, medicine, business, and technology. As our collection of facts and figures grows, so will the opportunity to find answers to fundamental questions. Because in the era of big data, more isn’t just more. More is different.

We are living in the petabyte age.

At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later. For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn’t pretend to know anything about the culture and conventions of advertising – it just assumed that better data, with better analytical tools, would win the day. And Google was right.

This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear. Out with every theory of human behaviour, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.

Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.


5 responses to “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

  1. Bullshit.

  2. The author of the article did not say that the scientific method has been suspended, only that for certain purposes something new works better. A Model T Ford will still get you where you want to go.

  3. While BS may be a bit succinct, the article is in the sub-Malcolm Gladwell range of ‘so what’. We can now forecast the weather a little bit better than before thanks to lots of data and lots of processing. Mentioning Google to try and give the thinking some grounding doesn’t work for me either. Spurious non-fact to try and give the thinking some credibility. Super maths were also responsible for the ‘no risk’ of CDO’s and other instruments that helped get us into the current financial mess.

  4. David Schatzky

    I met an evolutionary biologist yesterday. He told me he performs his investigations (intense research, scanning centuries of biological change) by sitting at his computer. He likes field work, but says it is no longer essential, except for occasional forays into the wild to collect new cell samples.

  5. I wish to add my voice to those who have trouble with the assertions of this article. Not only does the quantity of data not contribute to our ability to analyse it (a separate skill), the supposition that we have more usable data is not necessarily so. Governments’ capacity to measure and assess is on the decline. Private sector research is, surprisingly, not (yet) picking up the slack in many key areas. If we do not measure what we value, will we end up seeing value in only what we can measure?

    Mike Sky