The Petabyte Age – Will You Survive?

As someone who works in the online marketing space, I understand that my job is primarily one of math. In fact, one of the most significant differences between the interactive medium of the Internet and traditional offline channels comes down to the massive volume of data produced by people browsing the web. Succeeding in online marketing is highly correlated to interpreting and analyzing this data. It is not just OK to have the data, although it must be collected. Data must inform decisions.

In a recent Wired article, Chris Anderson explores the data intensive world of the Internet, through the lens of Google. His thesis is that the Petabyte Age, the age of what appears to be infinite data, represents the end of models:

There is now a better way. Petabytes allow us to say: “Correlation is enough.” We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

Google represents the ultimate data-driven company. Not only does it use tools like its own Google Website Optimizer to constantly tune and improve customer interaction with its sites, over the years it has produced one of the best pulses on culture – the Google Zeitgeist – has recently upgraded its trends tool, continues to buy or innovate in the online media space, and has more servers than McDonalds has hamburgers. They’ve got a lot of data and that’s just the public face of the company.

Anderson notes that the humble beginnings of the Stanford dropouts was really just about math applied to data:

For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn’t pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.

Later, he writes:

Out with every theory of human behavior, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.

Anderson believes the Google philosophy is “poised to go mainstream.” Google’s data-driven, computer-intensive approach will have significant impacts on science and will offer “a whole new way of understanding the world.”

I agree. The concepts in the Petabyte Age are only beginning to surface. Google is the model and the far surpassing leader. Those that follow their model, at some level, will survive. Those that fail to understand the importance of this paradigm will become obsolete.