There was the gold rush and the oil boom. Now, there's the data dash. With the digital age still in its infancy, we're just beginning to mine and refine the mountains of data accumulated from our mobile devices and Internet activity. And while all of this data gathering sometimes leads to anxiety -- often misplace anxiety -- it's important to remember that it allows us to learn an incredible amount about ourselves and society-at-large. Let's examine some of the possibilities of how Big Data may change society for the better:
1) Individual-Based Governing
Last year, Alexander Pentland of the MIT Media Lab put forth a fascinating presentation on Edge.org: "Reinventing Society in the Wake of Big Data." In his presentation, Pentland proposes that the most crucial data we're unearthing is behavioural data -- and that includes everything from location data from mobile devices to transactional data from credit cards and mobile wallets.
This kind of data is already fawned over by marketers in order to target consumers with the right ads and offers, but it can also be applied to how we govern our societies. Pentland told the New York Times that much of the way we think about governing is grounded in Adam Smith and Karl Marx and their conception of markets and classes. "But those are aggregates," Pentland told the Times. "They're averages."
Behavioural data, conversely, provides governments for the first time with the ability to track social phenomena at the individual level. That means that government incentive and benefit programs could potentially be customized on an individual level through powerful algorithms, eliminating the inevitable waste that comes from working with averages. Just imagine, to take one example, how much more efficient Social Security could be if we adjusted payments based on local fluctuations in cost of living.
2) Digital Immortality
In bygone years, people lived on through a few physical remnants: photographs, some old shirts, a shoebox full of old letters. But now, much of our lives are being tracked and recorded in data sets.
When you consider how we're connected most of the time -- and how smartphones and Google Glass will allow us to record and share more and more of our life experience -- we are likely headed towards a future where 90 per cent of someone's life experience is recorded in some sort of data. Humans will be able to examine not only the lives of the billions of people living on this planet, but also billions of people who came before us. Even when we're gone, our lives will stay alive in data, available for anyone who wishes to examine it.
Google is already anticipating this development. Last week, they became the first company to provide users with a tool to decide what happens to their data when they die.
3) Predicting the Future
On his blog, Numerate Choir, Silicon Valley data geek Mike Greenfield has a fascinating post on how Paypal used their robust armory of data to create statistical models that predicted the future -- in this case, that future was fraudulent transactions. Those models allowed Paypal to "identify (and ultimately stop) half of the fraud in the system while blocking or examining well under 1 per cent of the transactions."
Best of all, this allowed Paypal to delegate its manpower in the most efficient way possible. If a transaction had a 0.000001 per cent chance of fraud, Paypal need not worry about it. But if a transaction had a 2 per cent chance of fraud, Paypal could have someone on the case monitoring it.
The increased use of data in this way will allow governments, businesses, and humane organizations to delegate their resources with much greater efficiency in the future, but it can predict other things as well. We're already seeing how sites like Football Outsiders use complex combinations of data and algorithms to correctly predict the results of sporting events, and how Nate Silver famously used polling data to correctly predict which candidate would win every state in the last presidential election.
At Circle of Moms, Greenfield has expanded his predictive modeling to create the most appealing possible content for Circle of Mom's audience on any given week. Meanwhile, companies like Netflix and Amazon are also using data modeling to present users with the most appealing content possible.
These practices are getting better and better each year, and we're moving towards a future where brands and governments know what we want and need before we even know it ourselves. That's the revolutionary power of Big Data, and it's going to make the world a better place.