Over the past couple of years Big Data has been a topic that's been gaining momentum in organizations, consultancies and ad agencies and they've all been struggling to work out a strategy of how to deal with it.
I've spent the last 15-plus years wrestling Big Data, driving analysis and formulating insights that lead to strategies to help grow business. Now more than ever I find the world to be more confused about what to do with data than ever before. The curveball that has hit people has been the rise of web and social activity by customers, communities and businesses.
In a recent study it's found that 64 per cent of organisations have invested in big data or plan to do so and 55 per cent say that they are doing so create "enhanced customer experience." Some 70 per cent of data being analyzed is transactions followed by log data at 55 per cent. Wait. Is that new? Not really. Transactional data has been analysed for well over 25 years. What this says to me is that organizations are still struggling with the basics and are now trying to jump into the deep end.
In addition to the already Big Data world of transactional data we have Petabytes (that's big) of data created every day as people navigate the hyper-connected world and information-hungry world. Searches, updates, uploads, check-ins, likes, RTs, comments, app usage and browsing can really help us understand customer behaviour. Increasingly though, people are reluctant to connect with brands in ways that allow them to take advantage of the data they produce.
Just hashtag your picture and you'll be in the running for a prize. Check-in and we'll give you a discount. Give us your opinion and feel like you've been heard. It's a world full of customer wins but what the hell can we do with that data? Can we know you better? Can we predict a behaviour that applies across our customer base? Maybe a little, but it is fuzzy logic at best and the gut feel reflex is alive and well but with a little more colour.
Collecting and processing data is hard and now digital and social data is confusing matters and making our job even harder. The foundation of analysis still comes from transactional a data that is created every day and we need to veneer that with the insights from web and social activity as well. Organizations need not only data processing expertise in house but also people that can reduce it to a useful set of data, combine it with insights from social data so that it provides useful insight for smart thinkers to plan holistic customer experiences.
Over and above that we need the following to be onboard as well:
- Customer service. Processes need to be in place to act as the human face of data collection and great customer experience. Communicating with customers in a more familiar (but not creepy) way will help build the relationship and strength of brand. Then inviting customers to provide their data with clear benefits in a way that is human and connective will yield more, and better quality, data. Just pulling data from channels and then cleaning data will mean less data and less quality overall.
- Data operations. Identified individuals in organisations need to own the data and need to have that written into their job descriptions with clear accountability for actions and decisions relating to that. Ask yourself. Who owns the customer? Who owns the social follower? What happens when the two overlap? Who will help us guide a customer experience? Scenarios for creating customer experiences, communications, service and even campaigns must be considered holistically and very carefully or the apple cart will be upturned and the customer relationship will be affected or, at worst, broken.
- Data processing. This is more than just machine-processing muscle. Organizations still need to not only watch the rules and regulations that they must adhere to around data but they must also adapt over time and feed insights back into the rules engines they build. This is a lot easier for transactional and log data but very hard for social data. Organizations are desperately trying to bring web and social data into their standard processing. It's costly and sometimes futile as it's hard to key data together. I recommend moving this hard task, for the time being, into the next bucket, "data analysis and strategy."
- Data analysis, strategy and experimentation. OK, this is where the rubber hits the road. Organizations will build bigger and stronger analytic capabilities over time. In fact, more than ever before. They will also split these into simple (and automated) analysis on transactional data where predictive models will aid service and targeted communications and more manual, in-depth analysis that spans the fuzzy data from social and the outliers of behaviour in your organizational data sets. Strategists will have to create a 'third-eye' to get the most out of it. One eye on behaviour we know over time, one eye on behaviour we think is happening every day and what impact that has and lastly (and kind of spiritually) eye on those behaviours that are starting to emerge in the corners of the Internet that will ripple across all data, including transactional data. That being said, hypothesis-based experiments will be 'de rigueur' in this new world and that will jar organizations.
It's actually scary to think that often organizations neglect the final part here and feel that taming data and storing it with a existing processes for deriving insights will give them a competitive edge. It's not only the organization's fault though. Even many strategists cannot think using this 'third-eye'. Everyone needs to recognize that they need to change.
Organizations need to start planning holistic training on Big Data and Strategists need to broaden their thinking. In they don't smaller, and more agile, competitors will start to erode your business.