It’s easy to get seduced by the idea of Big Data. Data alone - well, that’s dull conversation fare. But throw in some hype, an adjective and a little capitalization, and you have this year’s big strategic topic. All that aside, there is one thing about this debate that makes it more important than the usual industry fad. Big Data is about Big Business, not just high tech startups.
Traditional companies are hoping that by leveraging massive data sets and real time prediction models - that they can start making realtime decisions just like a Facebook, Google or an Amazon. Some will succeed, most will fail but either way chances are that sometime in the next twelve months you will be either asked to approve a big investment in this space, or have to sell one yourself to your board. And when that happens, will you know what to do?
You have heard all the cliches, and they won’t help you. Data is more precious than gold. It is the new oil. Theory is dead, long live the algorithm. The sexiest job on the planet is a data scientist (seriously?). And so it goes. Yet ironically, despite the general exuberence, most people aren’t even sure what Big Data is, other than it must be considerably bigger than normal data. The real power of Big Data is the ability to make reliable inferences based on seemingly useless information. But what types of information are there?
For the sake of simplicity, let me outline the three major types of data, from a business perspective, that you will need to worry about in the near future:
The Data Cloud - social interactions, mobile communications and digital entertainment consumption - consumers are creating and having more information created about them than ever before. Today it’s Twitter and Facebook, but tomorrow the ever expanding Data Cloud will swell with information from mobile payments, smart energy meters, connected cars and all kinds of network connected sensors.
The Data Warehouse - most big companies have spent millions carefully collecting and organizing data that directly relates to their traditional business. This information, from customer records in a CRM system to inventory tracking data, is very useful when it comes to getting visibility into your daily operations. But as you may have already discovered if you have ever asked for a new kind of data report - it is neither flexible nor cost effective to make quick changes, to look for new patterns or combine your data with other external datasets.
The Data Trash Can - if you start digging around, you will realise that companies also collect a lot of other data, mostly inadvertently. Their systems record information about ambient temperatures in their stores, products sold on certain days of the week in certain outlets, the geolocation of their sales staff, the number of cars that enter their car park and so on. Some of this data is in their warehouse, some of it in other systems - but it may as well all be in the trash, because for most purposes, executives can’t get a handle on it in any reasonable time to make decisions.
When you look closely at these three categories of data in your business, you might start to understand that the true Big Data revolution is not just about the amount of data that has to be processed, but a mindset change about how data gets used in the enterprise. In an ideal world, you could combine the entire Data Cloud with information in your Data Warehouse and lost treasures from the Data Trash Can - then with the help of a giant IBM Watson type computer, end up with the answer to incredible profitability. Not as easy as it sounds, but you have to start somewhere. So here, in my view, are the five things you need to action next year:
1. Create a business intelligence team outside the IT department.
This may seem counterintuitive, after all Big Data involves serious tech. However, chances are the team running your legacy systems may not be entirely comfortable with open source, exotic Hadoop clusters as they are with maintaining their traditional IT platforms. Take a leaf from ING Direct. In order to get traction with thei
r plans, they created a new business intelligence team, formed from a combination of data warehouse people, analytics and marketing experts.
2. Focus on getting early internal case studies around small, tough problems
You may know that at some point your company needs to spend hundreds of millions of dollars on Big Data, but it will certainly be a lot easier selling that idea if you can demonstrate that there are real benefits to be had. My advice is to quickly identify a few, small, tough problems in your business, and then see if there are some simple data driven solutions to them. For example, T-Mobile in the US took that approach by targeting a problem they had with customer churn. By leveraging social media data, along with transaction data from CRM and Billing systems they were able to significantly reduce churn levels.
3. Don’t get too distracted by the expanding consumer data cloud
Yes, it’s exciting that you can see in realtime what every teenager in the world is saying, thinking and doing across Facebook and Twitter - but will overlaying this dataset with your rental car utilisation schedule really create that many new insights? Maybe yes, maybe no. But my prediction is that within the next few years there will be a lot more intemediary data providers who will do a much better job than you in eliminating the noise in the consumer data cloud. Different companies will specialise in different data types (e.g mobile location, social sentiment, retail or travel reviews), and allow you to subscribe to data streams cost effectively. For certain, keep an eye on trends in the consumer cloud, but don’t commit too much capital to it when there may be more to gain from leveraging up your own company data first.
4. Take visualization seriously
Never underestimate the power of pretty pictures to get management buy-in. Data on a page may appear meaningless, but on a giant wall screen moving in real time, can be pure magic. Find the data, combine it in new ways, and then get some good interface designers and creative types to bring it to life. A great example of a world class data design team is Stamen, but there are many others now emerging.
5. Adapt your decision making
In the end, none of your new investments in Big Data will bear any fruit if you don’t also work on changing your management style. Andrew McAfee and Erik Brynjolfsson wrote a great piece in the HBRabout how tomorrow’s leaders will justify decisions based on data rather than charisma. I only half believe them. Data driven companies are nothing new. Take P&G for example. But in a world of very large data sets, the hard part will not be just critically evaluating and articulating data assumptions but also being able to intuitively grasp new patterns as well.