Views on Customer Insight

A practical view on gaining real insight to drive better decision making

Big Data – AKA The Emperor’s New Clothes

Leave a comment

This post is brought to you by Tim Spooner. Managing Director at Consumer Cloud 

I’ve just read Imageyet another article on the new, revolutionary approach that is Big Data. Big ticket, definitely, Big Egos, invariably, Big load of B*&&*!ks more accurately.

Is it just me that’s sick and tired (I don’t think so given I’m not that bright), of countless articles espousing the virtues of this wonderfully spun suit from the finest cloth, using gold yarn and perfectly put together by master craftsman that seems unashamedly to set out to simply justify big ticket consultancy, technology and other such services that masquerade as a not only business changing but almost shaming those that can’t see it and don’t get involved as dim witted, luddites?


Almost more laughable are that the majority of so called Big Data examples clearly aren’t Big Data at all. For example,   I’ve just read one such article which cited the story of a New Jersey healthcare provider who was able to identify that 1 percent of patients were accounting for 33 percent of visits .No shit Sherlock!. This hardly requires a Big Data solution   by any stretch of the imagination. in fact in this case it is doubtful it even need a Single customer view crafted from multiple, disparate data sources – I would have though a bit of trivial analysis of the operational system storing patient details and visits could have delivered this nugget – or   maybe even a simple tabulation in early prehistoric code written on a Sinclair ZX Spectrum!

It was Einstein that said “Not everything that can be counted counts, and not everything that counts can be counted”. It seems to me that much of Big Data theory and discussions to date ignore the core truths of what Einstein spoke. All too many Big Data projects, like all too many data analysis projects before the word Big Data was even in our lexicon, think that if you through everything at it that the answer will miraculously reveal itself. Nonsense, if the data that matters isn’t within what you are analysing then you’ll not get the answer – or worse still you’ll get a wrong one that seems convincing due to the sheer volume of data that led to it being identified. Just because certain data exists does not necessarily mean it matters or can help in answering you key business questions.

As always, you need to think carefully about what you want to know and why? Then you can start to understand what data might actually be useful in finding the answers – or at least giving you some useful pointers that can be explored further. After this you can understand if you actually have the data that counts, if you don’t whether and how you can get it and then assess whether you can realistically (within an appropriate time and budget) progress any further. All too often, whilst it does sound as glamorous, doesn’t fill some vociferous technology providers pockets and doesn’t seem, well, so sexy – the answers are relatively easy to find. I’m reminded of the infamous example of the when NASA started sending astronauts into space, they quickly discovered that ball-point pens would not work in zero Gravity. To combat this problem, NASA scientists spent a Decade and $12 billion developing a pen that writes in zero Gravity, upside-down, on almost any surface including glass And at temperatures ranging from below freezing to over 300C …………………………………………………The Russians simply used a pencil!

I’d be interested in anyone’s views on this increasingly irritating subject and to see if there is anything we can do to start more realistic, worthwhile discussion around data.