"Too much ends in smoke.”
― Toba Beta, My Ancestor Was an Ancient Astronaut
I have just attended a BI conference, where the major subject - again – was the concept "Big Data", i.e. the common name for unstructured information, collected on the internet. Two days, where I have been sitting as a patient "disciple", listening to speakers from large corporations and consultant agencies, talking – again – about the “wonderful world of big data”.
The angle to the subject is almost always the same: information on facebook, twitter, instagram, blogs, newsletters, mails, affiliate traffic, etc. is so valuable for all business activity, that companies need to focus all analysis resources in collecting and analyzing Big Data in order to beat the competition and provide cutting-edge marketing! “Better start today, than tomorrow! ”, seems to be the key strategy in order to avoid total market exit.
There I sit in the auditorium, being “all ears” everytime, due to the fact that I agree “Big Data” to be an invaluable amount of information regarding customer behavior, demand, opinions and thoughts. All flying around in cyberspace in an accumulating speed and scale.
Invaluable, due to two reasons:
- The amount of “participants”. Hundreds of millions of “volunteers” of all ages, social classes, nationalities and cultures all over the world. Needless to say, a survey on this scale and variety is practically impossible.
- The nature of the information. We are not talking about feedback based on predefined surveys or structured interviews with clearly defined focus groups. Procedures we normally use, when we collect information about customer behavior and mindset. No, what we have is a tremendous amount of unfiltered and unstructured thoughts and attitudes, knowledge sharing, “do’s and don’ts”, impulsive behavior, short term needs and long term dreams regarding almost everything. Due to the fact, that modern people have a need to practically write whatever they think on social medias and take pictures of whatever catches their attention, we are really talking about a "crystal ball" of feelings and mindsets to an extent, we won’t be able to collect elsewhere. Not even close.
So far so good. As professional information analyst, I have no difficulty in seeing the potential in “Big Data”.
However, every time I hear these speeches, I keep asking myself the question, which almost no speaker has an answer to: “How?”
Every specialist working with Business Intelligence knows, that it all comes down to one question: ”Does the end result give a true picture of the real world/business/market ?”.
If you are a small carpenter managing your own business, and the monthly finance report suddenly shows a turnover of £100 mill., you probably get the feeling, there “is something wrong” with the numbers. This is a natural reaction based on experience, logic, and the fact you’re dealing with “numbers”, which are structured and easy to measure.
But “Big Data” is not only numbers, it’s EVERYTHING!! Numbers, text, pictures, symbols….the lot!!!.
You may be able to develop a programme, which is capable of recognizing one from the other, but how do you dechifer a statement, demand, written feeling or dream, all information based on cognitive(based on feeling) values rather than relative values as numbers ?
I mean, how do you tell a programme the difference in value of the word “Great!” in these two statements:
•“I took the best picture with this camera. Isn’t it GREAT!”
•“I only had this shitty camera on my holiday…..GREAT!”
When you read these two statements, you can tell the difference between sincerity and sarcasm. Right?
But how do you tell a programme to dechifer the cognitive values on all information on Facebook or Twitter and set it up in a way, which gives you a true picture of the overall attitude regarding a particular subject?
I have read about highly advanced holistic programmes developed in research laboratories, which are capable of dechifering cognitive based information. One thing is a controlled experiment in a closed lab, but to do the same on Facebook with millions of users…………..? We are talking "a giant step for mankind", here !!
Back in my seat in the conference room I raise my hand and ask my question again. This time to a Business Intelligence Director from a big financial institution in Denmark, who has just spent a hour explaining their “Big Data” visions: “ I have heard and understood your “Big Data” strategy, but how will you carry it out in practice?”
The reply shows, quite well, the state I consider all the fuss about “Big Data” to be in: enthusiastic, but pointless – “We are not quite sure, but the sooner we start, the better!”