By Stephen Smith, Eckerson Group.
The hype over big data is on the wane.
The cloud, Hadoop and their variants have solved it.
But ‘big data’ is where a lot of people are still spending a lot of money building bigger infrastructures to process, hold and curate these immense databases.
This blind pursuit of ‘big’is generating some sizable and avoidable costs in infrastructure andhuman resources.
It is now time to change the discussion from ‘big data’ to ‘deep data’.
Instead of collecting all the possible data to achieve ‘big data’ we now need to be more thoughtful and judicious.
We need to now let some data fall on the floor and seek out variety over volume and quality over quantity.
This will have many long term benefits.
To understand this transition from ‘big’ to ‘deep’ let’s first look at some of mistaken notions we have about big data.
Here are some of the big myths:Here are the realities:The problem with believing these myths is that you will architect your information systems in ways that look good on paper or in the long run but are too cumbersome in the immediate time frame to be useful.
Here are four problems with blindly believing that ‘more is better’ when it comes to data:There are several things that you can do to combat the ‘dark side’ of big data and move towards a deep data mindset:If you focus on deep data rather than just on big data you will enjoy many benefits.
Here are some of the key ones:Big data and the technological breakthroughs that have supported it have greatly advanced many companies drive to become data-driven in their decision making processes.
With the rise of AI and our ability to saturate even those powerful resources we now need to be more precise in what we need from our data.
Building a culture of understanding of deep data and not just big data is what is needed now.
Bio: Stephen Smith is a well-respected expert in the fields of data science, predictive analytics and their application in the education, pharmaceutical, healthcare, telecom and finance industries.
Resources:Related: var disqus_shortname = kdnuggets; (function() { var dsq = document.
createElement(script); dsq.
type = text/javascript; dsq.
async = true; dsq.
src = https://kdnuggets.
disqus.
com/embed.
js; (document.
getElementsByTagName(head)[0] || document.
getElementsByTagName(body)[0]).
appendChild(dsq); })();.