When it comes to analytics, enterprises today have a surplus of data but a shortage of insights.Why the data surplus? Data warehouses, data lakes, and other repositories are brimming as volume, variety, and velocity continue to grow. Even as the tide continues to rise, enterprises are tapping into new data sources, such as social media and Internet of Things sensors, in order to gain new analytics opportunities. Today’s enterprises seek both to analyze more of their data and to reduce the costs related to their unanalyzed dark data.
When it comes to analytics, enterprises today have a surplus of data but a shortage of insights.
Why the data surplus? Data warehouses, data lakes, and other repositories are brimming as volume, variety, and velocity continue to grow. Even as the tide continues to rise, enterprises are tapping into new data sources, such as social media and Internet of Things sensors, in order to gain new analytics opportunities.
This explosion of information has made it difficult for some IT teams to generate the necessary insights thanks to bottlenecks like cumbersome manual coding for Extract, Transform, and Load (ETL) processes, a lack of understanding about how data is being used and, more importantly, how it can be used. The result: dark data.
Dark data is collected and stored as part of typical business activities, but it’s not used for anything other than compliance and retention purposes. Forrester estimates that the average enterprise analyzes just 37 percent of its structured data and 22 percent of its semi-structured and unstructured data.
And that dark data matters for two reasons. First, this data costs money to capture and manage, and it often necessitates capacity upgrades for premium data warehouses. Second, dark data can hold latent analytics insights that enterprises are failing to realize.
Today’s enterprises seek both to analyze more of their data and to reduce the costs related to their unanalyzed dark data. Here’s how enterprises can begin to achieve both goals.
We find that enterprises can best reduce the amount and cost of dark data by adopting three basic best practices.
IT organizations understand that dark data is unused data, and that unused data assets are liabilities. By extracting new value from once-dark data and reducing management costs associated with dark data, they can improve the economics of their analytics initiatives.
NOTE: This article was originally published on Data Informed.