Have you Ever Come Across the Term Dark Data?

Are you among those who still try to wrap their minds around the volumes of Big Data being left unattended in their systems? If so, this might help improve offers and personalization but at the same time, we might have a new thing to lose sleep over, i.e. Dark Data. What is Dark data? It is digital information, especially that is not being in use. According to Gartner, dark data is information assets that an organization collects, processes and stores in the course of its regular business activity, but generally fails to use for other purposes.

There are times when an organization leave data dark for a few practical reasons. The data might be dirty and by the time it can be scrubbed, the information becomes too old to be useful. In such scenario, records may contain incomplete or outdated data, be parsed incorrectly or be stored in file formats or on devices that have become obsolete. Now as the term dark data is being more associated with big data and operational data, for instance, server log files that could provide clues to website visitor behavior, customer call detail records that incorporate unstructured consumer sentiment data and mobile geolocation data that could reveal traffic patterns would help with business planning.

The Following Categories are Usually Considered in Unstructured Data

•   Customer Information

•   Log Files

•   Previous Employee Information

•   Raw Survey Data

•   Financial Statements

•   Email Correspondences

•   Account Information

•   Notes or Presentations

•   Old Versions of Relevant Documents

How to Put Dark Data to Work?

Do you know what is the most crucial point here? It is to understand about dark data that it doesn't have to remain dark. Yes, it may quite interest you to know that there are several big data companies that have the potential to take dark data and leverage it to gain insights, the data becomes actionable and is no longer dark.

For Example

•   Networking Machine Data– As noted above, servers, firewalls, networking monitoring tools and other parts of your environment generate large amounts of machine data related to network operations. In addition to this, avoid dark networking data by using this information to analyze network security, as well as to monitor network activity patterns to ensure that your network infrastructure is never under- or over-utilized.

•   Customer Support Logs– Most of the businesses have this tendency to maintain records of customer support interactions that include information such as when a customer contacted the business, especially in terms of which type of communication channel was being used, how long the engagement lasted and so more. Don't make the mistake of leaving this data in the dark, or using it only when you need to research a customer issue. Instead, build it into your analytics workflows by leveraging it to help understand when your customers are most likely to contact you, what their preferred methods of contact are and so on.

•   Legacy System Log– If you have mainframes or other older types of systems running in your environment, then chances are there that you may think that there is no way to use modern analytics tools to understand them. But by simply offloading system logs and other data from these systems into an analytics platform like Hadoop, you can make sure you are not leaving this "legacy" data in the dark.

So are you ready to make the most out of the Dark Data?