Skip to main content

Do you have a dark data problem?

How much of your company's data do you actually use? According to search technology firm Lucidworks (opens in new tab), businesses typically only analyse around 10 per cent of the data they collect.

The rest becomes what the company rather ominously calls 'dark data' - information that lurks unused. Much of this data is unstructured and doesn't fit into any convenient database format. This means that companies don't have the tools to make sense of it or simply find they have too much to handle.

Lucidworks reckons that 7.5 septillion (7,700,000,000,000,000,000,000) gigabytes of data are generated worldwide each day and a large proportion of that will go dark. This data may contain sensitive information or details that could be used to commit crime, so it needs to be looked after. But it also has the potential to uncover details of undetected fraud, or boost business by revealing patterns in consumer habits, or finding untapped sources of revenue.

There's an interactive graphic (opens in new tab) on the issue of dark data on the Lucidworks site. You can see a short extract from it below to give you a flavour.

Dark data infographic

Ian Barker worked in information technology before discovering that writing about computers was easier than fixing them. He has worked for a staff writer on a range of computer magazines including PC Extreme, was editor of PC Utilities, and has written for TechRadar, BetaNews, IT Pro Portal, and LatestGadgets.