Skip to main content

Why Big Data insights isn’t so much panning for gold as it is shooting fish in a barrel

Big Data as a concept has the potential to provide so many useful insights into our daily lives and business processes, but often organisations can get caught up in the hype.

Having access to tons and tons of data is not the only step to coming up with useful insights, but organisations can use it effectively if they know how. It has been estimated that for every day in 2012, 2.5 Exabytes of data were created (opens in new tab) – today the same figure is produced every few minutes, meaning that companies have to be more aware than ever before which data is useful and which is not.

Despite the vast quantities of data, discovering Big Insights needn’t be difficult. In order to use Big Data effectively, businesses should have some sort of goal in mind before they begin their analysis. Organisations sometimes make the mistake of thinking that simply running analytics tools on their data samples will magically add value to their business. Instead, organisations need to have an understanding of what they want to find out beforehand – only then can they apply the correct analytics tools, and spot potential flaws in the data.

Moreover, Big Data doesn’t just have to be about numbers. Often by simply focusing on the figures, businesses can leave out important contextual information, leading to misleading or erroneous conclusions. Incorporating text analytics into your Big Data reading can help you come up with valuable insights.

Companies should also look into visualising their results in order to spot correlations and patterns in the data. It’s not just about better presentation either. Graphical and other visual representations can make it much easier to glean important conclusions.

The ease with which we can gain Big Data insights doesn’t mean that organisations can forget one of the primary tenets of data analysis: correlation versus causation.

Causation means that A directly leads to B, but correlation simply means that A and B are observed at the same time. For example, over the last one hundred years global temperatures have increased while the number of pirates have decreased. The fall in the number of pirates, therefore, correlates with a rise in temperatures, but it certainly does not cause it.

Again, companies need to be aware exactly what they are trying to find out from the data before they begin their analysis. When answering some questions with Big Data, correlation may be enough, but for others causation will be required.

For many companies, the key is to think smarter, rather than bigger. The temptation of using more and more data may not lead to more accurate results, in fact it may serve to muddy the waters further.

Businesses should invest heavily in their data scientists because human input and contextual thinking is often required to formulate useful insights.

More complex algorithms and bigger data lakes will not, in isolation, lead to an effective use of Big Data. Instead, businesses sometimes need to think smaller to see the bigger picture.

The best ways of analysing Big Data through insights and more effective management will be on show at this year’s IP Expo (opens in new tab), taking place on the 20-21 May at Manchester Central

Register to attend IP EXPO 2015 FREE today (opens in new tab).

Barclay has been writing about technology for a decade, starting out as a freelancer with IT Pro Portal covering everything from London’s start-up scene to comparisons of the best cloud storage services.  After that, he spent some time as the managing editor of an online outlet focusing on cloud computing, furthering his interest in virtualization, Big Data, and the Internet of Things.