The facts behind the hype: What the Gartner big data 2013 report means for the industry

Gartner released its 2013 big data survey recently, reinforcing some of the long-held beliefs surrounding big data, and backing them up with survey evidence.

While the survey primarily focuses on companies currently working with Gartner, it's still a broad and impressive overview of the industry due to the sheer number and variety of businesses involved.

Staggering big data potential

One of the most striking results to come out of the survey was that 64 per cent of companies are planning or have already implemented big data systems. This is a significant number, demonstrating as it does a genuine drive within companies to adopt the latest systems. The fact that more than half of respondents are looking at new ways to use data, and investigating new data science initiatives, will mean a massive boost for industry growth.

This is especially pertinent when discussing the future of the industry, as this 64 per cent represents overall plans for implementation. Perhaps surprisingly, only eight per cent of those who are planning on implementing big data solutions have actually made moves towards it. This 92 per cent gulf represents a long list of potential clients for consultants, suppliers of cloud database products and technology providers.

The 92 per cent potential is a sign of the longevity we can expect from the big data industry in the next decade. Far from a flash in the pan, with this kind of demand and potential revenue, big data investments will be safer, and the industry will have a better foundation for growth.

Banking and media are the industries making the biggest strides in this area. That these are also two traditionally affluent industries is unsurprising, but the example of the banking sector especially gives clear evidence that there's a business case for utilising this technology, and a definite profit to be made.

Why are these companies looking to implement big data?

There have been countless success stories of companies using big data to make billions. The Facebooks, Googles and YouTubes of the world have set a strong precedent for how correct data usage can have a significant impact on revenue. Following in the wake of these behemoths, it's unsurprising that many other companies are looking at how they have been using their data.

In each of these cases it has been about customer experience - using the correct information at the correct time to create money-making situations. This aligns with the survey findings, where the most important reason for wanting to implement big data systems is the improvement of customer experience.

However, according to the survey, organisations struggle most with understanding how to derive value from big data. This means that although the merits of a successful implementation are evident from the success of several companies, individual companies find it difficult to determine how to access the system's many rewards.

This is largely due to one of main stumbling blocks for new data implementations - talent acquisition. This is a key concern for many companies, as the conversion of meaningful data analysis to actionable business strategies is difficult and requires the kind of skills that are difficult to find in such a new and quickly-growing industry.

In Gartner's discussions with companies looking at implementation, they found that despite the drive towards new data initiatives, skills are often not sufficiently available to achieve significant business gains, and big data implementation often stumbles after the first experimental stages. After initial failures, business leaders can become reluctant to move forward after these tentative steps promise much but deliver little.

What are people doing with big data?

Despite the hype around new types of data such as text analysis, image analysis and sensor data, the vast bulk collected is still transactional in nature. This is a testament to the focus on customer experience found in many companies today, and how they associate certain individual actions with viable business processes.

Gartner's studies revealed that there's an increase in the use of machine-gathered information such as sensor-fed data collection. With the increased use of the sensors found within electronic devices, this is only likely to increase in the future.

Overall, the Gartner report backs up many of the widely held beliefs within the industry. As the famous saying goes, "if it can't be measured, it doesn't exist," and this report puts numbers behind assumptions. The report thus gives genuine weight to what industry insiders have been saying for a relatively long time.

The Gartner report will give companies looking to implement a real drive to invest in big data service initiatives while also giving practitioners the confidence to expand. We know that one of the main reasons startups fail is aggressive expansion too quickly, creating unsustainable business models with no genuine numbers behind future business.

This report will hopefully give these companies the confidence to go ahead and expand to fill what will be a shortfall in the market within the next decade.

Gil Allouche is the vice president of marketing at Qubole. Gil began his marketing career as a product strategist at SAP while earning his MBA at Babson College. He is a former software engineer.

Image credit: Flickr (Chris Devers; loungerie)