Skip to main content

Overview: Getting value from Big Data

This article was originally published on Technology.Info.
As part of our continuing strategy for growth, ITProPortal has joined forces with Technology.Info to help us bring you the very best coverage we possibly can.

Big data is far too important these days to be left entirely to data scientists. In a world awash with data, a working knowledge of analytical methods, technologies and terminologies is an essential business skill for business managers.

After all, they’ll be the ones who’ll be signing big cheques for big data - and on average, organisations will pour, on average, around $8 million each on

big data initiatives

in 2014, according to IT market research firm IDC. The company’s survey of over 750 technology decision-makers worldwide, released earlier this year, found that in many cases, the spending spree has already begun. Over two-thirds of respondents from large enterprises said their organisations had already deployed or planned to deploy

big data projects

. Over half (56%) of respondents from SMEs said the same.

In this chapter, we take a look at some of the ways that organisations are getting business value from big data and the tools they are using to extract new insights. For example, we take a look at the burgeoning market for SQL-on-Hadoop tools and chat with Mike Olson, co-founder and chief strategy officer at leading Hadoop distribution specialist


, about how the market is developing . We also explore the ‘smart city’ concept - a technology intersection where big data and the

Internet of Things

collide, with a look at how two organisations are using the data generated by smart meters to encourage consumers to reduce their energy consumption.

Given most organisations’ spending intentions in these areas, it’s vital for bosses to have a clear understanding, upfront, on the value they hope to get from their investments, but they will be entering unknown territory.

The good news about big data is that it makes it possible for executives to ask questions they’ve always wondered about - and get rich, multi-dimensional answers back in return. The bad news about big data is that traditional methods and tools for data analysis, with which bosses might enjoy a passing familiarity, have a nasty tendency to fall short in environments where the pressure is on to explore vast volumes of both structured and unstructured data, from a wide variety of both internal and external sources.

In other words, companies looking to get business value from big data must familiarise themselves with a world populated by new technologies with strange, unfamiliar names: Pig, Hive, Flume and Squoop, to name but a few.

On the plus side, cloud technologies will help: there are many ways now to access big data tools and expertise on an on-demand basis, and the IT infrastructure needed to store and process big data can be supplied by a host of

cloud providers

, such as Amazon Web Services and Microsoft Azure.

With smart choices, big data can mean big insight - if organisations have the ability to turn insight into actions that create competitive advantage. Delivering on that final step, in fact, will likely be the biggest challenge of all.