Big data trends: Hadoop is the clear frontrunner for enterprises

Big data has sparked a growing trend among businesses to rely less on intuition and more on data analysis when making business decisions. Ambitions of better customer segmentation, improved customer service, enhanced product creation and of finding meaningful customer insights has many business leaders paying attention to the developments in big data technology that have taken place the last couple of years.

For many, Hadoop has become the technology of choice as the two terms, big data and Hadoop, have become almost synonymous with each other.

A clear frontrunner

In a recent survey, IDC asked businesses about their plans for implementing Hadoop, and the results showed that for enterprises looking for a big data solution, Hadoop is the clear frontrunner.

Of the businesses surveyed, 32 per cent said they had already deployed a Hadoop solution. An additional 31 per cent said they planned to deploy Hadoop within the next year, and 36 per cent said they planned to deploy Hadoop but it would be more than a year before it was deployed.

Analysing raw data

What are the motivations for such a large chunk of businesses to be planning on deploying Hadoop? The survey found the primary motivator to be the analysis of raw data, from operations data to transactional data to customer behaviour data, which makes sense given that one of the biggest obstacles business face right now is managing the huge amount of multi structured data being created every day.

A second motivator businesses indicated was service innovation, including modelling "if-then" scenarios of products and services. Some other motivators mentioned included replacing an older data warehouse and using Hadoop as a platform for Web analytics and content-sharing.

Business benefits are, of course, the ultimate measure of whether a Hadoop solution is a worthy investment or not. When asked what the quantified business benefits of their Hadoop solution were projected to be, 82 per cent of those surveyed were able to provide a numerical range. The most common response (about 24 per cent of those surveyed) was that the business benefit would range from $5 million to $9.9 million.

A mix of databases

A common pattern the surveyors found was that businesses tended to use a mix of databases in order to complete big data analytics. NoSQL databases, such as HBase, were most commonly mentioned as being used in conjunction with Hadoop after the traditional database that most businesses had already been using for many years.

Another trend surveyors noted was the use of commercial options of Hadoop, rather than the open source version, which requires a lot of manual setup and maintenance. Respondents said the main criteria they used when selecting a Hadoop platform was support, management and storage costs. For respondents who used Hadoop for critical business operations or sensitive data, using an alternative system to HDFS, similar to what MapR offers, that was more secure and reliable also became important.

Finally, the survey addresses how businesses handled data security in their Hadoop solution. A large majority indicated that sensitive information is removed before it is imported into Hadoop and after it is exported from Hadoop. Common security measures in place include storage-based cloning or snapshots and data protection software.

While there are always some challenges businesses must deal with when implementing a new technology, Hadoop offers businesses a key opportunity to improve customer satisfaction, speed up production and increase profit margins by making data accessible that wasn't accessible before.

Michele Nemschoff is vice president of corporate marketing at big data platform solutions firm MapR Technologies.

Image: Flickr (thegirlrg)