The top 5 Hadoop misconceptions that catch out most IT managers

Even though Hadoop has been part of the business landscape for the past few years, there is still a lot of confusion about big data and how businesses can benefit from Hadoop. It's time to put some of these misconceptions to rest, starting with the top five:

Misconception# 1: Hadoop is about data volume

Hadoop is credited with the ability to harness big data, but there's a misconception that the sole purpose of Hadoop is simply to collect massive amounts of data.

While it's true that Hadoop can handle large data volumes, one of the key benefits of Hadoop is its ability to collect many different types of data, and analyse the data quickly to come up with fresh insights — all without having to structure the data first.

Essentially, Hadoop gives businesses faster access to a broad variety of data, from social media conversations to sensory device data, allowing businesses to gain valuable insights even if the amount of data the business has is relatively small.

Misconception #2: Hadoop requires a data expert

Many business leaders assume they will need to hire a data scientist in order to use Hadoop. However, several Hadoop vendors now offer turnkey solutions complete with data integration, visualization and analytical tools as well technical support services, meaning that companies can get up and running quickly with a Hadoop cluster without needing to hire a data scientist.

However, when managing and analysing highly complex Hadoop clusters, your programmers should have some knowledge of NoSQL and Hadoop, and there are several vendors that offer Hadoop training and support for this purpose.

Misconception #3: Hadoop is limited to batch processing

Although the open-source Apache Hadoop distribution is a batch-oriented framework, and thus can only manage data slowly through a batch process, there are several Hadoop vendors that offer enterprise-ready distributions that take Hadoop beyond batch processing.

MapR, for example, radically simplifies Apache Hadoop by making it accessible through MapR Direct Access NFS, which enables real-time read/write data flows.

Misconception #4: Hadoop is a single entity

Hadoop is not a single product. Instead, it's a collection of several components overseen by the Apache Software Foundation. Apache Hadoop includes the Hadoop Distributed File System (HDFS), MapReduce, HBase, Pig, Hive, Zookeeper and several other packages. Businesses can pick and choose between these Hadoop packages and combine them depending on their Hadoop project requirements.

As mentioned above, there are also a growing number of vendors that offer an enterprise-ready Hadoop distribution of Hadoop, and these commercial distributions contain numerous Hadoop ecosystem packages. Some of these components can be enhanced or replaced with other technologies. MapR, for example, includes NFS in addition to HDFS and includes a full random-access, read/write file system.

Misconception #5: Hadoop is only for Internet companies

If you only pay attention to the world's largest Hadoop users, you would think that Hadoop is only useful for Web 2.0 or Internet-based companies. The truth is that big data comes from many sources, such as a utility company's grid monitors or a retailer's CRM database, and is useful in many different industries.

In the financial industry, for example, sophisticated financial fraud is difficult to detect without being able to analyse the huge, multi-structured data sets that Hadoop can handle. Hadoop is also being used in the information services, transportation, utilities, financial services, healthcare, government, entertainment, and manufacturing fields, among others.

Although Hadoop has been around since 2005, there has been a lot of confusion about Hadoop and related technologies. Now that you understand what Hadoop really is and what it can do, you can understand how Hadoop can help transform big data into actionable insights for your business.

Michele Nemschoff is vice president of corporate marketing at big data platform solutions firm MapR Technologies.

Topics