UK Hadoopists: fashionably late to the big data party

People are such an important aspect of data analytics, and I was reminded of this at the recent Strata+Hadoop World event, where I saw first hand that the UK is indeed facing the same skills gaps as elsewhere in the world. Perhaps that didn’t surprise me, but I also noticed the different journey the UK has taken to adopting big data analytics.

Without delving too deeply into comparing and contrasting the UK with other markets using big data analytics, we can see at least one stark difference: that the late adoption of big data has been a help, not a hindrance. While US companies might have vast amounts of data to work through, in general, UK companies are not facing this same scale of project quite as often. Instead, the UK is mainly facing the complexities of incorporating and consolidating a wide variety of distributed data – from a variety of different sources and located in different parts of the business.

Thanks to its late arrival at the party, data analysts in UK businesses can look at the experiences of US companies and learn from how they’ve overcome those challenges of volume and complexity.

UK businesses are skipping the experimentation stage and jumping straight to an understanding that it’s not big data and analytics, but that the two come hand in hand. In doing so, the skills held by data scientists in the UK are less likely to have that gap in understanding. In the US, there remains some long-held confusion that the two are separate projects despite having more hands-on trial-and-error experience.

This won’t be everything that UK companies need to accomplish to be successful of course. Next, I recommend focusing on the ABVs for the Aspiring Hadoopist.

- Agility: Hadoop evolves at an incredible rate, so to keep up you need to be flexible

- Balance: It’s important to maintain a balance ­ whether it’s between older/newer tech, or maintaining the status quo and being innovative

- Vision: The art and power of ‘imagination’, along with the ability to negotiate multiple fields of vision, will arm you with the tools to be a successful Hadoopist

These principles, developed as a result of the experiences of Hadoopists, data scientists and data analytics projects, provide a framework for successful Hadoop projects and I advise everyone, wherever they are from, to bear them in mind.

You’ll notice these ABVs are not about the data, the method, or the data source, but the people. Today, two-thirds of the issues listed as barriers to success are people-related, not technology related. Though UK data analytics projects are on a different journey, people and skills remain at the centre of how that journey develops. So even though the UK might be lagging behind the US in big data implementations, the understanding that analytics is inextricably linked to Hadoop will stand the UK in good stead for the future.

If you missed my session at Strata+Hadoop World, check out a video of it here. For a more in-depth view of making Hadoop work for your business (whether you’re in the UK or not), take a look at our report Bringing the Power of SAS to Hadoop.

Check out some more blogs on the importance of getting the right people in your big data team, whether the skills gap will kill Hadoop, and the need to close the big data skills gap.

Tamara Dull is the Director of Emerging Technology, SAS Best Practices