The big data advantage, part one: ‘It’s down there somewhere, let me take another look’

As with many people working in the technology sector, the Cray team likes to debate and muse on industry trends when we meet over lunch or at the pub. I enjoy our banter – it’s part of what makes it so much fun to work at Cray.

In the spirit of these casual discussions, I wanted to share some of what I’ve found most interesting in a few articles about what I like to call “big data advantage.”

By now, nearly everybody is used to seeing articles detailing all of the potential benefits of big data, whether it’s making new breakthroughs, gaining unforeseen opportunities or the disruptive impact it’s making across different industries. Despite the fact that we’ve seen serious advancements in recent years, it doesn’t change the fact that most businesses are still struggling to take full advantage of big data; wrestling with not only huge datasets but often the speed and diversity of data itself – not to mention bottlenecks in data movement.

At the core, big data and digitalisation are two sides of the same coin: modernisation. This means that we’ll likely see big data have an increasing impact on organisations across industries. With this in mind, both IT teams and business leaders must find new ways to transform the potentially valuable volumes of data they are sitting on into truly valuable information they can use to improve operations, customer experience, and innovation.

But yet when I think of big data, its advantages still seem hard to quantify. Personally, I’m often reminded of a line from the film The Big Lebowski, where The Dude says, “it’s down there somewhere – let me take another look.”   So let’s do that now. Cult comedy films aside, the first thing to consider is what companies expect the impact of big data to be. From my own experience, the general expectation seems to be a combination of improved business opportunities and operational efficiencies.

For a bit of a deeper look, here are a few statistics around the potential influence big data is poised to bring to business in the near future.

  • By 2020, big data will be used in 450 billion internet transactions per day (IDC)
  • The sale and use of Internet of Things (IoT) technology will account for a $1.9 trillion economic value-add (Gartner, Dec. 2014). Industries leading adopting are manufacturing, healthcare and insurance.
  • 50 per cent of enterprises will compete using advanced analytics by 2018, which will lead to the disruption of entire industries (Gartner 100 Data and Analytics Predictions, March 2016)
  • Also by 2018, decision optimisation will no longer be a niche discipline; it will become a best practice in leading organizations to address a wide range of complex business decisions. (Gartner, March 2016).

The next thing to consider is what organisations are really looking to gain from utilising big data. After all, what’s the sense in collecting such enormous volumes of data if they are not being put to best use?

Considering the above, organisations may initially feel the urgency to just keep up in such a dynamic environment, but then they quickly look to use advantages from within their information in a few key areas, including customer experience, stakeholder engagement, business models, efficiencies and new opportunities.

So, what kind of payoff is there for companies that seize their big data advantage? According to a 2015 report by PwC, the companies that join the “information elite” will likely see:

  • A significant improvement in productivity amongst various departments.
  • The ability to offer customers a better experience throughout all interactions, leading to significantly improved retention and increased sales.
  • The capability to conduct much more effective and faster business decision making, based on reliable and hard evidence.
  • A greater ability to identify new and highly profitable product development opportunities based on customer insight.
  • Better identification and targeting of potential new customers, clients and partners.

Gaining access to a big data advantage requires companies to leverage their information effectively in order to make decisions with near immediacy, continuously pivot strategy and tactics, and merge streams of inquiries into meaningful action.

Collectively, I refer to this concept as high-frequency insights. Personally, I believe it represents a serious competitive advantage for businesses to keep pace and thrive in today’s often frenetic world. However, we are seeing the sheer size and complexity of big data creating a tipping point, overwhelming conventional enterprise systems. For organisations wanting to unleash the full promise of big data, a new approach is required — one that delivers agility and performance while maintaining an enterprise-accessible format.

As big data is used to make more informed business decisions in urgent situations, agility is absolutely crucial. This means that data must be processed within a timeframe that the insight is still applicable. This concept of perishable insights means that over time the value of knowledge often diminishes and has a shelf-life. In many cases, this will mean enabling real-time data analysis – in order to make the most effective use of big data and gain a true big data advantage.

There’s no doubt that the “convergence of big data and supercomputing” is the trend of the future. In my view, the supercomputer of 2025 will be indistinguishable from a big data analytics platform. This coherence is going to enable a number of important benefits across many different sectors for example: Reducing risk in mass transportation and in financial markets; optimising in real-time the flow of commodities around the world; and interpreting the effects of weather patterns or diseases on the flow of goods and services across borders. Those are just a few of the very complex problems that require simulation, modelling, and analytics – together.

In the future, a more efficient prosperous world economics requires businesses have reflective, real-time and predictive insights. This collective ability will be accelerated by the convergence of big data and supercomputing.

In my next article, I’ll take a look at and offer more information on the hurdles companies commonly face when trying to capture their own big data advantage.

Amy Hodler, Senior Analytics Product Marketing Manager, Cray

Image source: Shutterstock/Wright Studio