Skip to main content

Decade in data

(Image credit: Image Credit: Flickr / janneke staaks)

Data has become big business in the past 10 years. To put this into context, seven out of the top 10 global technology companies are built around the sharing of data. So how did we get here, and what were the key moments that shaped the data revolution? Let’s have a look back at the biggest data advancements, achievements, and milestones from the past 10 years that have changed how we work and live.

AI enters the enterprise

Films such as Blade Runner, the Terminator, the Matrix and I, Robot captured imaginations, giving us a glimpse of what the future of AI could look like. While these are Hollywood depictions, which aim to entertain and take plenty of creative license, AI in real-life has also taken its place at the forefront of innovation over the past decade.

We have witnessed this technology move from science-fiction to reality thanks to the cloud and the subsequent huge volume of data that scientists and engineers have been able to use to train algorithms. These algorithms now process massive volumes of data, automate repetitive tasks, make predictive recommendations, enable better decision making, and so much more.

While AI in our daily lives might not fully resemble what’s presented in science-fiction, we’re seeing incredible examples coming into fruition – from autonomous vehicles, to personalised shopping and entertainment recommendations, to spatially aware robotics which process human speech in real time. Chances are you’ve been carrying AI in your pocket for a fair chunk of the past decade.

We have seen rapid adoption of AI by enterprises, too. AI-focused tech startups abound but it’s also become the norm that progressive companies, of all sorts and across all industries, are building AI into their core products, supporting everything from accounting and HR to customer services and sales, in turn transforming business operations and saving companies time and resources.

Big data for big decisions

Data and insights is a combination that has become, in some form or another, a daily imperative in most business decision-making processes. In the last decade the drive to make smart, data-driven choices around developing products, targeting customers, setting KPIs, and ramping growth strategies has become the expectation for any successful company. As a result, we’ve seen companies collecting more and more data, as they begin to understand the impact data-driven decisions can have on the bottom line, providing not only a factual view of the past, but also, and more importantly, a more accurate projection for the future as well.

This drive to discover more through data has given way to new predictive analytics tools, new ways of working, and new job roles within organisations, as businesses seek an edge over their competitors.

However, this explosion of data, and the race to take advantage of it has also resulted in organisations leaving important data sets in silos, leading to decisions based on incomplete subsets of data rather than the whole picture. Looking ahead – as the data talent, approaches, and technologies that organisations employ only get smarter – we’ll start to see better integration of data from across the entire business, delivering valuable insights to the executive team, line of business heads, as well as every employee across the organisation.

Cloud

The last decade also saw cloud computing platforms make a big impact on the way many organisations now approach their IT. Many of us use cloud-based applications every day as part of our work, most of which have been adopted by the entire organisation. This is a marked difference to the start of the decade, when an organisation taking a cloud-first approach was barely whispered as a possibility whilst cloud services battled with a reputation of security vulnerability.

Over time, for the businesses who weren’t quite ready to make the switch to a completely cloud-based infrastructure, we saw several public and hybrid cloud solutions coming into play. This created flexibility that enabled greater interaction between SaaS and on-premises applications, ensuring cloud migration when and where possible while also leveraging the valuable data assets of the past that may continue to reside for some time within the physical confines of the enterprise.

Without the growth of cloud computing over the last ten years we likely wouldn’t have many of the technologies which will impact the next decade too. Take IoT for example: without the cloud the dream of smart cities wouldn’t be possible, as monitoring and collecting data in real time simply couldn’t be done as it is today. 

New regulations

Underpinning all of this, and arguably one of the biggest changes in the last ten years, has been the introduction of regulations around data management and use. Mining data for valuable information has become so commonplace in the 21st century that the lack of overarching law became painfully apparent at the start of the decade. As data breach after data breach hit the headlines, trust in companies holding personal data took a nosedive. 

After years of deliberation, the General Data Protection Regulation (GDPR) was enforced in the EU in 2018, followed by the California Consumer Privacy Act (CCPA) in the US. Though they weren’t introduced until the latter part of the decade, they have become some of the most comprehensive pieces of legislation to address how data is managed, shared, and used. GDPR has even been hailed as the most consequential regulatory development in information policy in a generation.

Since the introduction of GDPR and CCPA we’ve seen sweeping changes to how businesses procure, manage, store, and secure data. In the last year alone we’ve seen record-breaking fine proposals made for Marriott, at £99 million, and British Airways, for £183 million – numbers that certainly provide concrete incentive to maintain proper governance and best practices for those looking to make data a key part of their business.

Looking ahead

The last decade witnessed some significant steps in the development of the technology which underpins our daily lives. But without the astronomical volumes of data we now generate many of the technologies we rely on wouldn’t have flourished. As we look ahead to the next decade we need to make sure we take lessons from the data failings in history, from the fines, the company closures and missteps to ensure that data can still contribute to the next evolution of our collective futures.

Craig Stewart, Chief Technology Officer, SnapLogic