Big data: What's the big deal anyway?

Not so long ago collecting data required considerable effort. You had to measure and count things, then turn that raw information into useful data by inputting it into some sort of model. It was time consuming and took time to produce results.

Now there are millions of everyday devices collecting data, and lower storage costs mean that more of it can be retained and used to spot historical trends. This shift in how data is gathered, stored and analysed is starting to have a significant effect.

BSA | The Software Alliance has released a paper highlighting how data is changing the way we live, in many cases without us even realising it. Data is being used to solve complex problems around the world.

Examples highlighted by the report include farmers using data from seeds, satellites, sensors, and tractors to make better decisions about what to grow, when to plant, how to track food freshness from farm to table, and how to adapt to changing climates.

In the United Arab Emirates, new data tools are being used to design the world's first positive-energy building that actually produces more energy than it consumes. The city of Stockholm in Sweden has installed 1,600 GPS systems in taxi cabs to collect data on traffic flows, and then used software to analyse the data to inform the city's plans to reduce congestion. As a result traffic has been reduced by 20 per cent, travel times have been cut in half, and emissions are down 10 per cent.

In Kenya, mobile data is being used to identify malaria patterns and identify hotspots to guide government eradication efforts. Also on the medical front researchers have developed a machine learning algorithm that can predict cardiac arrest four hours in advance, and is accurate 66 per cent of the time, by combing real-time data with a patient's medical history.

There are economic implications too. Economists estimate that if better use of data made industry only one per cent more efficient it could add $15 trillion to global GDP by 2030.

The full report, What's the Big Deal With Data? is available to download from the BSA website. There’s also a video overview of the findings below.