Is small data the next big thing?

Small data is a buzzword that has been floating around the Internet for a long time now. Many articles suggest that big data is overhyped, that there is too much information and that real value is actually found in small data sets. It seems that much of the confusion comes from a misunderstanding of what big data is, as well as why big data is truly important. As much as this topic seems overdone, it appears there is still some confusion out there, so let's once again break down big data and find out why it matters.

Defining big data

Much of the confusion about the benefits of big data stems from struggling to define big data in the first place. For example, some falsely assume that big data is only about large volumes of data, and thus define any smaller projects, such as mobile analytics, as 'small data'.

The problem with this assumption is that big data is not just about size but also about the type of data being collected and analysed. Unstructured information, such as social media posts and clickstream data, cannot be collected by a traditional database without first being converted into a structured form. Big data technology, such as Hadoop as a service on the other hand, can capture unstructured data. This advantage is just as important as being able to collect large volumes of data in a cost effective manner.

The big data advantage

Once big data is better defined, it's easy to find examples of the many ways it is being used throughout many different industries.

Healthcare

The healthcare industry has many examples of successful big data projects. In the US, a healthcare system analyses clinical records and insurance claims to find patients who are at risk of future ailments. This information helps the healthcare providers to offer better preventative services. Big data is also being used to better monitor the care of patients and prevent mistakes. A study by Pediatrics found that this kind of support system could reduce negative reactions to drugs by 40 per cent.

Fashion

Determining which fashions to stock stores with can be a real gamble. Even if a particular style is very popular on the runway, it can ultimately be a bust when on sale to the general population. Fashion forecasters are now using big data to analyse fashion shows, events, current market offerings and art exhibitions to provide the fashion industry a better idea of which colours and cuts will be most popular during a particular season.

Sports

The National Basketball Association over in the US started using motion detector cameras this season in all of its courts to capture the positions of each player 25 times a second. This data can then be analysed to better determine player performance and identify factors that affect performance. For example, perhaps a certain player runs faster on a certain day or is more successful when paired with a certain teammate. All of this data provides ample marketing opportunities for the association and gives coaches more to work with when planning their game day strategy.

Ultimately, quibbling over whether analysis of multi-structured data should be called big data or small data, based on the size of the data set, takes away from the real reason big data analysis is so important.

Big data can improve the way we interact with customers, improve health care results and even change the way we see sports, and those changes are already occurring. The questions is who will take advantage of this opportunity and who will get stuck defining it?

Gil Allouche is the vice president of marketing at Qubole. Gil began his marketing career as a product strategist at SAP while earning his MBA at Babson College and is a former software engineer.