Big data means big opportunities

The concept of big data has been around since the late 1990s. Since then, most businesses have begun to recognise how it can benefit their competitive position and bottom line. The catch is that recognising those benefits is often easier than achieving them.

The primary challenge is the volume of big data. It requires businesses to fundamentally change not only the technologies they use, but the way their organisations are structured. Conventional database systems struggle to efficiently process the amount of information that big data involves. Therefore, a new breed of systems and data scientists have arisen to handle these large data sets.

How to Maximize Your Big Data Strategies

Big data success stories often involve changes that go beyond the technological. Think about it: “data” and “technology” don’t make decisions, people do. The companies which excel at big data, namely Amazon, Facebook, Google and Netflix, have fostered a company culture of analytics. For them, data-based decisions are the rule, not the exception.

To begin working with big data, it’s important to first establish a culture that embraces data-driven thinking. Encourage teams throughout your organisation to participate in nimble exploration, such as cross-dataset analysis to uncover correlations that would otherwise be missed. Small intricacies in the data can identify massive opportunities for your business.

Next, invest in machine learning platforms, such as scikit-learn and TensorFlow, which enable predictive analysis. These tools comb through seemingly disparate parameters to allow businesses to discover key insights, and the more data you store, the more data you have to train your machine learning algorithms, the more accurate they will become.

For example, trucking companies can identify drivers who are at a higher risk of an accident based on driving behaviour, road and weather conditions. 

Picking the Right Technologies

To effectively gather and use your big data, it’s important to choose the right technology. Three items to consider when evaluating a solution are:    

  • Expandability: Choose products with APIs or similar tools that allow you to integrate your data from a variety of tools and software solutions. This lets you tap into multiple sources of data, providing real-time, cross-functional analysis of your business.
  • Data collection: The type of data collected also helps narrow product choices. For example, some solutions are optimised for unstructured big data sources, such as Twitter, while others are designed to store and analyse structured data, like you would find in a more traditional relational database like SQL Server or MySQL.
  • Interaction with the data: Consider how you will use the data. Are you simply looking for a place to store trillions of records that will be periodically analysed? Or will the analysis be in real time? Will the data be accessed only by internal teams, or by outsiders such as business partners and customers?

These same considerations and questions will also influence your choice of complementary technologies such as data visualisation, machine learning, business intelligence and data analysis tools.

Common Big Data Mistakes and How to Avoid Them

One of the biggest mistakes is underestimating the value of big data. Using its insights, you can gain a better understanding of your customer base, business operations, and opportunities for improvement.

For example, in the case of savvy trucking companies, mining fuel, traffic, distance, and driving behaviour data can identify which drivers are more efficient and in turn understand why they are. They can then use this information to coach other drivers, optimise routes and improve asset utilisation.

Too often big data is stuck in silos, highlighting another common mistake companies make when beginning to integrate Big Data. Using it in conjunction with existing data assets from your CRM, maintenance software and other systems contributes to achieving goals such as increased customer retention and reduced maintenance costs.

Finally, another common mistake is failing to establish an internal big data team. A great big data team will have a blend of skills and will make the data work for the organisation. In 2014, Geotab created a dedicated team of automotive engineers and big data scientists to focus on engine data. This experience shows that the ideal team should:


  • Span a wide variety of departments: When multiple skillsets collaborate on common objectives, they frequently uncover insights that otherwise would have been missed.
  • Include data scientists: People who have technical expertise in working with large datasets, machine learning algorithms and statistical analysis.
  • Have some team members focused exclusively on delivery: Traditional software developers, UI developers and user experience experts have the skills necessary to present big data in ways that everyone else in the organisation can understand and use. When you expand the application of Big Data, you have a better opportunity to improve internal processes, develop new products, and so much more.

Don’t Overlook Benchmarking

You’ve likely established key performance indicators and measurement tactics to guide your business operations and decisions. However, you can really drive results through benchmarking. Benchmarking involves comparing your data against peers, competition, or similar businesses to better understand how you stack up, and identify improvement opportunities. Here are a few things to consider when benchmarking.

If a service provider caters to a wide variety of transportation companies, from trucking to ambulances to taxis, should the benchmarking be done against fleets of the same size? In the same part of the country? In the same vertical? 

The answer to these questions depends on your goal. For example, if a fleet wants to reduce its emissions, it makes sense to compare its data to those of peers that have the similar vehicle types and use cases. Using those insights, they can develop strategies, implement new tactics, and measure progress to achieve their objectives.

Making data-driven decisions is what separates good companies from great companies. Begin gathering all of the data you can, and use our tips to develop your big data strategies and accelerate your management by measurement.

Mike Branch, Vice President, Business Intelligence, Geotab

Image source: Shutterstock/Wright Studio