Data Discovery: Are you driving without a windshield?

You don’t drive your car staring at the dashboard. Sure, it provides excellent information; You can see how fast you are going, whether or not you have enough gas, and even find out if the engine is running properly. But, if you don’t look out the windshield, you’re guaranteed to crash into something very soon.

Data discovery is very much the same. Internal data sources shed light on key internal issues, but if that is all you are taking into consideration, you’re missing out on some of the most important information.

An insider’s look with no external validation

When business intelligence (BI) tools first hit the market they focused on data from internal systems, largely data from operational databases and other enterprise software systems.

When properly implemented, these systems gave companies a really valuable insight into operational issues ranging from the performance of the call-center to the results of the latest efforts to fight customer churn – in short, pretty much anything that falls under the “COO’s world”.

This is a great way to understand what’s happening inside the organisation. But what about the external business environment? Things like market conditions, competitive landscape and economies that the company is operating in can provide critical information that is often left out of view. The truth is that companies can operate perfectly based on internal metrics, but still come crashing down in flames if they don’t navigate the market environment effectively.

In other words, decision-makers in the enterprise spend a lot of their time looking at the dashboard — sometimes literally — but only have a blurry and fragmented view of their surroundings: the markets they operate in, the economies they belong to and the demographics they target.

[full_width_ad]

Good in theory, but difficult in reality

The truth is that there’s a lot of good data out there, from public and proprietary sources alike. Government databases are opening up and contain more valuable information than most people realise.

Syndicated research — trackers, forecasts and surveys — is plentiful but hard to find and quickly gain insight from. And data from custom research, whether internal or from research vendors, is usually delivered in static formats. As a result, too much of it ends up sitting on hard drives somewhere with no good way to search, compare or access later – let alone to keep an eye on updates to the underlying data.

This is fundamentally inefficient as it can be near impossible for the average user to track down this information and integrate it into their data analysis. As a result, decisions aren’t made with reference to the best available data.

Instead, time is lost digging through piles of static documents and companies are unable to make the most of the sizable investments they’ve already made in market intelligence.

Directions for success

I’m often asked how this actually works in real life. Offerings that allow users to consume data as a service provide insightful data direct to users in a format that can be easily integrated and analysed. And the truth is some of the more interesting use cases come from basic, even rudimentary, information like weather.

For example, users often look at metrics on sales and wonder why particular spikes or drops have occurred over time. Layering on weather data can show all sorts of information. Perhaps there was a snow storm that stopped people from coming to the store or differences in temperature and precipitation that impact consumption trends.

One of my favorite podcasts “Under the Influence” actually did a show all around the power of weather on marketing. One of the most interesting tidbits was around supermarkets in the UK, which found they sell more ice cream on a sunny, cool day than a cloudy, warm day. When the temperature reaches 77F, ice cream sales plummet.

The reason was simple: shoppers worried the ice cream would melt before they could get it home. This type of insight can significantly impact the way an organisation handles their inventory and even their marketing approach and it all comes from looking at external data sources.

In the future I truly believe that the leaders in the industry will be the ones who find ways to leverage external data in order to find better insights within their own information.

Not only will they gain a better understanding of their organisation, they will be able to better predict and anticipate outcomes based on external factors.

Josh Good is Director of Product Marketing at Qlik.