In the Spring of 2020, Dimensional Research completed a global study looking at how many organisations now rely on Business Intelligence (BI) tools as part of their day-to-day operations. The findings were overwhelming: of the almost 500 data professionals polled, 98 per cent stated that BI tools play some role in their companies’ everyday business operations.
It’s clear that many companies now recognise the value BI tools can deliver, whether that is by serving up information and insights to help streamline the business decision-making process, by delivering actionable reports to senior executive teams, or by providing the intelligence required to boost operational efficiencies and, of course, profitability. However, the research did not stop there – in addition to tracking overall BI utilisation trends, Dimensional Research also investigated whether these tools were being used to their full potential or whether there was a danger that organisations were, at least to some extent, wasting their investments.
Here, the answers are more mixed.
Business intelligence: well-resourced but underutilised
On the plus side, the report provided strong evidence that companies understand that BI tools alone cannot deliver the insights required to pinpoint and act upon key insights. Indeed, seven out of 10 of the companies surveyed said they had either added new teams to help leverage their BI investments or plan to hire more analysts. Analysts are clearly considered instrumental to unlocking value from BI tools, and this is reflected in the headcounts of most businesses’ data analysis teams.
But when you drill down into how those analysts spend their days, it is a slightly different picture. More than two-thirds (68 per cent) of the analysts told researchers that, while they have ideas for driving more company profits, they lack the time to implement these more strategic initiatives. One factor impacting their productivity is that the typical analyst spends just half of their day on actual ‘analytics.' The greatest drain on their productivity is the time they waste trying to access disparate data sets. This should be a relatively simple process, but can, alone, account for as much as a third of an analyst’s time.
Analysts voiced a number of other, inter-related frustrations. Nearly all (90 per cent) complained that the data sources they need to do their jobs were sometimes unavailable, broken, or only accessible on an intermittent basis. Unreliable data sources often lead to delays that have a wider impact on the overall business. Any lag in the process means analysts are required to generate reports and suggest decisions based on information that is out of date.
Given the rapid change and upheaval that nearly every company has had to cope with during recent months because of Covid-19, it is hard to envisage how months’ old finance, marketing or sales data could be used to steer accurate decisions in the here and now. For especially time-sensitive applications like AI models, such delays are simply out of the question. It’s worrying, therefore, that 86 per cent of the respondents indicated they were sometimes forced to use ‘old’ data. Worse still, of these respondents, more than half admitted that their data was, on average, at least two months old.
Furthermore, it only takes one data source to be unavailable for wider delays to occur, which is an important factor to consider as today’s analysts rely on many disparate data sources. Of those polled, the majority were juggling 11 or more different data sets, with 96 per cent of firms drawing at least some of their information from cloud-based platforms.
The survey also exposed another challenge around data schemas, the blueprints for how these data sources are constructed. With businesses continually changing their operational focus and constantly re-cutting their data in order to uncover new efficiencies and opportunities, these schemas are in perpetual flux, with more and more data sets added all the time. While this improves the accuracy of reporting and the decision-making process, for the analyst, these changes often equal more work and more delays. 60 per cent of respondents report that schemas update monthly.
A lack of available engineering resources to support these updates compounds this problem. Data analysts are forced to take on tasks outside their job descriptions – and core expertise – in order to keep projects moving at the pace required by business leaders. For example, many find themselves creating reports in Excel because they cannot access the information they need via a dedicated dashboard. Others have had to code their own scripts in order to ingest new data streams.
Time to rethink data transit
While organisations are firmly committed to resourcing BI tools and the analyst teams that manage them, when you scratch below the surface, there are worrying signs that both the technology and the people with the necessary expertise to maximise its value are underutilised. Throwing more staff at the problem may help – and is in some organisations’ plans – but it won’t address the root problem that data can be hard to access, cumbersome to manage, and is all too often out of date.
If companies are to drive more value and profit from their BI investments, their first priority must be to allow their existing analysts to spend more time ‘analysing’ and less time ‘finding.’ In practice, this means re-assessing their data transit infrastructures to ensure they are fit for purpose. It is critical that these infrastructures can extract data from multiple sources – including numerous cloud-based applications – in real-time, before replicating, transforming and integrating this information into one coherent data set that can be quickly and efficiently analysed and reported on.
These infrastructures should automate much of the legwork that today’s analysts are forced to do. For example, schemas should instantly adapt to source changes without the need for engineering intervention or creative workarounds.
In short, businesses often have the right people and ambitions to make the best possible use of their corporate data. Many can still improve the efficacy of their investments with careful consideration of the infrastructure and tools that support the efforts of analysts.
Nathaniel Spohn, general manager, EMEA. Fivetran