Skip to main content

Harnessing unstructured data during and beyond the pandemic era

Flatfile data
(Image credit: Flatfile)

While we have been spending more time indoors during the pandemic, the extra data that we’ve produced during that time – the Tweets, the Instagram pictures, the product reviews – have taken on an active and highly exploitable life of their own, winging their way into the pockets of companies who can now take advantage of this valuable marketing intelligence – provided they know how to use it. A rich online life correlates with the production of more and more data, providing companies with direct customer information straight from the horse’s mouth – on what customers want, their likes and dislikes, where existing services or products aren’t meeting their needs, and other immeasurably useful insights. According to broadband network Openreach, customers consumed 50,000 petabytes of data in 2020, which is over double the 22,000 consumed in 2019. This provides an impression of the sheer quantity of internet usage during the pandemic, and – by extension – tells us that the voice of the customer has never been clearer, if businesses are prepared to listen.

The challenge of all this new data is that it is unstructured. Unstructured data (UD) is data which is not numbers in tables: it does not necessarily fit neatly into spreadsheet, and it can take a huge variety of forms including text, images, social media posts, and even audio or video files. There has always been a huge amount of UD, in the form of books, magazines, movies, shows etc. but the rise in volume of product related UD in the wake of lockdown life should act as a wake-up call to businesses who have yet to realise UD’s enormous potential for gaining a competitive edge.

The time is certainly ripe to seize the opportunity according to a 2019 survey by Deloitte, only 18% of organizations were capable of taking advantage of UD – and this was at a time just prior to UD’s pandemic-related growth. Chris Colley, Principal of Customer Experience at Medallia, has rightly noted that the shift towards online interactions over the last year has been “creating a ton of new, unstructured data.” With the right tools, more businesses can exploit this new UD to gain valuable insights – on customers, on market trends, and on the industry competitors who haven’t yet unearthed the rich seam of information threading just beneath their feet.

Insights, business intelligence, and customer analysis 

Of course, this has not always been the case. UD has always existed – according to the leading industry analyst firm IDC, 90% of enterprise data is unstructured, and Datamation notes that it grows at a rate of 65% annually – but that does not mean we have always had the ability to extract meaning from that data and turn it to good use. At best a customer care manager will read some consumer emails and chat threads or maybe even listen to some call recordings between a consumer and a company representative but that is what we call: anecdotal information. 

Now, however, with the help of artificial intelligence (AI), and other sophisticated advances, it is possible to transform the mountain of once-unusable data which all companies sit on into concrete, useful intelligence. 

Consider, for example, the insights UD can provide in the world of customer behaviour. AI is now capable of trawling through reams of social media posts and other direct consumer interactions with the company in order to find references to specific products and services and determine, through sentiment and topic analysis, how customers think and feel towards those products and services. Companies can then action these insights in a very personalised way through an enhanced understanding of their customers.

But, again, UD is not limited to text. Valuable business intelligence can be gleaned from audio data – perhaps from customer service call recordings, for example. By analysing audio files, AI is capable of spotting patterns in this vast and unstructured quantity of data and providing insights into how customer experience might be improved. If a customer thinks or feels anything of use to a business, and they express those thoughts and feelings through an array of data sources, then UD analysis can find and capitalise on that data.

The future of UD analysis

With so much knowledge available through what University of Bristol researchers call the “exponentially mounting and extraordinary volume of data” accrued during lockdown, the post-COVID landscape is likely to be one which places a greater emphasis on the importance of using UD. There is too much to ignore, and the wealth of insight it can offer businesses will grow alongside its volume. As a result, it will become increasingly untenable to ignore the 99% of unstructured data which, according to Harvard Business Review, currently goes unused. Knowledge is power, and – to businesses – that power will be reflected in the advantages of synthesized/actionable/unique insights into customers and competitor companies alike.

Ultimately, the value of UD in business fused together from multiple internal and external sources lies in the boost it provides to taking immediate action. During the very pandemic which produced so much extra UD, business leaders have been forced to make unprecedented decisions about the future with little evidence to go on. By embracing the insights offered by UD analysis tools and services, business leaders can enjoy a welcome break from uncertainty in the customer journey optimisation. Instead, they will have the security of in-depth, refined analysis which informs decisions that lead to actions by drawing from breath-taking volumes of information which is theirs for the taking.

Michalis A. Michael, CEO, DMR

Michalis A. Michael, CEO at DigitalMR Ltd, a tech company specialized in using Artificial Intelligence to create intelligent datasets.