The big dream of Big Data was that it would empower people and organisations to find out just about anything they needed from the vast pools of data being collected all around them. That with the right data, there would be no great unanswerable questions holding you back, or leaving your future performance to chance.
The problem, as we soon discovered, is that data on its own doesn’t solve anything. Unless you know what to ask and how to frame your question, it won’t help you at all. Even many so-called self-service Business Intelligence and data analysis platforms have failed to help ordinary business users access data in the way they want to get the answers they urgently need.
The problem doesn’t stop with those who lack technical knowhow. As the data streams we use day-to-day become more varied and complex, simply organising, cleaning, structuring and preparing that data for analysis can gobble up a data scientist’s time and resources, before you’ve even begun to query it or generate reports.
To make the process smooth enough and rapid enough to yield valuable answers while they’re still useful enough to put into action, you need to be able to automate any elements you can. But while in the past, automation was limited to simple, manual tasks that took no reasoning capabilities, the rise of Artificial Intelligence has taken this to amazing new heights. Today, intelligent technologies can be applied in astonishing ways that not only speed up tasks that support your data analysis, but actually enrich your entire approach to BI.
Let’s take a look at 5 AI technologies that are changing the face of BI - and how to deploy them in your organisation.
Machine learning is a broad church, encompassing a range of algorithms, APIs and other tools. Combined with data sources and computing power, these make it possible to design and train models that can translate into applications capable of making calculations and suggesting conclusions without human intervention.
Uses are far-reaching, but tend to relate to prediction and classification. The anomaly detection capabilities of machine learning are particularly handy for BI, as they highlight unusual activity or customer behaviours that don’t fit the expected pattern. This helps to catch things like fraud attempts or technical issues before they wreak havoc, as well as highlighting business-critical issues in real time, so that you can adapt and adjust before the problem hits your bottom line.
Note that the majority of Machine Learning capabilities are used to create the linear regression, decision-tree models that are typical of AI, but by linking to data sets with domain-specific algorithms, it is also possible to start building complex social graphs.
This is a specific type of machine learning that brings together artificial neural networks, mimicking the structure of the brain, and multiple abstraction layers. At present, this tends to be used mostly in applications for pattern recognition and classification, when dealing with very large datasets.
This deep learning neural network can also be trained to identify and respond to threats, adjusting and adapting constantly, to detect potential security breaches and malicious behaviour and even intelligently generating defences against cyberattacks.
Natural language processing (NLP)
NLP is a way of understanding text that’s presented in a ‘natural’ way – i.e. as ordinary people speak or write, rather than in computing language. It does this by finding ways to interpret sentence structure, literal meaning, sentiment, tone and intent using statistical analysis and machine learning techniques.
Applications are broad, ranging from tools like chatbots that interact with users and allow them to query data or retrieve dashboards quickly and easily, through to data mining functions and various security and fraud detection features.
When it comes to BI, this has the potential to dramatically overhaul the process. Remember the difference it made when Google found a way to scrap Boolean search terms and introduce today’s semantic search, which invisibly translates a casually asked question into a Boolean structured search the system can understand? Now imagine that non-technical users can use a similar system to query complex data sets. This is likely to play a central role in the future of self-service BI.
Predictive analytics draws on data mining, statistics, predictive modelling, machine learning and other AI techniques to analyse historical and current data in order to make informed predictions about the future. This provides a way of applying insights gained through big data analysis, guiding business-critical decisions and resource planning.
“So where’s the AI?” you may wonder. It’s true that predictive analytics doesn’t necessarily involve artificial intelligence, but bringing in AI techniques can make it far more effective. In particular, by using AI, you can apply regression analysis on a massive scale, helping to identify big, unexpected patterns that a human just wouldn’t spot - not least because this identifies unexpected variables that impact one another, beyond the scope of what we’d normally consider “safe assumptions”
For example, who would have thought that customers buying baby diapers online were more likely to buy beer, too? This is the kind of unexpected discovery that’s made possible by AI-powered predictive analytics. By carrying insights like this through to prescriptive analytics, you can begin to apply your findings proactively to improve results, too.
Multiple neural networks
If deep learning mimics the behaviour of the brain, then multiple neural networks harness the collective power of many brains at once, almost as if they were in conversation.
Specifically, the term “multiple neural networks” refers to a number of independent network architectures, all of which operate separately and are designed for a distinct task.
Based on the information or discoveries of all of these independent networks, another network called the “expert network” or agent then makes a final decision, based on the decision system that’s in place – be that a rule-based expert system, a logical majority vote function, or by deferring to another neural network.
Typically, you would use a multiple neural network when dealing with different sensors or information sources that provide data on the same object, or when you want to process the same input data using different filters etc, with each network handling a different pre-processed input.
Think of it a bit like the pre-cogs in the movie Minority Report: all three have their premonition dreams fed into the system separately - and usually the result is the same. Occasionally, though, one will differ, in which case a higher system compares data from the three brains and dismisses the “Minority Report” as an outlier.
AI, BI and data analytics: Bringing it all together
When it comes to Business Intelligence, the exciting thing about all these developments in AI is that they make it possible at last to draw out those insights from Big Data that we always knew were there, but - frustratingly - couldn’t quite tap into. Even better, they allow us to discover those insights at incredible speed.
From detecting cyber breaches and fraud attempts to monitoring social media channels to establishing patterns in buyer behaviour or predicting which customers are most likely to switch to another brand, the scope to streamline processes and boost performance is enormous. That said, it’s crucial to remember that, while these tools make it easier than ever to engage with data, conduct regressive analyses and spot links and patterns across enormous data sets, they can’t do any of this without guidance.
As ever, you need to know what you are trying to find out. You need to be sure what questions you need to ask. You need to know how you are going to use and apply your findings. And crucially, you need to remember that you are at the helm - that while AI can suggest answers and responses to an impressive degree, the final decision, the final analysis, will always rest in your hands.
Ilan Hertz, Head of digital marketing at Sisense
Image Credit: PHOTOCREO Michal Bednarek / Shutterstock