Skip to main content

How to make use of machine learning in finance?

(Image credit: Image Credit: Sergey Nivens / Shutterstock)

In the previous article, I’ve described why financial services companies should care about machine learning and what use cases they should pay attention to. Now, it’s time to see how exactly financial businesses can adopt this technology to make it a success.

The problem is, in spite of all the advantages of AI and machine learning, even companies with deep pockets often have a hard time extracting the real value from this technology. Financial services incumbents want to exploit the unique opportunities of machine learning but, realistically, they have a vague idea of how data science works, and how to use it.

Time and again, they encounter similar challenges like the lack of business KPIs. This, in turn, results in unrealistic estimates and drains budgets. It is not enough to have a suitable software infrastructure in place (although that would be a good start). It takes a clear vision, solid technical talent, and determination to deliver a valuable machine learning development project.

As soon as you have a good understanding of how this technology will help to achieve business objectives, proceed with idea validation. This is a task for data scientists. They investigate the idea and help you formulate viable KPIs and make realistic estimates.

Note that you need to have all the data collected at this point. Otherwise, you would need a data engineer to collect and clean up this data.

Depending on a particular use case and business conditions, financial companies can follow different paths to adopt machine learning. Let’s check them out.

Forgo machine learning and focus on big data engineering instead

Often, financial companies start their machine learning projects only to realise they just need proper data engineering. Max Nechepurenkoa senior data scientist at N-iX, comments:

When developing a [data science] solution, I’d advise using the Occam’s razor principle, which means not overcomplicating. Most companies that aim for machine learning in fact need to focus on solid data engineering, applying statistics to the aggregated data, and visualisation of that data.

Merely applying statistical models to processed and well-structured data would be enough for a bank to isolate various bottlenecks and inefficiencies in its operations.

What are the examples of such bottlenecks? That could be queues at a specific branch, repetitive tasks that can be eliminated, inefficient HR activities, flaws of the mobile banking app, and so on.

What’s more, the biggest part of any data science project comes down to building an orchestrated ecosystem of platforms that collect siloed data from hundreds of sources like CRMs, reporting software, spreadsheets, and more.

Before applying any algorithms, you need to have the data appropriately structured and cleaned up. Only then, you can further turn that data into insights. In fact, ETL (extracting, transforming, and loading) and further cleaning of the data account for around 80 per cent of the machine learning project’s time.

Use third-party machine-learning solutions

Even if your company decides to utilise machine learning in its upcoming project, you do not necessarily need to develop new algorithms and models.

Most machine learning projects deal with issues that have already been addressed. Tech giants like Google, Microsoft, Amazon, and IBM sell machine learning software as a service.

These out-of-the-box solutions are already trained to solve various business tasks. If your project covers the same use cases, do you believe your team can outperform algorithms from these tech titans with colossal R&D centres?

One good example is Google’s multiple plug-and-play recommendation solutions. That software applies to various domains, and it is only logical to check if they fit to your business case.

A machine learning engineer can implement the system focusing on your specific data and business domain. The specialist needs to extract the data from different sources, transform it to fit for this particular system, receive the results, and visualise the findings.

The trade-offs are lack of control over the third-party system and limited solution flexibility. Besides, machine learning algorithms don’t fit into every use case. Ihar Rubanaua senior data scientist at N-iX comments:

A universal machine learning algorithm does not exist, yet. Data scientists need to adjust and fine-tune algorithms before applying them to different business cases across different domains.

So if an existing solution from Google solves a specific task in your particular domain, you should probably use it. If not, aim for custom development and integration

Innovation and integration

Developing a machine learning solution from scratch is one of the riskiest, most costly and time-consuming options. Still, this may be the only way to apply ML technology to some business cases.

Machine learning research and development targets a unique need in a particular niche, and it calls for an in-depth investigation. If there are no ready-to-use solutions that were developed to solve those specific problems, third-party machine learning software is likely to produce inaccurate results.

Still, you will probably need to rely heavily on the open source machine learning libraries from Google and the likes. Current machine learning projects are mostly about applying existing state-of-the-art libraries to a particular domain and use case.

At N-iX, we have identified seven common traits of a successful enterprise R&D project in machine learning. Here they are:

  • A clear objective. Before collecting the data, you need at least some general understanding of the results you want to achieve with AI and machine learning. At the early stages of the project, data scientists will help you turn that idea into actual KPIs.
  • Robust architecture design of the machine learning solution. You need an experienced software architect to execute this task.
  • Appropriate big data engineering ecosystem (based on Apache Hadoop or Spark) is a must-have. It allows to collect, integrate, store, and process huge amounts of data from numerous siloed data sources of the financial services companies. Big data architect and big data engineers are responsible for constructing the ecosystem.
  • Running ETL procedures (extract, transform, and load) on the newly created ecosystem. A big data architect or a machine learning engineer perform this task.
  • The final data preparation. Besides data transformation and technical clean-up, data scientists may need to refine the data further to make it suitable for a specific business case.
  • Applying appropriate algorithms, creating models based on these algorithms, fine-tuning models, and retraining models with new data. Data scientists and machine learning engineers perform these tasks.
  • Lucid visualisation of the insights. Business intelligence specialists are responsible for that. Besides, you may need frontend developers to create dashboards with easy-to-use UI.

Small projects may require significantly less effort and a much smaller team. For instance, some R&D projects deal with small datasets, so they probably don’t need sophisticated big data engineering. In other instances, there is no need in complex dashboards or any data visualisation at all.

Key takeaways

  • Financial incumbents most frequently use machine learning for process automation and security.
  • Before collecting the data, you need to have a clear view of the results you expect from data science. There is a need to set viable KPIs and make realistic estimates before the project’s start.
  • Many financial services companies need data engineering, statistics, and data visualisation over data science and machine learning.
  • The bigger and cleaner a training dataset is, the more accurate results a machine learning solution produces.
  • You can retrain your models as frequently as you need without stopping machine learning algorithms.
  • There is no universal machine learning solution to apply to different business cases.
  • Tech giants like Google create machine learning solutions. If your project concerns such use cases, you cannot expect to outperform algorithms from Google, Amazon, or IBM.

Tetiana Boichenko, marketing manager, N-iX
Image Credit: Sergey Nivens / Shutterstock

Tetiana Boichenko, marketing manager at N-iX LLC. She’s been working in technology companies for three years writing about emerging technologies like blockchain and their implication for business.