Driven by digital transformation, all businesses now run their customer experience inside and out using digital products. Businesses are competing in a new way, with faster-moving, more agile competitors, harnessing digital technologies to delight their customers and beat their competitors. And this is not just online retail; it’s telecommunications, healthcare, financial services – almost every industry. Even more traditional businesses have harnessed new business models to effectively become digital products, competing to provide an amazing customer experience. Continuously tuning that digital experience is the new business battleground, and new techniques are being employed to test, monitor and continuously improve the digital experience and thus, the business outcomes. One of these is the application of Artificial Intelligence (AI) and machine learning to learn what’s important, to spot challenges in the business that need addressing and to identify how to provide unrivalled customer experience to meet and exceed business outcomes.
Out with the old, in with the new
So how do we test, monitor and continuously improve customer experience and business outcomes? Effectively, this equates to not just testing software, but more about testing the business itself. Testing has changed and improved drastically from the traditional ways of software testing. By far the most common form of testing is manual testing: using humans to explore a customer experience. However, in this new fast-moving battleground, where cloud-based services are evolving daily and software releases run on a “continuous delivery” model, this is just too slow – failing to provide coverage of all the necessary paths. This method is based on intuition, rather than informed science, which in turn leads to an increased chance of human errors.
Automating tests has often been lacking too; running tests on underlying code to see if it conforms to specification. Where these tests fall short is by not looking at a workflow in the same way a customer does; leaving for the potential of missing a screen going blank at a critical point in a transaction, or a tool-tip obscuring a key button, or a very annoying pause of a minute during a transaction - perhaps causing an impatient millennial to give up out of boredom. As a result, we need a new, efficient and improved way to measure customer experience. Often, automation has run as a series of “regression tests” which run the same tests every time a new release is ready. This isn’t aligned to the new digital era, as it doesn’t focus on the paths through the apps or websites where the most important business impacts are generated, and doesn’t necessarily focus on what has changed since the last release. It usually doesn’t look at the app or website as a user would and doesn’t actively hunt for new bugs that might have crept in, using an informed approach.
Testing lies in the eyes of the user
In order to optimise customer experience, we need a way to understand what the most important user journeys are to test in terms of value to the business. We need to be able to test these journeys quickly before the next release, through the eyes of the user. For example, through the lens of customer experience. We also need to understand what parts of the app or website have been modified since we last tested, to be able to balance a focus on those modified elements. Additionally, we need to ideally be able to predict where bugs are most likely to be. We need to be able to test the software before it goes live, monitor it when live and fix any shortcomings immediately to continually improve the customer and experience, in addition to the business effectiveness and performance. The big question is how best to achieve this?
Enter a totally new paradigm in software quality - AI-driven exploratory testing.
- The customer is always right: The importance of keeping the customer experience at the forefront of digital transformation
Enter AI-driven CX exploratory testing
There are three core drivers to AI testing.
Firstly, AI facilitates test automation through the eye of the customer. This can be enabled using next generation automated intelligent image and text understanding engines, combined with automation engines that can control any system. In other words, just like a human, smart software can understand a stream of screens, identify visual objects and parse text in the moving stream, and measure the customer experience. This might be whether the screen is rendering correctly (functionality), whether it is responsive to user interaction (performance) and whether it looks right with no customer experience glitches on colours, layout or interaction (usability). The typical way such an engine works is capturing a Remote Desktop or a video output feed and then sending in control signals for keyboard, mouse, joystick or other input. The image and text engines use the latest intelligent AI image processes techniques, including deep learning.
Secondly, AI improves customer experience through automated exploratory testing, informed from learning about which parts of the customer experience needs testing the most and which are the most valuable. Early techniques are often based around model-driven testing and an associated AI exploratory testing engine. By creating a digital twin model of the application, website or workflow, it is possible for non-technical users to mark their desired user journeys to test. Subsequently, an AI engine can be engaged. Over time, can learn about what is new in the product (by connecting to Jira, GitHub or the like), what the most valuable user journeys are (by analysing logs and monitoring data for the app or website), where bugs are most likely to be and then prioritising journeys based on an ensemble of algorithms collaborating to identify the most important user journeys to test in the application. In combination with a customer experience test engine, these can be tested through the eyes of the user to improve their overall experience.
Finally, AI can monitor and improve an app or company website using testing techniques to a live running system. By the addition of simple tags to a web server or app, it is possible to track real user interaction with the system - and thus understand what journeys real customers take. It is also possible to track positive business outcomes against those journeys and compare them to desirable levels. For example, an insurance company releases a new version of its app and website, and new policy subscriptions are down by 20 per cent. Since AI was previously applied to the app and website, all the relevant user journeys were recorded, allowing the company to analyse those and diagnose if it’s a functionality, performance or usability issue. A remedy can be identified with a measurable customer experience and ROI improvement. In this way, the business can be continuously improved.
Taking the customer experience to the next level
Companies are essentially now digital products themselves and they now must compete based on customer experience. Improving customer experience has become a central focus for companies around the world. It is vital for companies to work to optimise customer experience and leverage tools available, especially AI, to understand how consumers use their platforms, what tests to run to ensure these platforms are functioning, as well as monitor the platforms to ensure there are no bugs and track changes in use of platforms. Using AI allows companies to provide invaluable insights to help meet their business goals.
John Bates, CEO, Eggplant