Skip to main content

Taking a new approach to reducing software testing costs

(Image credit: Image source: Shutterstock/TechnoVectors)

Companies today are under increasing pressure to automate more and spend less. In some cases, investing in new technology can actually be counterintuitive when it comes to reduce testing costs. But, for the majority of enterprises that still do 80 per cent of their testing manually, this type of investment can yield a rapid return.

Scaling test automation can significantly reduce an organisation’s testing costs. And AI-powered impact testing can reduce those costs even further - pinpointing the specific areas of an application affected by a given release, and identifying the test cases needed to test those areas.

Together, these technologies enable organisations to take a focused, tailored, intelligent approach to testing that not only achieves broader coverage, but also significantly accelerates test cycles.

High cost, low value

In traditional development models, testing tends to happen at the end of the development cycle, where bugs are expensive to fix, and will often cause release delays. Today, testing can still stand out as a bottleneck - even for those organisations who have modernised their development and operations processes. Indeed, according to a GitLab survey, testing represents the most significant delay in the software development pipeline; even for DevOps teams.

Additional research has shed further light on this finding, revealing that most testing still occurs manually - the majority of organisations only automate 20 per cent or less of their total testing effort. This huge manual effort comes at a cost. According to the 2019-20 World Quality Report, testing still represents nearly 30 per cent of IT budgets.

Testing is often the last element of the software delivery pipeline to be modernised. But, in de-prioritising it, organisations are missing a significant opportunity to save significant amounts of money and add significant value to their offerings by improving quality and the ability to adapt customers’ needs.

There are ways, however, for an organisation to rapidly increase automation rates, focus its testing efforts, and dramatically reduce testing costs. Here, then, are examples of best practices that can be implemented to transition testing from a necessary cost to a value driver.

Automate at scale

Many organisations lack the coding skills need to scale test automation. With demand for these skills far outstripping supply, resources will instead be dedicated to more visible or mission-critical areas. As a result, not every part of a business will be covered, and overall test automation rates will remain low.

A model-based approach to test automation exists, though, which doesn’t require coding skills, enabling enterprises to rapidly scale test automation with the resources they already have. This approach breaks the automation process down into reusable building blocks, or models, which can be accessed and reused across different projects and teams. A scriptless tool means that, if most or all of an organisation’s testing is still performed manually, it can rapidly scale its test automation using only the resources it has available.

The benefits speak for themselves. After moving away from manual testing and significantly increasing its test coverage, global payment processor Worldpay calculated that achieving the same level of coverage manually would have cost around $500,000 a month.

And by automating all of its manual testing cases, Allianz, one of the world’s largest insurance and asset management providers, increased its testing efficiency by 80 per cent.

Implement a risk-based testing approach

A risk-based approach to testing will catalyse the benefits of automation. Enabling testing teams to identify and create the tests most important for minimising business risk, such an approach will significantly reduce the total number of test cases that need to be created and run before a release can be deemed production-ready. By aligning its testing activities with its business priorities, a focused, risk-based approach will enable an organisation to reach optimal coverage more quickly, and assess its release readiness with greater confidence.

Medical device manufacturer Varian, for example, reduced its testing time by 93 per cent and its cost by 35 per cent after transitioning from manual to automated testing for its enterprise applications. Its complex enterprise landscape comprises SAP, SQL Server, Dell Boomi, Salesforce, and ServiceMax among others, each with independent release cycles. Perhaps unsurprisingly, manual testing wasn’t able to support the complex custom work processes being deployed to production every two weeks. But, by fully automating its testing - without the need for technical expertise - and identifying those that would cover top business risks, it was able to significantly reduce the total number of test cases needed.

Dolby Laboratories also recently automated business process testing across a large SAP footprint and a diverse suite of technologies. During a 90-day pilot, it reduced regression test cycles by 75 per cent, and production defects dropped close to zero. A risk-based approach clarified which tests were most critical, saving a significant number of hours for its business system analysts and other functional leads. The organisation has since automated almost two-thirds of testing across the applications identified as financially significant.

Understand the impact of enterprise application upgrades

Rather than running an entire regression test suite for each update to their enterprise applications, organisations should work to understand what impact these updates will have on critical business processes, and focus testing on those areas instead.

Traditional update tools will reveal what areas of a system have changed following an update, whereas a tool that can also identify the areas that need to be tested as a result of that update will deliver far greater efficiencies.

An AI-powered impact analysis, for example, focuses the scope of a test down to the impacted objects with each release. Any testing gaps discovered will be automatically added as requirements so that automated test cases can be created to fill the gaps. Pinpointing the areas most at risk, and that need to be tested, has been found to reduce testing times by 85 per cent, on average, while ensuring 100 risk coverage.  while ensuring 100 risk coverage.  

Manual testing can be a slow and expensive process for an organisation - the benefits of automation are clear. But many organisations lack the necessary resources and skills to automate testing at scale. As we’ve seen, however, this needn’t hold them back. Changing their approach to testing can enable organisations to enjoy the speed, quality and flexibility they need to meet the demands of an increasingly competitive environment.

Wolfgang Platz, founder and chief strategy officer, Tricentis (opens in new tab)

Wolfgang founded Tricentis in 2007 as a testing consultancy. Today, he’s responsible for driving Tricentis’ vision for making Continuous Testing a reality for enterprise DevOps.