Skip to main content

Don’t let test environments slow you down – avoiding the top five testing pitfalls

(Image credit: Image source: Shutterstock/niroworld)

Without test environments, application delivery simply would not be possible. They are a crucial part of the pipeline that ensure your applications are fit for purpose, as they catch any issues with the code and allow developers to perfect this before it is released. But managing test environments can pose a range of challenges as well, which can quickly escalate if they are not dealt with when they first arise. By attending to these now, businesses will be in a better position to work towards a long-term goal of environment as code (applying DevOps methodology to the whole production environment).

There are five main test environment pitfalls that businesses typically struggle with, but there are ways that teams can tackle these to prevent the difficulties from spiralling out of control.

1) Lacking visibility of environment bookings

Environment schedules are key when it comes to keeping track of your test environments and when dev teams have booked them for – or would like to. Without a transparent schedule in place, managing this process is near impossible unless you have a very small number of environments to coordinate. As most businesses are rapidly increasing the number of environments that their teams require, a scheduling system is crucial to avoid clashes, misconfigurations or poor use of resources.

Ideally every business would be able to scale environments up and down as required – and produce as many copies as required – with the right data and components, all on demand. But realistically this isn’t always possible, and therefore you need to schedule it as well as determine how to share any individual elements to build an overall picture. While many organisations may rely on Excel spreadsheets, implementing a solution that includes scheduling will vastly improve the way your business is able to manage its environments and should thereby make this aspect of the delivery process smoother and more efficient.

2) Not understanding your non-production environments

Most organisations know that they have non-production environments, but they are unsure of how they are configured, what applications are on them, and what data is present. You need to have a source of truth about these which should show all of the applications, the environments and who is using them – but often these are just for the production environments because the applications have been implemented by operations teams that are focused on production. By implementing this automated system for non-production environments as well, every team can clearly see all environments in real time, keeping every process on track.

The most efficient way to introduce this to your pipeline is by starting with what you know – use any projects that you already have and get the details of what processes they use. Then, once you start to build on this, you can talk to more people and create a more comprehensive overview of all of the production and non-production environments across the business.

3) When non-production environments aren’t in sync

The challenge of completing initial testing, such as systems integration testing, is that often this does not occur at a production baseline, so when everything moves into UAT for the users, this is when all of the defects appear for the first time. When you make the application and configuration changes to the environment, you aren’t comparing apples with apples, and the best solution to this is configuration management.

To manage the configuration of your environments, check the resources on the box up to what the operating system is, as well as the version of Java in use or the versions of the applications and the data. Keep your environments in step and then the only changes that you will need to make to them are the new code and configuration as and when you deploy them. If you don’t have a configuration management solution, start with the programmes in use – such as Confluence or SharePoint – and work to put this in a single location that everyone updates.

4) Uncertainty of how it all fits together

Environment diagrams may sound simple, but they are one of the most comprehensive ways to see all of the applications and components in your non-production environments and how they hang together. Without any clear pictures of this, teams will struggle to understand what everyone else is working on; as components become more critical, dependencies increase and so ensuring proper versions exist will become more important. Currently, teams rely on going directly to every other team to find out what progress they have made, which in turn makes it harder for developers to coordinate their actions.

The most effective way to resolve this is by starting with the very original environment designs – they must have been drawn up right at the beginning to enable the developers to create the first versions of these environments. Take these designs and ask the developers to help you update them until they’re accurate for the current layout and include anything that can pictorially describe the non-production environment and how it works.

5) Having to take a new team approach

A new approach to your test environments will very likely require a new environment management team to support it. But it can be challenging to know what size team you are going to need or what you will want them to focus on. And if you work out your numbers wrong by under- or over-estimating the size of your team, you risk either burned out workers who are taking on too great a workload, or employees that have too much free time and are not used to their full potential.

The goal here is to organise a team to address the Achilles heel of development – the test environments. Maturity can be established through the following step by step process:

  • Get organised – Put a system in place to track what you have, even if this is only a spreadsheet.
  • Manage – Use a test environment management (TEM) tool to manage bookings, change requests, etc.
  • Incorporate full stack and some level of automation – Treat environments like a full deliverable that includes compute/memory along with middleware, applications, test data, and test infrastructure.

Ultimately, the biggest lesson to take away is to get to know your test environments thoroughly, both production and non-production. A weak environment strategy will create fragility in the delivery pipelines with a wealth of unnecessary, wasted effort. It’s easy to automate, it just requires the focus to solidify your strategy, which overall will pay big dividends in the end.

Jeff Keyes, Director of Product Marketing, Plutora (opens in new tab)

Jeff Keyes is the Director of Marketing at Plutora.