While software testing has always been a challenge, early-stage testing—unit tests—are getting a lot more attention these days due to the important role they play in facilitating DevOps adoption. The earlier you find bugs and problems in your code, the faster you can fix them, and the faster you can ship. It’s hard to adopt continuous integration if you keep building software that doesn’t work.
When the problem seems easy to identify (e.g., "We need more unit tests"), it's tempting for engineering managers to try to solve it by simply instructing their development teams to change their behavior (e.g., "Write more unit tests"). And this is what's happening: More organizations are implementing code coverage targets or instructing teams to practice test-driven development and its offshoot, behavior-driven development.
But merely telling developers to change their behavior is not working, because developers are often unable (and unwilling) to spend more time writing tests. Why is that? Developers have the skills, and most organizations are capable of providing adequate motivation, but working environments aren’t always conducive to effective testing. We frequently underestimate the power of environment to shape behavior, but we find it easiest to do things when they are convenient.
So, how do we make writing tests easy and convenient?
- Many organisations aren't testing their software enough (opens in new tab)
The problem with testing
Almost all code needs tests of some kind. This means it can be daunting to start prioritizing what to write tests for in a codebase with tens of thousands of lines of untested code, and it's hard to write tests alongside new code under tight deadlines. Testing also has a "fun" problem: writing tests is tedious and repetitive, requires strict attention to detail and is less interesting than other work. Finally, testing is plagued by the archaic belief that it is optional and comes at the end, when all the coding has been done.
Most developers know they need more tests, but they are under pressure to prioritize new feature development instead. In a 2019 developer survey, 48 percent of respondents said they struggle to hit the code coverage targets imposed by management, and 42 percent admitted to skipping unit testing to work on new feature development instead—effectively sacrificing code quality in order to meet deadlines.
A recent survey of 300 developers and senior-decision-makers across the US and UK found that, though many developers express an openness to practicing TDD in theory, its actual implementation appears to be overreported. While 41 percent of the developers who responded to the survey said their organizations have "fully adopted" TDD, only 8 percent said they write tests before writing the code under test—the very definition of TDD—at least 80 percent of the time.
The status quo of telling developers to write more tests without making it easy and convenient to do so will change very little, because their environment and culture doesn’t support them.
What is culture?
In the corporate world, culture is typically defined as the mixture of the norms, values, and behaviors shared by an organization. Classic definitions of culture, however, distinguish between non-material culture (all of the above) and material culture: the tools, machines, and other objects used by a group. This is important because a company's toolstack is as much a part of its culture as the values it espouses and the behavior it encourages; more than that, tooling actually enables cultural change.
Often, organizations decide to start adopting DevOps culture, and then introduce tools later. However, introducing tools first is often more effective. Consider the impact Slack, for example, could have on a team that values real-time communication but has always relied on email. If Slack makes real-time communication faster and easier, teammates will start to communicate more.
It is not that tooling is a “culture magic wand”, but thoughtfully applied tooling can make doing the “right thing” easier and more convenient, and what’s convenient is what gets done.
If you value something, invest in it
One of the core issues with testing is that it is so resource-intensive to write tests that it typically can't be done without investment. If a company says it values code quality, leaders must ensure that their resources (the toolstack, budget, and time developers are given to allocate to different tasks) actually support it.
By investing in the tools and processes that make testing easier and more convenient, organizational leaders don’t just signal to developers that code quality actually matters — they are actively smoothing the path to better code quality. Developers feel this more keenly than their managers: In a survey of 300 developers and senior decision-makers, 55 percent of managers agreed that the biggest hurdle to adopting a DevOps testing culture is a lack of dedicated resources by management. In contrast, 81 percent of developers agreed with this statement.
Investing in tools to make testing faster and easier—for example, by automating not only how unit tests are run, but also the creation of the unit tests themselves—can drastically improve testing abilities. What’s more, it’s an investment with a solid return. Allocating time and establishing processes for unit testing makes it easier to catch issues in the earliest stages of the pipeline, when they are 30 times less expensive to fix compared to production or post-release. Companies that don't allocate budget to testing or time to discovering new tools will be less likely to implement these solutions.
- Organisations rarely test for vulnerabilities (opens in new tab)
How to become a testing culture leader
So what does (and doesn't) make an effective testing culture? If very little budget is devoted to making testing easy and convenient, then by definition any testing that occurs will be informal, ad hoc, and driven by individuals who make their own time to do it. For these testing culture beginners, implementing TDD and code coverage targets will rarely be effective.
In contrast, the characteristics of organizations with a leading testing culture include:
- Building testing time into software development schedules and sprints.
- Organization-level adoption of formal testing processes, as well as methods for evaluating their efficacy.
- Allocating budget to finding and implementing tools that make testing easy and convenient, and automating as much as possible.
- Allocating budget for training teams, so they have a common understanding of testing principles and can share testing responsibilities.
For an organization with a solid testing foundation, code quality is actively valued and supported, and behavioral changes that encourage more testing are achievable.
Testing culture is essential to DevOps culture
Without tests that run early, run quickly and help developers find bugs, adopting DevOps and driving down cycle times will be difficult. Tooling can help make “doing the right thing” the easiest thing to do: by changing the developer’s environment, it drives the testing culture you’re looking for. Married with motivation and a solid feedback loop, organizations can shift behaviors in the right direction.
- How penetration testing can help secure your company’s infrastructure (opens in new tab)
Mathew Lodge, CEO, Diffblue (opens in new tab)