Organisations in today’s highly competitive marketplace are under increasing pressure to rapidly deliver greater value to their customers, while achieving critical business objectives, such as increasing market share.
These demands have led to dependency on the DevSecOps function to deliver ever higher quality applications and services at ever higher speeds, while ensuring an exceptional customer experience and enhancing security. DevSecOps’ many advantages are in turn driving significant growth. The growing importance of the DevSecOps function is in turn driving significant investment in this area; the global DevSecOps market is predicted to reach $6.1billion by 2023, growing at a CAGR of 33 per cent.
However, the practice also presents challenges. The demands of facilitating a secure continuous service delivery pipeline can often mean that DevSecOps is required to build, test, integrate and deploy several new releases a day; a task which requires considerable proficiency and organisational maturity.
Chaos in the system
Consider the example of a cutting-edge financial services firm launching innovative machine learning solutions to predict market fluctuations by factoring multiple attributes related to investor behaviour. The firm’s DevSecOps team found itself under considerable pressure to deliver new functionality to its customers, while scaling rapidly from just a few hundred thousand to millions of subscribers within a year and assuring a delightful customer experience.
To achieve these ambitious digital transformation objectives, the company turned to a hybrid cloud deployment. Not only did this provide on-demand, scalable, and elastic environment for lifting and shifting applications, but it also allowed DevSecOps to natively develop and test new applications in the cloud, utilising a microservices architecture.
In preparation, the DevSecOps team carried out a SWOT analysis, defined Minimum Viable Products (MVP) requirements, and validated concepts with A/B testing. Once satisfied with the outcomes, the team was happy that it had a solid service launch strategy.
An agile development methodology enabled the team to accelerate delivery of new releases from a weekly to a daily basis and monitor the user experience with regard to service availability, reliability, and responsiveness, as well as Net Promoter Score (NPS), the index used to gauge a customer’s overall satisfaction with a product or service. Unfortunately, the DevSecOps team soon discovered that the user experience was beginning to degrade, leading to attrition of customers and reducing the number of new customers joining the service.
The team realised that an unpredictable level of chaos in the system was causing an excessive number of issues in the production environment, preventing DevSecOps from accelerating beyond a certain frequency of delivering new service functionality. Even with upgraded automation tools, the team was unable to solve the problems it faced. A fresh perspective was therefore required.
Instrumentation and monitoring
A consultant was hired to offer an objective assessment as to what was holding the firm back from achieving its goals, and found the main issues to be shortcomings in its instrumentation technology, its monitoring tools, and the DevSecOps team’s own maturity.
As an important part of the IT toolkit, the right instrumentation provides much needed visibility, illuminating applications and infrastructure whilst delivering the insights and telemetry necessary for service assurance. In this case, the bytecode instrumentation technology used by the DevSecOps team offered limited server-centric visibility into production environments. What it needed was pervasive visibility across individual applications, and across the entire hybrid cloud infrastructure and their respective interdependencies. Providing such levels of visibility requires analysis of wire data, or IP traffic flows, which offers a far more detailed view of the infrastructure, applications and dependencies, allowing the extraction of actionable insights in the form of smart data.
The consultant also noted the importance of the need for DevSecOps to develop a new strategy for continuously monitoring in complex hybrid cloud environments. Within most complex systems, there can be a high degree of interdependencies between coupled components. These can exhibit behaviours distinct from the properties of their parts, and will often respond in different ways to the same input depending on their state or context. Looking independently at individual service delivery domains, such as applications, networks, and servers, will therefore not necessarily offer insight into the system-level issues related to interdependencies across those domains.
For DevSecOps to gain true insights of the system required a single system-level monitoring tool for the entire team. With pervasive visibility into all the individual domains and their interdependencies, everyone would have common situational awareness, and could collaborate effectively and share the same actionable insight into application performance and user experience.
The final issue seen to be hindering the firm’s progress was the disjointed nature of the DevSecOps team itself. A lack of DevSecOps maturity occurs when Dev, quality assurance (QA), Ops and Sec teams each work independently, their objectives siloed with little regard to feedback from other teams. If a service problem is uncovered, the Network, Server, and Database administration teams will point fingers at each other, leading to extended triage and root-cause analysis. Not only can this be extremely time consuming, it can also create chaos and prevent the DevSecOps from operating efficiently.
For the firm to achieve its objectives, it therefore needed to transform its disjointed DevSecOps function into an overlapping organisation with overlapping goals. In a collaborative environment, based on a common situational awareness across the entire DevSecOps team, Dev, QA, Ops and Sec teams would share goals and closely interact with each other throughout the software development life cycle process.
During the development stage, for example, the Sec team would be responsible for assuring that code was developed in compliance with app development best practices, as well as corporate policies and industry regulations. Then, during the QA phase, it would be tasked with ensuring test scenarios and use cases emulating real-life production environments, to assure that application performance and security are properly validated at scale.
This next level of transformation should lead to the formation of a congruent DevSecOps team, with common goals and team-level accountability based on a common situational awareness established through the monitoring of application performance and security at the system level, including all respective interdependencies. In achieving the highest level of maturity, the resultant DevSecOps team will be able to continuously learn from experience and improve its effectiveness, efficiency, and the quality of the code it delivers. Empowered to achieve application and security assurance, the team will ultimately improve the user experience.
Faced with the challenge of delivering new functionality to its customers, and facilitating a rapidly scaling continuous service delivery pipeline, the financial services firm urgently needed to transform its DevSecOps team from being disjointed to becoming first an overlapping and eventually a congruent organisation, or risk losing existing and future customers.
Following the consultant’s recommendations, the team implemented a new and pervasive service monitoring solution based on wire-data, processing this data at its source to convert it into smart data for deep insights into application performance and security. This, in conjunction with the consolidation of its monitoring tools, granted the system-level visibility it needed for the common situational awareness for effective collaboration that sits at the heart of a mature DevSecOps team.
With this transformation complete, the firm is now on its way to achieving the continuous delivery pipeline with speed and agility that will enable it to meet its critical business objectives, and keep its customers satisfied.
Michael Segal, area vice president of strategic marketing, NETSCOUT
Image Credit: Profit_Image / Shutterstock