Using ARA to achieve higher quality software releases

Application Release Automation, or ARA, is the process of packaging and deploying applications (or updates to applications) in an automated, repeatable and consistent way.

In today’s market landscape software has become a major differentiator for many businesses, across all industries and verticals. In a world of apps, Internet of Things and customers’ expectations to be always connected, – just about every company is now becoming a software company. Software development is now critical to the success of your product, and its delivery process needs to be fine-tuned and streamlined to ensure that you get quality software in the hands of your end users as quickly as possible.

To understand how ARA can help you achieve higher quality software releases (which contribute to your bottom line), you must first understand the challenges you might face if your business is still using manual deployments, or other archaic methods. If that is the case, though, you’re not alone: a survey we ran last year found that over 75 per cent of deployments are still done manually, despite the many problems with this model, which is extremely brittle and error prone. This creates not only many failed deployments, but the loss of hundreds of men hours trying to troubleshoot those failures. This is certainly painful in the mission critical production environment, but carries great cost long before we reach production. Even in the lower environments (such as testing or pre-production), 66 per cent of respondents report that it is very time consuming just to troubleshoot why the failure occurred. As we promote code to be deployed in production- the stakes are higher. Costs of failed deployments, delayed releases and loss of men hours are exponentially aggravated when you add possible fines on top of that, cost of SLA violations, customer churn, and loss of business. There’s no question the implications of failed deployments put a major dent in your bottom line, and with some companies – failed deployments have literally put them out of business.

A prime example of how a failed release can have huge impacts on a business is the case of Knight Capital. Knight Capital WAS a financial firm. In August of 2012, they manually pushed out a new release to their software when the markets were down over the weekend. Monday morning came and when the market opened, their new software kicked in. The results were disastrous. They accumulated (extremely quickly and erroneously) a $9 billion [H1] position in the market – represented by the movement of 397 million stocks in just 45 minutes. Two hours later, when Knight was still chasing the culprit in their release process and trying to rollback to a previous version of the software, the New York Stock Exchange said, “We’re done waiting for you,” cut their hardline to the market and Knight couldn’t make any trades.

The effects were disastrous and swift. Knight Capital endured a pre-tax loss of $440 million. Within weeks, Knight Capital was sold for pennies on the dollar – a failed release literally killed the company. While this is certainly a worst-case scenario, it is a great example of how it is necessary to understand that once you start talking about production environments, there is a lot of regulatory and best practices that have to be followed or big problems can occur.

Why are manual deployments so problematic?

First of all, manual deployments are error prone because each deployment has to be done by an actual human. Even the most veteran of developers or ops engineers make mistakes, and when you start talking in the hundreds (or thousands) of deployments, across complex stacks and configurations, an error is almost inevitable. While no system can fully-eliminate the potential for an issue, you can imagine that in a deployment process where many of the steps are automated – and therefore consistent across releases – your chances of an error are reduced dramatically.

With manual deployments, it’s also difficult to achieve true, fine-grained, visibility and auditability into the entire deployment process of all the application’s components and their corresponding infrastructure or configuration. Once a deployment is pushed live, the only person who really knows how, exactly, the update was promoted from Testing to Production is the individual (or individuals) who performed the release. And as good as they might be in keeping notes, they’ll never be as comprehensive, or bulletproof, as an automated log that tracks every single component in the multi-stages deployment process.

Manual deployments also are not adept at negotiating what is called the “Delivery Gap.” Essentially the delivery gap is the chasm that exists between three different factions of the software team. On one side there is the development team, which has a bunch of tools and practices they like to use. On the other side is Operations, which has their own set of tools and methods/requirements. Then, finally, there is the infrastructure, which spans across both Development and Operations and places requirements on both parties. So you have essentially three different players, using different tools, with difference practices and requirements, and negotiating between them is a tricky and difficult process. Ensuring there’s no configuration drift between infrastructure or app components across the different team, and ensuring consistency in deployment processes has proven difficult for humans to perform, particularly as each process includes hundreds – if not more – of artifacts, configuration files, dependencies and deployment requirements. For an automated process, however, consistency and repeatability are the default state.

So now that we’ve established the pitfalls of manual deployment practices, let’s talk about how Application Release Automation can solve some of these issues and support higher quality software releases.

We’ll start with the aforementioned “delivery gap.” Application Release Automation is a DevOps solution that bridges the gap between the Development teams and Operations teams. It does so by creating a solution that encompasses Continuous Integration, Configuration Management and Deployment practices that respect infrastructure, as well as operation’s needs and regulatory requirements. It achieves this because you can build these considerations into the automated process, which is recorded, logged and tracked by default. Rather than each deployment being a unique ‘snowflake’, software deployments become routine and predictable. It’s not on one person to mentally “flip the switch” and jump back and forth between groups to make sure everyone is satisfied. Rather, it is an automated process that takes everyone’s interests and requirements into account, and then makes sure everything fits within the infrastructure and configuration requirements.

Furthermore, ARA is a “model-driven” approach that allows you to reuse existing templates, configurations and processes over and over to make your deployments consistent, and require minimal overhead for the initial set up of the process. This means you can deploy any version to any environment quickly, with push-button deployments. This not only increases the speed of deploying across different environments throughout your process, up until the final release to Production, but also allows for repeatability in a way manual deployments simply can’t. This is especially important when it comes to testing. Imagine if the framework of every update you deployed was identical – how much easier would that make testing? And how much easier would your tests be if they were all run continuously and automatically? That is what ARA can offer.

Additionally, with consistency, your deployments are very predictable. You know what the run times will be, allowing you to plan your development processes more accurately and efficiently. This means you can allocate time and resources in a way that maximises your output – which allows your team members to spend more time on their core competencies and improving your product, versus having to manage the deployment process or the infrastructure.

Along the same line, one of the key tenets of ARA is packaging. When you manually deploy updates, bits and pieces that comprise your application are being deployed (“moving”) at different times, creating a “noisy” process, with lots of dependencies, that’s difficult to manage. With ARA, everything is wrapped up in a tidy “package.” This effectively creates a “bill of materials” for your release and all dependencies and related configurations, which allows you to easily indicate what components of the application need to be deployed where, and in what order.

Application Release Automation also solves the issue of having such a critical process to the company be dependent on what is often just one person – a “star student” in your team who “holds the keys” to the project and only he has the tribal knowledge and expertise to perform or debug a deployment. With ARA, there is an implicit transparency to the project that gives any team member the ability to access the system and deploy the software (pending the right security permissions, of course).

Lastly, with ARA, you have complete visibility of the process from start to finish. If something goes awry, you have a much better view of where the error originated from, making recovering much faster and easier, and the damage much less significant.

Ultimately, a successful ARA strategy eliminates the need to build and maintain custom scripts for application deployments, while simultaneously reducing configuration errors and downtime. It allows strategic allocation of resources, giving users the most “bang for their buck” from their developers and other IT professionals. Because of all these factors, Application Release Automation is a hugely effective strategy for increasing both the quality and speed of your deployments.

Anders Wallgren is Chief Technology Officer at Electric Cloud