A dozen years ago, when I was building my first web apps, security took a back seat in priority to the pace of delivery. I would do the basics but typically forego more complex security tactics that were difficult to easily accomplish. Although public-facing website traffic was encrypted with SSL, the encryption of data in transit and at rest internal to the app was not because it was so difficult to implement and manage. DevOps as a concept was in its early stages, and the Wall[s] of Confusion that separated developers, operations and InfoSec seemed thicker than steel. The idea that these seemingly disparate entities should work closely together seemed like a pipe dream.
But then digital transformation happened. Within a few years, the types and numbers of machines talking to one another every second of the day have made clear that these walls need to come down—something that still challenges many Global 2000 organisations. Moreover, the need for encryption to protect this explosion of machines (everything from 30-year-old mainframes to short-lived containers) has become more important than ever before—leading to new terms like DevSecOps. All three groups need to work together to ensure that all these machines—which can number in the hundreds of thousands, or even millions at Fortune 500 companies—interact only with internal and external machines that have legitimate keys and certificates, which act as machine identities.
This dawning awareness over the last several years has transformed IT organisations—but old notions about how these former silos must interact with each other has lagged, leading to several DevSecOps myths.
Myths around machine identities especially problematic
Recently, I realised that some of the most deep-seated DevSecOps myths revolve around machine identity challenges and the traditional divide between development teams and InfoSec teams. And I admit, I used to believe in some of these because the technology at the time wasn’t available, well-known, or required deep crypto and PKI expertise.
After working with a number of Global 2000 customers in my position at HashiCorp, however, I’ve learned that addressing these machine identity challenges early on pays off in the long run.
When you don’t consider these issues upfront, you risk making short-sighted technology or process decisions—and end up paying for them later, through outages, data breaches and even app performance.
The three myths I’m about to discuss have persisted because our ways of thinking are still, in many cases, influenced by old pre-DevOps paradigms. Things like great processes and technologies make DevSecOps methodologies possible—but ultimately, it’s the culture of communication and collaboration we build around these processes that makes DevSecOps successful.
DevSecOps Myth No. 1: DevOps teams don’t care about security.
This is the most persistent myth out there—maybe because it underscores the type of divide DevOps has been trying to undo since the movement started. But coming to this conclusion mistakes a fact (that a developer failed to incorporate security) with a reason (the developer did so because they don’t care about security). No developer thinks, “Yes, I want to send unencrypted traffic over the internet.” An insecure app—no matter how fast it loads or how much it delights the customer—can put the company at risk if an outage causes it to stop working or a breach leads to stolen customer data.
But the pressure to build great apps quickly never lets up for developers. Digital transformation forces companies to compete by offering the best customer experience available. Because developers are under such intense pressure, they have to prioritise. When faced with fast-tracking, say, a way to improve updates to online retail shopping carts in this age of Covid-19, security may fall in priority to speed of delivery. It’s not that they think machine identity protection is unimportant, but developers aren’t often judged on the quality of the keys and certificates being used to encrypt these apps but on the app itself. They are then faced with the decision of delaying deadlines or spending the extra time required to do things securely.
To solve this problem, InfoSec teams need to view DevOps-related security issues as an opportunity to improve without jumping to preconceived notions about why this may be the case. They should reach out to developers to understand their unique constraints and concerns and then take that information to implement processes that interweave machine identity protection into the core part of the software development lifecycle. These channels of communication must remain open so that they can adjust process and technology as necessary.
Fortunately, a host of technologies are available to integrate security into development workflows and accelerate application delivery in secure ways. Many of these solutions use APIs to help with things like secrets management, PKI encryption and certificate issuance. A developer may not understand the intricacies of PKI—but they do speak fluent API. If requesting or rotating certificates is achievable via an API and can be integrated into common orchestration tooling, developers are much more likely to adhere to best practices when managing certificates. By making these tools and processes available, developers can share ownership in making their builds secure, which ultimately makes security’s job a whole lot easier—and your organisation a whole lot safer.
DevSecOps Myth No. 2: It doesn’t matter if the TLS machine identities DevOps teams use match your company’s security policies.
This myth follows from the first one. Think about how TLS certificates in most companies have traditionally gotten issued. Typically, developers must:
- Submit a ticket to request a certificate.
- Wait for the human on other end to determine your use case and approve the certificate request.
- Wait for the certificate to be issued and available for the developer to use.
This turnaround can take days, weeks or even months to complete. If a developer is in the middle of a 10-day sprint to push out a new feature, they can barely afford an hour, let alone days or weeks, waiting on a certificate.
As a result, developers, pressured to complete that sprint, look to bypass this onerous process. So, they rationalise: “Hey, if I just put this in a container and deploy it on the cloud, then I don't have to go through all of that ticket-based process of requesting certificates.”
Again, developers aren’t being malicious or neglectful—they’re just trying to get their jobs done. As we discussed in the previous myth, security teams, along with operations teams and executive leadership, must take the lead in making sure that DevOps teams adhere to corporate security guidelines when they procure TLS certificates. The trend toward containerisation means that development teams must have a simple way to manage certificates for all services, regardless of how they're deployed.
It falls on InfoSec teams to come up with a better solution that bakes your organisation’s security policies into software development cycles. As I discussed in the first myth, you now have technologies that use automation to make sure that the certificates requested by developers adhere to certain standards and approval processes—none of which require specialised knowledge on the part of developers. Developers would like nothing more than the confidence that whatever code they’re pushing out has secured machine identities that won’t impact the performance of the apps and services they are building.
DevSecOps Myth No. 3: Corporate PKI won’t work for DevOps projects.
If you say that corporate PKI won’t work for DevOps projects, you’re assuming that some other PKI—like self-signed certificates or whatever your cloud vendor offers—will. The problem with this assumption is that you end up having different roots of trust. When this happens, the apps and services developed through DevOps methodologies may not be able to trust or communicate with on-premises applications that are connected using your corporate PKI.
In other words, if you want your systems of engagement that are born in the cloud to speak with your systems of record on-premises, you need a solution that takes your corporate PKI into consideration. If you don’t do this, you may inhibit your company’s ability to move applications to the cloud quickly and securely. If you want to provision in the cloud or secure what’s in your cloud instances, a single source of PKI across your IT environment can drastically reduce complexity, if properly implemented.
So, what do you do to make your corporate PKI issue secure certificates in a way that works for DevOps projects? One word: automation. DevOps as a methodology couldn’t happen without automating a majority of processes. In fact, during our first conversation, Mitchell Hashimoto, founder and CTO of HashiCorp, said, “I look to automate as much as possible. If there's something that can be automated, I try to figure out a way to do it.” That’s because by automating as many processes as you can, you reduce human error and accelerate the execution of that process.
Take advantage of the automation capabilities that these newer solutions provide to manage certificate lifecycle processes, as well as the security policies that limit their use. Development teams will thank you, your professional life will be made a whole lot easier, and most importantly, your organisation will be safer and more agile going forward.
Jon Benson, VP, Worldwide Solutions Engineering, HashiCorp