Skip to main content

Maintaining order in a hybrid cloud, multi-tool world

(Image credit: Image source: Shutterstock/Nattapol Sritongcom)

It’s a matter of survival. In order for enterprises to stay relevant and competitive, digital transformation grows more critical by the day. Increasingly complex IT infrastructures require DevOps and continuous integration to support digital transformation and efficient innovation. As a result, organizations have made a seismic shift to hybrid cloud, cementing it as the de-facto enterprise IT architecture.

Hybrid cloud does indeed provide enterprises with the agility and scalability they need to support their digital transformations. But the benefits also come with a number of new management and governance challenges which, if overlooked, can impact hybrid cloud success.

For starters, IT departments must change and adapt their processes and protocols in order to monitor, orchestrate and provision a variegated mix of public clouds, private clouds, and on-premise systems. They are also tasked with ensuring security and compliance across an increasingly heterogeneous cloud landscape, which can be difficult for staff that may not have expertise in cloud systems. And on top of all this, they’re still responsible for managing day-2 operations and keeping costs under control.  

Since manually performing all the tasks needed to achieve hybrid cloud success is pretty much impossible, enterprises must invest in new tools and platforms that can automate as many processes as possible, taking the burden off of IT and preventing costly human errors. This new multi-tool dynamic presents some challenges, but embracing best-of-breed tools ultimately will lead to greater overall success.

Multi-tool means multi integration challenges

In order to be able to manage hybrid cloud architectures and capitalize on the agility and scalability benefits that hybrid cloud promises, enterprises are aggressively investing in automation tools like VMware vRealize, Terraform, Ansible, and Kubernetes. These tools not only improve efficiency, but they also enable organizations to reduce the governance, security and cost-control challenges of hybrid cloud architectures.

Most companies have come to terms with the fact that there is no single solution that can provide a cloud panacea, which means they are free to choose the best technologies to meet their needs. The downside to this, however, is that organizations must now find ways to integrate these tools with existing software stacks without creating more overhead and technical debt.

Traditionally, IT teams would turn to extensive custom coding to integrate these tools with the underlying IT technologies they access and orchestrate, such as IPAM, DNS, networking & security, and cloud backup. In other words, before your organization could take advantage of the efficiency and control offered by these tools, you have to undergo a long, costly and oftentimes painful custom-coding experience. Of course, this code has to also be maintained as versions or environments change, so the tool you added to make things easier ends up giving you more operational headaches over time.

Custom coding automation tools costs time and money

To achieve the promise of automation, integrating your automation tools with each IPAM, DNS, or networking & security technology will require manual custom-coding by your IT people. Even when pre-built integrations or APIs are available from the infrastructure vendor, the one-size-fits-all nature of a plug-in will almost always need some level of configuration. And when APIs change, these integrations will have to be rewritten. So whether you're updating your own code or updating vendor-provided integrations, you’ll continue to accumulate technical debt as your infrastructure scales and evolves and more tools and technologies are added.

Even if your IT team can handle all this – after all, engineers are smart people – it will cost a lot of money. Research by the Standish Group reveals that over 50 percent of custom-coded projects cost almost double their original estimate, while about one fifth of custom-coded projects will be cancelled before they are even finished. Gartner estimates that custom integrations typically result in a 200 percent increase in long-term software maintenance costs for organizations.

Furthermore, time and expense aren’t the only issues you need to worry about with custom integrations. The need to write and maintain code and scripts for each new tool you want to adopt will present governance and security issues as well.

Governance and security pitfalls of custom-coded automation tools

Governance and security become more complicated with custom-coded integrations. A growing number of new tools and processes will eventually prevent IT from having end-to-end visibility of all integration policies and automation playbooks. Troubleshooting and auditing integrations becomes a nightmare scenario of sifting through multiple logs, from multiple systems, when trying to isolate and remediate issues.

Can IT integrate new tools, enforce naming standards for increasing cloud resources, and maintain evolving security protocols in a standardized, across-the-board way? Can compliance leaders easily audit integrations and policies? If not, you will have resource sprawl, patching issues, and you’ll be more vulnerable to attack.

Considering 60 percent of all infrastructure outages and a substantial number of cyber breaches are caused by human error, end-to-end visibility and enforcing standardization are must-haves for enterprise organizations. But the more custom code or one-off scripts in your IT architecture that you have to accommodate, the more difficult this becomes. And the challenges don’t stop at governance and security; the requirements of custom coding will also limit scalability and the time to value of new tools you want to adopt.

Custom coding limits scalability

As your company – and the digital landscape – grows and evolves, you will want to add more intelligent automation tools that perform more complex tasks, particularly machine learning and artificial intelligence (AI). But in order for these new tools to function optimally, they will have to abstract data from existing tools, which likely have been customized to accommodate business logic not available out of the box.

For example, if you want to integrate a new AI tool, that tool will not be able to automatically abstract data from custom-coded workflows. The code was written specifically for previous integrations, so the new AI tool won’t recognize a language that is unique in business logic. So in order for the new AI tool to integrate and provide the value you want, you’ll have to – you guessed it – write more custom code. Each existing line of code, which could number in the hundreds of thousands, will have to be examined manually to determine the business logic behind it. As the digital landscape evolves faster and your IT architecture needs to scale and incorporate more complex tools to stay competitive, this situation could significantly limit time to value of your new tools and, ultimately, your bottom line.

Custom code is no longer a necessary evil

Despite the time, money and resources custom coding consumes, and the security, governance and scalability challenges it brings, it is still seen as a difficult but ultimately unavoidable endeavor. As long as hybrid cloud exists, enterprises will need a best-in-breed, multi-tool approach to stay competitive. In order to cultivate a best-in-breed, multi-tool approach, you will need to custom code and you will need an army of consultants. What else can an enterprise do?

As it turns out, custom coding is no longer a requirement for integrating automation tools into hybrid cloud environments. There are now solutions available that allow enterprises to integrate without having to custom code, but not all of these solutions are created equal. 

Point solutions such as Zapier and Tray, for example, are becoming increasingly popular and do in fact provide some simple integrations. However, these solutions weren’t designed for complex data center and public cloud use cases like provisioning and orchestration. iPaaS solutions have become another option, but they often increase complexity since they are themselves fairly complex in nature. They may be able to support any part of the organization, but they require platform-specific expertise that few IT teams possess.

For cloud teams looking to maintain more control, there are now cloud management solutions that are built to facilitate both management and integration. For instance, codeless, policy-based solutions to integrations eliminate the need to custom code while maintaining integration standardization through a robust integration layer. Not only does this make it much easier to adopt a growing number of automation tools – tools that will only grow more intelligent and sophisticated in the future – but it also ensures the end-to-end visibility you need and a standardized approach to governance and security.

Going forward

The most agile hybrid cloud infrastructures will be the ones that allow for the easy adoption and integration of new tools in a best-of-breed approach. Attaining this level of agility requires an IT architecture built for extensibility and high levels of automation. Specifically, you need automation that eliminates the need for custom code while allowing you to quickly unleash the power of any tools your business demands. It’s important to keep in mind that automations and integrations must be able to scale over time as tools and processes change, which they frequently will given the sheer volume of growth in cloud.

Take the case of VMware vrealize (vRA), VMware’s flagship CMP with thousands of customers in North America. Many of these enterprises have built custom code to optimize vRA 7 – the current version of the product – and they’re now on a path to migrate to vRA 8, a completely re-architected platform. In addition, many of these enterprises also have other automation tools like Kubernetes, Terraform, and Ansible alongside vRA.

While this presents a good opportunity to bring all these technologies closer to cloud models and incorporate DevOps, enterprises will typically invest in armies of consultants and write custom integrations to migrate to vRA 8 and optimize the potential of their automation tools. But why should custom be the first instinct? A software-defined approach can be cheaper, it can be better governed, and it can scale much more easily. In the long run, these will all be significant boons to IT, security, and developer teams as they embrace more tools for hybrid cloud.

Grant Ho, CMO, CloudBolt