Cloud Management is a key aspect organisations are looking at on their journey to becoming a software-driven enterprise – in order to simplify operations, increase IT efficiency and reduce datacentre costs.
Enterprise IT has to juggle supporting many different types of applications (legacy, mainframe, monolithic, cloud-native apps, microservices, serverless, and more) – each requiring their own stack and type of infrastructure, that is operated and maintained in a fundamentally different way. In addition, enterprise infrastructure is commonly comprised of both on-premises data centres and public cloud workloads. Thus, a key challenge for enterprise IT is around managing hybrid/multi clouds – encompassing both private/public clouds – across VMs, containers, serverless applications, and more.
Hybrid clouds and Cloud Management Platforms (CPMs) have gotten a bad rep in recent years, and with good reason. We now have the knowledge, the technology, and the opportunity to change the narrative. Let’s review the challenges with CMPs and the key capabilities required for enterprises to support digital transformation, for both legacy and modern infrastructure.
Challenges with existing Cloud Management Platforms (CMPs):
- High cost – CMP technologies are typically proprietary and charge heavy license fees. Most prongs of CMP are implemented as point-products, which require considerable and lengthy investments in Professional Services and custom integrations in order to implement effectively. This typically breaks right off the bat one of the key benefits of the cloud, around reducing cost and increasing time to value.
- Unnecessary Complexity – CMP products have evolved into offering a lot of functionality that ends up as shelf-ware. These products are over-designed and overly complex from an install, provisioning, licensing and Day-2 operations standpoint.
- Lack of Developer Engagement – Being dissatisfied with the provisioning processes or unresponsiveness of legacy IT, developers often turn to the public cloud (“Shadow IT”) to accelerate their work so they can move faster. CMP products have done a poor job catering to developers as a critical audience – mainly around the UI, API support, governance controls and lack of support for IaaS/PaaS experience.
- Lack of Management for Cloud-Native Workloads and Public clouds – The majority of proprietary CMPs are mostly tailored to manage virtualisation providers such as VMware. A key challenge that needs to be solved in commercial CMPs is the built-in seamless support for Cloud-Native workloads and public cloud management capabilities.
Trends such as Shadow IT, cloud migration and developing cloud-native applications have created the need for greater environment visibility and control over all types of infrastructure. This means that cloud-native/containers workloads need to be managed in the same way as VMs on private clouds.
The market in multicloud and hybrid cloud management is still evolving, and many of the vendors come from the virtualisation management space. While this seems a logical evolution, the challenge is that the new cloud-native workloads do not operate in the same way as VMs, and we see established vendors struggling to balance this new world with VM-centric infrastructure. The difference between these two paradigms needs to be abstracted away from both developers and infrastructure teams.
So what are the key lessons we’ve learned over the years to enable enterprises to effectively manage their complex, hybrid environments?
The key capabilities required for effective cloud management solutions:
- True Hybrid Cloud Brokerage Delivered as a SaaS: The ability to connect to and abstract connectivity to any of the underlying platforms – VMs, Containers, private and public clouds – using a common open API that both developers and Ops communicate with. Hence, even if the underlying clouds change, the applications and their backend processes (provisioning, config, performance, etc.) do not need to change. The key difference between traditional CMP platforms and the new evolution in this space, is that the next generation CMP will deliver this brokerage capability as a managed service.
- Developer Self-Service via SaaS: The ability to provide both a self-service portal & API that enable developers to provision infrastructure for their applications using the Cloud Broker. The management of this integration should be automatically delivered as a service to IT Ops, and not require any additional management overhead or custom development on their end in order to support this, at scale.
- Application Catalogue: The ability to provision VMs has been around for years. Next generation CMPs need to provide the ability for users to provision complete application stacks from an application catalogue or from image templates. The ability to specify and automate the placement and provisioning of new instances based on business and security policies is a key capability that must be provided.
- Performance Metrics: The ability to provide visibility and detailed metrics on global cloud regions – across servers, network, and storage, as well as granular metrics on usage and performance across individual VMs & Containers. A key use of these metrics should be around improving datacentre efficiency and utilisation (which, obviously translates to costs), on both on-prem and public cloud resources.
- Enterprise Integration: Supporting seamless and open API-based integration points with existing ITSM tools, systems management platforms, service catalogues, configuration management and all the public clouds and key stacks/technologies.
Getting hybrid clouds right: key design principles
Next-gen hybrid cloud management solutions need to be designed from the ground up with these key principles in mind:
1. Cloud Management Should Be Simple
You want to simplify and accelerating time to value for complex hybrid cloud management and operations tasks.
- Cloud management should not turn into another ‘ERP’ project for the enterprise – this approach is what makes digital transformation initiatives fail so often. The cloud is a journey – whatever your infrastructure choices, CMPs should be quick to set up and allow you to deploy applications into the underlying cloud with a push of a button, instantly.
- Resist unnecessary complexity – it only lengthens the implementation and learning curve, slows adoption and increases your investment before you see results. Many CMPs bundle in automation, billing etc., to replace other tools that are already in use in the organisation. By simplifying the core essential components of cloud management, organisations can stand up a cloud with support for Day 2 operations within a day – seeing value faster, without having to rip and replace additional services that are on the periphery of the core competency they needed to support.
- To enable DevOps practices, CMP platforms should support both legacy applications and processes WHILE enabling IT to easily experiment and adopt new technologies, and to take advantage of modern architectures and delivery patterns (bi-modal IT.) This is critical so that (legacy) CMPs don’t end up getting IT stuck in the “middle-ages”, but allow them a path to DevOps-ify and modernising their applications and infrastructure.
- Not all applications are cloud-optimised. While Greenfield applications will use Kubernetes, Serverless, etc. a portion of your business will still run on VMs or bare metal for a while. Legacy apps may require refactoring but should not require re-platforming. The CMP should support a unified experience across all of these types of applications: VMs, Containers, Serverless, and whatever new technology comes next.
2. Cloud Management Should Be Delivered as SaaS
The most difficult thing about running a CMP is the setup, installation, configuration, and day 2 operations. I’ve seen enterprises take months (sometimes years!) of professional services and consulting in order to get a CMP running.
Public clouds have already set the bar for ease of use. CMPs should “just work” out of the box in a similar fashion – in terms of developer experience, easy setup, easy integrations, and automated operations. The CMP and the infrastructure should be installed, managed, and monitored using a SaaS-based delivery model. No more manual work, heavy lifting on the Operations side, or taxing management overhead.
3. Build on Open Source Frameworks
Today, Open Source (OSS) frameworks provide the core capabilities that matter for any modern development, IT operations or DevOps processes. Building on OSS as a fundamental principle ensures your solution can:
- Incorporate the latest and greatest innovation in the space, as it continues to evolve, and benefit from the vibrant open source community
- Be future-proof for whatever new technology comes next
- Allow you to avoid lock-in, and be portable and interoperable across any environment or provider
- Be easily extensible and is flexible to support new integrations, services and specific use cases
- Benefit from the open source economics and savings vs. high licensing fees of proprietary solutions.
4. Unified Experience
Whatever the underlying cloud or infrastructure provider are, CMPs should provide a unified experience across four areas:
- A single view of all types of infrastructure: servers, VMs, Containers, storage and network – across all VM providers and private/public clouds, all the cloud regions and the tenants across these regions.
- A single way for Site Reliability Engineers (SREs) to administer hybrid infrastructure across critical areas such as security & identity management
- Unified & open API for both developers and operations to perform lifecycle management and easy integrations with point tools or management processes
- Continuous monitoring across all of the different cloud regions and environments
Cloud Management has become unnecessarily complicated, largely as a result of legacy VM management solutions being extended or retrofitted to support public clouds or containers. A badly conceived CMP can significantly drain enterprise resources, and with so many failed implementations, no wonder traditional cloud management solutions have such a bad rep.
There is a way to get hybrid clouds and multi-clouds right. Enterprises need to be able to instantly turn any infrastructure into a cloud, and benefit from a unified cloud experience on ANY infrastructure, for ANY application — to be able to consistently manage VMs, Kubernetes and Serverless – running on-premises or in the public cloud.
Sirish Raghuram, CEO and co-founder, Platform9
Image source: Shutterstock/Nattapol Sritongcom