The “new” IT landscape: The rise of software-defined

The IT landscape is dramatically and rapidly changing. Business needs are driving and shaping it, while IT is navigating the trade-offs between Performance, Agility, and Cost.

Application design, development, and deployment have evolved, but IT is struggling to keep up. The building blocks of the software-defined data centre (SDDC) are there, but Operations teams are constrained by brittle management tools.

New Stack, Same Drivers

In the late 1990s and early 2000s we saw the rise of virtualisation. Fast forward 20 years and we are beginning to see the popularisation of platform-as-a-service (PaaS) and an emerging Containers-as-a-Service (CaaS) ecosystem. What brought us here? And where are we headed?

The Dawn of Virtualisation in the Open System

IBM is accredited with inventing virtualisation back in the 1960s. But it wasn’t until nearly 40 years later that VMware moved it to open systems. The masses were ready: virtualisation simplified infrastructure management by abstracting hardware into software, realising significant operational and capital cost savings. Still, decades later, significant unused hardware remains. A 2015 report from Anthesis Group, working in partnership with Stanford University and TSO Logic, estimates that $30 billion in IT assets sit idle, largely due to management silos.

Rise of IaaS

Businesses became increasingly dependent on applications to serve their day-to-day activities, a trend which only continues to rise today. Software was indeed eating the world. In many cases applications became the lifeblood of the business. Developers became kings and infrastructure teams had the mandate to do whatever they could to empower these developers to create better applications, faster. “The cloud” allowed developers to operate with agility; self-service portals were there for their deployment, anytime, anywhere.

Popularisation of Containers

By 2014 developers were ready for instant gratification that actually took an instant and would actually run anywhere. Docker’s commercialisation of Linux containers began a new wave. Deployable in seconds, able to run anywhere, and on a fraction of the resources required by a VM, containers were an easy win for developer and infrastructure teams alike.

Cloud OS Enters the Scene

Infrastructure teams are realising the major challenge that comes with easy deployment: sprawl on a scale as yet unseen. Call it “Cloud OS”, call it “Container Orchestration”, it all comes down to the fact that containers need to be managed. Kubernetes, Docker Swarm, Mesos and the like, were created to manage the placement and orchestration of containers.

Popularisation of PaaS

Meanwhile, the central role that applications play in business has only grown. Applications differentiate and drive today’s organisations. Developers must continuously create, test, and deploy at greater and greater speed. PaaS is allowing developers to focus only on their applications, paying no mind to the infrastructure that supports it.

Emerging CaaS

At the same time, the Containers and Cloud OS spaces are merging, becoming what people are calling Containers as a Service (CaaS). The new stack can be viewed as PaaS on top of CaaS on top of IaaS—each ‘as a Service’ creating isolated platforms and tooling ecosystems around them.

Taking stock

Where does this new landscape leave us? Applications remain paramount, regardless of the changes in the underlying infrastructure. Yet we’ve decoupled dependencies with layers of abstraction. We have created degrees of separation between the applications and the physical infrastructure they run on. We’ve cut Costs and increased Agility, but what about Performance?

The fundamentals haven’t changed. At the end of the day, applications need CPU, memory, IO and network to perform. This decoupling enabled agility and accelerated application development, but is IT keeping up?

Apps Are Eating The World

Apps are eating the world. These applications come in all shapes and sizes: different languages, different architectures, and different mandates. They are broken into micro-service architectures, distributed across VMs and containers, hosted on-premises, burst to the cloud, spun up and spun down across multiple clouds, deployed and migrated across everything.

Each layer of abstraction may provide better isolation and increase agility, but how do we guarantee application performance? The app needs compute, storage and network. It consumes these resources from the PaaS. The PaaS in turn consumes from the container-as-a-service (CaaS). The CaaS then consumes from the infrastructure-as-a-service (IaaS). Lastly, the IaaS consumes from the physical hardware. Each layer is a provider to the layer above it and a consumer of the layer below. Each layer assumes the layer below will be able to provide the resources to satisfy the demand. Well, we can only hope. Somehow some IT person is expected to make sure that the applications and every layer of the stack below them get these resources when they need them.

One Abstraction To [Unify] Them All

The only way to guarantee application performance is to control the entire stack through a common abstraction that ties it all together, one that recognises the interdependencies and manages the trade-offs that exist in applications in the cloud right down to the physical compute, network, and storage that feed them.

With the right abstraction it doesn’t and shouldn’t matter how the stack evolves. How we build, test, and deploy applications might change, but what they consume, and the fact that performance is IT’s number one goal, does not.

Cloud environments come down to 3 basic facts: applications consume, infrastructure supplies, and every interaction that occurs between them, across every layer, is an exchange of resources. Success means managing this system of supply chains so that resources are bought and sold between service entities at the best possible price. This will create what is essentially a fully functioning data centre economy where the organisation is ensured of the best performance for the best cost.

What’s Next? And Does It Matter?

I don’t know. And it doesn’t matter. We will probably see more and different layers of abstraction. We will definitely see more applications, along with technologies that help us build, deliver, and update those applications faster. But without control, without a unifying abstraction that manages the very real trade-offs that exist in cloud computing, the risks to performance will only increase.

Ultimately, the data centre stack’s evolution is being driven by business demands; and the need to have the right applications for the business with the right performance. The businesses that guarantee application performance will be the ones that dominate their industries. The only question then is how precisely they’ll do that.

Shmuel Kliger, founder and president of VMTurbo

Image source: Shutterstock/Ollyy