Skip to main content

The state of service mesh: How to right-size a deployment based on your microservices maturity

(Image credit: Image Credit: B-lay)

According to Gartner, almost half of CEOs globally are facing pressure from their board to advance digital transformation projects. Digital transformation is the shift to ways of doing business digitally, whether across the supply chain and inventory tracking or offering apps and online retail capabilities to customers.

Many enterprise applications are still not suited to this and were built for an offline era, rather than our online, mobile-first world. Consequently, many of the applications across the enterprise today need investment – but according to Gartner, by 2023, most applications will still have received insufficient investment to modernise them effectively.

This is a difficult situation; many legacy applications have become business-critical but have slowly become more difficult to manage and maintain in line with today’s needs, without completely re-architecting. As a result, C-level executives understand that they not only need to build new, modern applications in the cloud, but also modernise existing applications by retrofitting them with technologies like Kubernetes, APIs, and microservices.

But why are monolithic applications really causing such trouble, how did we reach this point, and what’s the answer?

Looking back

Ten years ago, service-oriented architecture (SOA) was very popular, because it gave developers the flexibility to write a small, specific piece of functionality for a software package, then call on it and re-use as necessary.

These applications were scaled, secured, and made resilient using dedicated hardware, and we dreamed of using virtualisation to get better utilisation from our resources.

Then we started to see the move to cloud. Compute and storage networks became services that could be programmed and automated easily, allowing developers to buy resources with their credit cards.

At the same time, open source software grew from being an enthusiast’s technology to being accepted across the enterprise. It moved into operating systems, SQL databases, in-memory databases, relational databases, and more. After this, open source tools were developed for middleware, hypervisors, automation, network, storage solutions. Today, there are even machine learning tools like TensorFlow available as open source.

Consequently, IT teams have access to a huge variety of tools and the resources to get applications live faster than ever, many of which are available as open source. We’ve solved the problem of modernising applications, right? Not quite and here’s why.

The challenge of complexity and management

Unfortunately, choice and the sheer pace of change are the problems. Technology has enabled the dream of SOA to finally become reality, but we call it something different now: microservices. Microservices run small parcels of application logic, typically sitting within containers. This approach gives organisations the flexibility to develop, run and maintain each individual service in isolation, publishing updates to services each day or hour, rather than releasing a whole new application every few months (or years).

The technologies underpinning the microservices revolution are well known. It started when Docker made containers fashionable several years ago. Shortly thereafter we saw orchestration platforms, and Kubernetes in particular, come onto the stage. But with the birth of these technologies, we began to see a network challenge. Additional observability, reliability, and security mechanisms were needed to manage the traffic flowing in and out of these containers and orchestrated by Kubernetes. That’s when service mesh architectures entered the microservices picture.

Containerisation technologies were developed before there was an answer for how applications run on them and how they connect to each other. Service mesh technologies are aimed at solving this problem, offering an intelligent, distributed command-and-control platform for microservices.

The best-known service mesh today is Istio. It supervises all of the management functions of a distributed microservices architecture like load balancing, authentication and monitoring. This should make matters easier, but developing a mature service mesh platform is a very significant investment. We’re about two years into the Istio journey and many companies are struggling to deploy it in production.

One of the challenges is that very few organisations today have a technology environment as broad and complex as Google, IBM and Lyft, which co-developed Istio. When you’re looking at Istio, it’s a little like taking a huge 18-wheel truck for a half-mile drive to the post office; it might do the job, but there are better ways of going about it.

But applications made up of microservices do need dynamic support, especially if those applications are the lifeblood of your organisation. Istio is the right end state, but companies need evolutionary options that take them from basic Kubernetes implementations to full-blown Istio service meshes.

Where to start?

Most development teams today have a mixed estate of legacy applications. According to a July 2019 survey of 290 NGINX users, we found that the average application mix includes:

  • 58 per cent classified as legacy, or monolithic, applications
  • 23 per cent classified as purely microservices applications
  • 19 per cent classified as a hybrid, or a combination of a monolith connected to some microservices

In the 42 per cent of apps that are either hybrid or purely microservices, a common technology is Kubernetes. Kubernetes gives users an ingress resource object in its API, which defines how services are accessed from outside the cluster, and then an ingress controller makes those policies happen in practice. For many organisations, that’s enough technology to support their application mix.

However, other organisations have more complex applications, or a great percentage of purely microservices-based applications, that need greater monitoring, security or traffic management needs. For example, these organisations may need:

  • To secure traffic between services with mutual TLS (mTLS) on each microservice, using SPIFFE or an equivalent method.
  • To identify performance and reliability issues for each microservice by exporting Prometheus compliant metrics for analysis with tools such as Grafana.

Some of these changes mean further changes in turn, for example, adding modules for Prometheus, more complex certificate management, or additional tracing and observability tools. Thus, each small change brings a small but not insignificant burden to the developer and operations team.

Consequently, software architects are embracing patterns of organising and managing microservices that can gradually lead them to what is considered to be a ‘service mesh’, but without needing to do it ‘right now’. One approach that we have advocated is a three-step model, referred to as the Microservice Reference Architecture.

In the first step, the Proxy Model, organisations use load balancing software such as NGINX as a Kubernetes Ingress controller (KIC), handling inbound requests to microservices, as well as dynamically handling DNS routing as new services are created. The service registry also handles interservice communication within the cluster. Many organisations deploy a KIC as a standalone solution, or as part of a larger platform such as Red Hat OpenShift or Docker Enterprise Edition.

The second step, the Router Mesh Model represents a mid-point of complexity and functionality. It uses two instances; the first to handle overall traffic routing (the KIC mentioned above) and places another in the middle of each microservice cluster. This second instance manages the connections amongst the different microservices, balancing the load between them. This includes the use of the circuit breaker principle to monitor the health of instances, for example, throttling or splitting traffic as needed. This model allows SSL/TLS termination at the cluster level, but not at the individual service level.

Finally, the Service Mesh (originally coined as the Fabric Model) is the most sophisticated and complex model for microservices traffic management, also representing some of the greatest scope for growth and flexibility across the application environment. It places a proxy instance in each container that has a microservice instance within it. This is referred to as a sidecar proxy and handles application traffic in and out of each microservice.

As in the Router Mesh model, sidecar proxies in the Service Mesh model handle the load balancing, service discovery and ‘health checking’, but also allows the deployment of SSL/TLS security at the microservice level without compromising on performance. Furthermore, these secure connections can be established between microservices in a stable and persistent fashion, allowing organisations application teams to create ‘mini VPNs’ either between services themselves or services and external points.

We recommend these models be deployed in order, taking on increasing sophistication (and complexity) as needed. Rough rule of thumb? If you have tens of microservices, then a Kubernetes Ingress controller as part of the Proxy Model is sufficient. If you have hundreds of microservices, then add a second instance acting as an internal router as in the Router Mesh Model. Finally, if you’re running thousands of microservices, then consider a sidecar proxy instance for each microservice and deploy a full service mesh.

Where next?

Service mesh technology is still new, and there are all kinds of projects aimed at making stable, functional and easy-to-operate implementations. At some stage, this technology will reach the point where production-ready implementations are ready for mainstream enterprises.

But only a small fraction of enterprises need to jump to one of the early service mesh implementations. The immaturity and rapid pace of change in current service mesh implementations make the cost and risk of deploying them high. As the technology matures, the cost and risks will go down.

Do not let the lack of a stable, mature service mesh delay any initiatives you are considering today. We estimate 70 per cent or more of your needs can be met by a Kubernetes Ingress controller. Kubernetes and other orchestration platforms provide rich functionality, including many of the features you’d find in a service mesh. You will know when you reach the tipping point when you’re either running thousands of microservices or require extremely complex intra-service traffic management. Then consider bringing a service mesh implementation to bear. The key is to match the technology appetite with your microservices maturity, thus right-sizing with your digital business efforts.

Rob Whiteley, VP of Marketing, NGINX

Rob Whiteley
Rob is VP of marketing for software-defined storage startup Hedvig. Prior to joining Hedvig, Rob served as a research analyst for Forrester and VP of product marketing for Riverbed Technology.