Skip to main content

Monitoring multi-cloud environments

(Image credit: Image source: Shutterstock/bluebay)

In today’s fast-growth, faster-results market, customers expect applications to be consistently available and up-to-date — which is often easier said than done. Meeting that demand often involves migrating to the cloud, which offers heightened scalability and flexibility, allowing engineering teams to innovate more quickly and produce the delightful user experiences customers are not only looking for, but continue to demand.

Although the cloud offers a wide array of benefits, migrating from legacy infrastructure to the cloud brings its fair share of challenges: siloed data sources, fragmented toolsets, and increased security threats.

With the rise of multi-cloud, microservices, IoT devices, and containers (not to mention the fact that applications are often made up of a combination of legacy and new technologies), it’s harder than ever to have complete visibility into organisational infrastructure to make sure everything’s running as expected. When failures occur, you need to be able to diagnose and fix them, auto-remediating common, repetitive issues, while limiting downtime. Downtime is not only costly — Gartner places the average cost of unplanned downtime at $5,600 per minute — it erodes customer trust.

The bottom line? Effectively monitoring multi-generational, multi-cloud infrastructure and applications — and mitigating unexpected failures — is critical to immediate and future success.

In this post, we’ll look at the challenges of monitoring multi-generational environments, and how to choose approaches and tools that facilitate a successful cloud migration.

Challenges of monitoring multi-generational, multi-cloud environments

Multi-generational, multi-cloud environments are the new norm. But spreading services across multiple environments fractures the ability to have a holistic view of organisational infrastructure — making it exponentially harder to connect disparate data types and monitor the health of systems. 

You can define monitoring as the action of observing and checking the behaviour and outputs of a system and its components over time. In short, you need to know about a problem before your users do — and you need to monitor cloud instances just as proactively as you monitor your private datacentre.

Monitoring multi-generational, multi-cloud environments introduces unique challenges such as data security. Security threats across all environments are a major concern today; the cloud's vast attack surface and multiple endpoints make it particularly challenging. According to research from Analyst Lawrence Hecht, 87 per cent of organisations are worried that poor visibility inside public clouds hinders their ability to manage security threats. A second challenge is disparate toolsets. Keeping tabs on old and new infrastructure often requires different tools. Legacy monitoring tools don’t provide the scalability, reliability or customisation that the cloud necessitates. Siloed, cloud-supplied monitoring is another challenge. The monitoring tools provided by a cloud vendor don’t extend beyond that environment, which means a loss of complete visibility and will likely cause a run up against governance issues across multiple cloud providers. Lastly, gaps in experience and expertise. The massive growth of cloud drives demand for cloud skills that many organisations lack. In fact, Information Age Editor Nick Ismail, shares that 64 per cent of IT decision-makers in the UK say their organisation is losing out on revenue because it doesn’t have the required cloud expertise, which extends to monitoring tools as well.

Overall, managing these challenges isn’t easy, but it becomes easier with a unified monitoring environment. When an organisation has a centralised view of both on-premises and cloud environments, they can embrace the value of migrating to the cloud — without suffering downtime and management headaches.

Choosing visibility tools that complement a successful cloud migration

When planning for cloud migration, it’s vital to look for tools that are built to handle both public cloud (Amazon Web Services (AWS), Google Cloud Computing Services (GCP), Microsoft Azure) and private cloud (OpenStack, VMware, Xen), as well as container infrastructures and bare metal. Here are some key capabilities to look for:

Go beyond (just) alerts: Of course, wanting to be notified when something goes wrong is a given, but that’s just the first step. alerts need to be actionable and if they’re not, they should be automated (i.e., auto-remediation). Save alerts for tasks that can’t be automated, which allows operational teams to focus on their tasks at hand.

Automate, automate, automate: Per the above point, automating tasks means faster resolution while protecting operators from alert fatigue. Choose a monitoring tool that integrates with an intelligent IT incident management platform to help IT operations teams detect, investigate, and resolve incidents quickly.

Secure transport: Monitoring tools should support and use standard cryptography for communication. A great start is to partner with a company that’s implementing the latest and greatest in terms of transport layer security (TLS), including staying up to date with industry best practices around implementation and management. Just as migrating to the cloud comes with its own implications in terms of security — though confidence is growing, 37 per cent of respondents to Nominet’s survey of 300 CIOs, CISOs, and CTOs say the risk of a breach is higher in the cloud —  it’s important to find a monitoring tool that helps (rather than hinders), when it comes to securing infrastructure needs.

Flexibility: Monitoring should be agile and able to collect data in a variety of ways. Use a flexible, agent-based monitoring solution for customised workflows that are tailored to a particular infrastructure system. Bonus tip: a flexible solution that allows the collection of disparate data types allows for the recycling of work done on existing, legacy tooling.

Essentially, it’s important to find a monitoring solution that empowers users to monitor their entire infrastructure (the old as well as the new), and that integrates well with many other tools for the monitoring ecosystem — such as tools for metric storage, alerting, and data visualisation. With the right tool sets in place, users can streamline their CI/CD pipeline, and accelerate responses to both customer needs and business changes.

Migrating to the cloud brings risks and opportunities that require integrated, multi-cloud strategies. With cloud growth on the rise, it’s critical to adopt modern tools that keep pace with that growth, giving total visibility across the entire infrastructure and the power to automate incident management.

Sean Porter, co-founder and CTO, Sensu