AWS re:Invent, arguably the biggest and most important event in the cloud computing calendar, has just wrapped up for 2017. Held in Las Vegas for over 40,000 people re:Invent is an impressive piece of event management at scale, bringing together customers, partners and Amazon employees for 5 days of keynotes, workshops and networking.
I attended re:Invent 2016, and gave myself RSI trying to live-tweet the keynotes, barely keeping up with the pace of new service and feature announcements. To prevent the same happening to you this year, I’ve pulled together some of the key announcements with an overview of what this means for the industry. Here are my top three takeaways:
Inter-region VPC peering
Amazon Web Services were still announcing new features and services all the way up to the conference this week. Some of these hinted at a wider plan. At the beginning of November, it was announced that it would now be possible for users of Direct Connect to route traffic to almost every AWS region on the planet from a single Direct Connect circuit.
From that announcement I predicted that we’d see other region restrictions lifted, and AWS came good on that expectation this week when they announced that it would now be possible to peer two VPCs across regions.
VPC peering is a mechanism by which two separate private clouds can be linked together so that they can pass traffic between them. We use it to link staging and production networks to a central shared management network. Other use cases include allowing vendors to join their networks with clients’, to allow a private exchange of traffic between them.
Up until now, when working with customers who require a presence in multiple regions, we have had to build and configure VPN networking infrastructure to support it, infrastructure which then needs monitoring patching and so forth.
With inter-region VPC peering, all that goes away: we’ll be able just to configure a relationship between two VPCs in different regions, and Amazon will take care of the networking for us, handling both security and availability themselves, and letting us spend time on the more differentiated aspects of our solution.
There were no doubt glum faces all round at security vendors’ costly AWS re:Invent exhibition stands this week, as Amazon announced a new threat detection service for their public cloud offering.
AWS GuardDuty monitors traffic flow and API logs for your accounts, letting you establish a baseline for “normal” behaviour on your infrastructure, and then watch for security anomalies. These are reported with a severity rating, and remediation for certain types of events can be automated using existing AWS tools.
Last year, AWS announced Shield, a managed DDoS protection service made available for free to all AWS customers, with CTO Werner Vogels acknowledging that this is something Amazon should have provided a long time ago. AWS employees often say that security is job zero, and that if they don’t get security right, then there’s no point doing anything else. It’s no surprise therefore that we’re seeing more security focused product releases this year.
Intrusion detection solutions gather network traffic and log data, looking for the fingerprints of known attacks, and using machine learning techniques to identify new anomalies. AWS have increased their focus on machine learning in recent years, and so it was to be expected that we’d see this expertise applied to the security landscape. These systems work best with a large corpus of live data to analyse and with AWS so dominant in the cloud solutions space, they’ll have no shortage of that.
AWS GuardDuty is a welcome announcement for both customers and systems integrators. The incumbent vendors in this space offer clumsy solutions based on previous generation on-premise hardware appliances. These had the luxury of connecting to a network tap port where they’d be able to passively observe and report on traffic as it went by, without impacting on network or host performance. However, since network taps aren’t available in the cloud, vendors have had to resort to host-based agents that capture and forward packets to virtual appliances, affecting host performance and bandwidth bills. AWS GuardDuty lives in the fabric of the cloud itself, and other vendors will find it hard to compete with this level of access.
It’s likely that over time, existing security vendors will pivot their business model further towards becoming AWS partners, adding value to Amazon services rather than providing their own - a move we’ve seen from traditional hosting providers such as Claranet and Rackspace over the years.
EKS and Fargate
Much anticipated by industry experts, Amazon Web Services announced their managed Kubernetes offering, EKS (Amazon Elastic Container Service for Kubernetes).
In the last three years, Kubernetes has become the de facto industry standard for container orchestration, a major industry hot topic, and an important consideration in the running of microservices architectures. This open source project was originated by engineers at Google, who based their solution on experience operating production workloads inside their employer.
Google themselves have offered a hosted Kubernetes solution for some time as part of their public cloud offering, with Microsoft adding support to Azure earlier this year. Whilst this announcement shows AWS playing catch-up against the other providers, research by the Cloud Native Computing Foundation (CNCF) shows that 63% of Kubernetes workloads were already deployed to the AWS cloud, by people who were prepared to build and operate the orchestration software themselves.
EKS will now make this much easier, with Amazon taking care of the Kubernetes master cluster as a service; keeping it available, patched and appropriately scaled. Like its parent service, ECS, this is a “bring your own node” solution: users will need to provide, manage and scale their own running cluster of worker instances. EKS will take care of scheduling workloads onto them, and provide integration with other Amazon services such as those provided for identity management and load balancing. It’s notable that EKS the upstream open source version of Kubernetes as-is: compatibility with other vendors’ hosted offerings, and with open source add-ons will be maintained.
Alongside EKS, CEO Andy Jassy announced another new container service: AWS Fargate. Potentially much more game-changing, Fargate users won’t need to provide their own worker fleet - these too will be managed entirely by AWS, elevating the container as a first class primitive on the Amazon platform, on a par with EC2 instances. Initially supporting just ECS, Fargate will offer support for Kubernetes via EKS during 2018.
It’s an exciting time for AWS users - with the ability to adopt the latest in container scheduling technology, but without the challenges of operating the ecosystem it requires, companies’ tech teams can now spend more of their valuable time on generating business value.
Jon Topper, CTO of The Scale Factory
Image Credit: Gil C / Shutterstock