Demonstrating scalability with HP CloudSystem and KVM virtualisation

The cloud by its very nature is supposed to address scalability. The main reasons to implement the cloud in your company revolve around the flexibility to deploy rapidly when required, and scale quickly both up and down as the need arises. With version 7.2 of is CloudSystem Matrix, HP has implemented technology that further improves the scalability it provides. The release has introduced Kernel-based virtual machines (KVM), which can be demonstrated to allow scaling up to thousands of virtual machines in CloudSystem Matrix and CloudSystem Enterprise configurations.

HP CloudSystem Matrix can implement infrastructure as a service (IaaS) and then provide an automation layer that allows customisation with pre-packaged content or workflows and automation developed by the customer. HP has performed real-world tests on this configuration to show how it can reach the maximum number of virtual machines supported by CloudSystem Matrix 7.2. For the demonstration, HP called upon 32 HP ProLiant BL460c G7 blades installed in two HP BladeSystem c7000 enclosures, all managed by CloudSystem Matrix and running Red Hat Linux Enterprise 6.3. A HP DL360 G7 was hosting the CloudSystem Matrix 7.2 CMS and a HP DL580 G7 the Matrix KVM controller. Each blade sported dual Xeon X5660 processors, providing 12 cores per blade, with 96GB of memory and 600GB of storage each.

The blades were configured as 32 KVM compute hosts and then pooled to provide 250 single servers, 100 groups of five servers, 75 groups of 10 servers, and 50 groups of 20 servers, for a total of 2,500 virtual machines running on 475 IO services, supplied by 432 cores, 3TB of memory, and 19.2TB of storage. These servers were all running a Red Hat Enterprise Linux 6.2 disk image, with each virtual machine receiving one virtual CPU, 1GB of memory, and a 6GB boot disk. Deployment uses the open source OpenStack. The disk image for the virtual machines was imported into CloudSystem Matrix KVM controller. This could then be deployed to each virtual machine with a few clicks. The configuration used three networks, with two carrying management information and the third providing user access to the virtual machines, but all three ran across a bonded pair of physical network connections, providing greater availability and increased bandwidth over single, dedicated connections.

The goal of the test was to provision the full 2,500 virtual machines, showing the scalability of the CloudSystem platform. Performance benchmarking was not a part of the test, which is why G7 rather than G8 HP hardware was used. There was also only a single server each for the CloudSystem Matrix 7.2 CMS and Matrix KVM controller, when normally there would be standby backups in place for both. The virtual machines were provisioned using scripts, rather than manually, creating services ten at a time, with a new provision added to the queue as each one completed, so there were always ten jobs going at once. During the test, all 475 IO requests were completed, with all 2,500 servers up and running and no failures. Using a command to list the virtual machines, it was confirmed that all 2,500 had been deployed successfully.

A second test attempted to show that 2,500 KVM virtual machines could be created using the HP Cloud Service Automation (CSA) platform as the service portal, and also that these virtual machines would automatically be registered with Server Automation (SA), so their operating systems and applications could be actively managed. This was a test of HP CloudSystem Matrix 7.2's abilities in an enterprise-level IaaS environment, using HP CSA 3.1 and HP SA 9.14. The difference with the previous test is that CSA can create services that include infrastructure, middleware, application software, monitoring, and other features, rather than just servers running in virtual machines. SA then automatically manages the installation, maintenance and compliance validation of operating systems and applications. Together with CloudSystem Matrix, they provide a CloudSystem Enterprise solution.

The second test was similar to the first, except that it installed the SA client alongside the operating system into each virtual machine, configured so that they self-register and can then be actively managed. The same set of 250 single servers, 100 five-server groups, 75 ten-server groups, and 50 groups of 20 servers were deployed. The 475 server IOs were created on the same combination of 32 HP ProLiant BL460c G7 blades installed in two HP BladeSystem c7000 enclosures, with a HP DL360 G7 running the CloudSystem Matrix 7.2 CMS and a HP DL580 G7 the Matrix KVM controller. But there were also an additional two HP DL360 G7 systems to run the CSA and SA Core respectively. The operating system used for each virtual machine was again Red Hat Linux Enterprise 6.2, but with the SA client pre-installed.

However, deployment involved an extra layer, rather than simply installing each server or server group 10 at a time. Instead, CSA Service Offerings were created that mapped to the CloudSystem Matrix server deployments. A CSA service design was constructed with the Matrix Infrastructure component, and then a series of these used to construct the service offerings. The deployment script then called these service offerings, rather than the server provisions themselves directly. This not only showed the scalability of HP CloudSystem with KVM, but that its virtual machines can also be deployed using complex service designs with many composite parts that are presented to users in an intuitive way through the portal.

Overall, these tests showed that the KVM ability of CloudSystem Matrix 7.2 doesn't just allow a theoretical maximum of 2,500 virtual machines, but that this level can be reached in the real world, too. With the CloudSystem Enterprise example, the SA agent automatically registered that all 2,500 virtual machines had been deployed without incident. The SA installation also meant that related sets of virtual machines could be managed as a set, so a single administrative activity could be initiated against hundreds of servers at once. Using federated CloudSystem Matrix CMS configurations, an even greater number of virtual machines than 2,500 could be deployed and managed, showing that HP CloudSystem can scale to meet the needs of large corporate cloud provisions.

To learn more, download the whitepaper below.