Open Source Comes To Cloud Computing : A closer look at MRG

This 5 page special report explains how to:

1. Enable your business to move to a cloud-based model of computing wherever and whenever you required it.

2. Run any workload, from sub-second calculations to server applications to High Performance Computing (HPC) workloads on any computing resource available, from local grids to remote grids to idle desktops to virtual machines to clouds.

3. Dynamically manage changing priorities, loads, workloads and the majority of computing environments

Read this and have access to the case for data services whitepaper (16 pages) by registering here.

Imagine a product that could

1. Enable your business to move to a cloud-based model of computing wherever and whenever you required it.

2. Run any workload, from sub-second calculations to server applications to High Performance Computing (HPC) workloads on any computing resource available, from local grids to remote grids to idle desktops to virtual machines to clouds.

3. Dynamically manage changing priorities, loads, workloads and the majority of computing environments

Just think of the issues that such a product could resolve such as being able to address data center deployment and scaling issues, power issues, using virtualisation with seasonal workloads, capitalizing unused capacity, decreasing costs, increasing flexibility and… a lot more…..

Over the last year Red Hat has released and enhanced several products that require distributed computing resources. In addition Red Hat which has traditionally been known for their work in the Linux environment now has several product offerings that work equally well in other environments (e.g. Windows, Linux and various flavours of UNIX). One such product is their MRG product that enables companies to robustly run and manage applications that use distributed computing.

Applications that are showing quantifiable benefits from using MRG include everything from a basic webfarm to automated trading systems using sophisticated what-if algorithms requiring high performance computing grids, bespoke internal based applications and defence command and control systems.

MRG (pronounced ‘merge’) stands for Messaging Realtime Grid and has a number of powerful features that are setting standards but with Open Source value. Each of the components of MRG has several strengths but when combined show the true strength of MRG. Let’s examine the Grid technology first.

MRG Grid - Next Generation IT Infrastructure

As MRG is aimed at the distributed computing/middleware environment it naturally works across multiple workstations and servers. As a result of this access MRG also has the ability to allow you to take advantage of unused processing power within your organisation via the Grid component. In other words, your applications need never be process starved as the MRG Grid component can use the combined processing power from all inactive workstations and servers.

This additional processing power is useful for many high end applications. E.g. a website with multiple webservers, automated trading systems using sophisticated what-if algorithms requiring high performance computing grids and bespoke internal based applications. (Note: MRG Grid has been shown to excel at traditional HPC workloads due to the extra processing capacity)

In addition, MRG Grid can also integrate with virtualisation to enable companies to run any workload across any computing resource, regardless of the underlying physical platform. There are two major benefits:

1. Finance for extra computing resources can be saved as you can now take advantage of every spare processing capacity you have for your distributed applications.

2. Finance for extra computing resources can be saved as an application that requires extra process power, can use the MRG Grid functionality to locate idle systems and be allocated the necessary compute resources to enable that application to continue processing.

The Grid functionality is further enhanced because it can give the application access to memory and hard disk resources as well as processing power. Figure 1 shows some management statistics of jobs being processed by MRG Grid

Grids in the Clouds

But wait the story gets better. Many companies do not have the spare processing power when they need it. Ie in an over-worked environment, you could only use the Grid functionality when most of your employees are away (ie bank holidays, weekends and evenings or of course during the major holidays if you are a school).

So the Grid functionality also has the ability to take advantage of the cloud computing facility offered by companies such as Amazon where you can buy time on their systems. As time is expensive, the Grid functionality of MRG can automatically connect and use the cloud power when needed and automatically tear down the connections when the application has finished its processing.

In addition, the Grid functionality also lets you create and offer your own cloud should you have spare processing functionality. By creating their own cloud, companies can then offer to branch offices and subsidiaries extra compute power or even lease it to other companies in the same way companies such as Amazon and Google do.

MRG Realtime

Realtime computing is important to industries using applications that care about reliability and time or latency-sensitive transactions. The true benefit of realtime comes from providing deterministic performance. i.e. Applications will always respond or perform in the same amount of time (deterministic). For example, an automated trading system may have a very fast average/mean response time.

But, if every few minutes, one trade takes several seconds rather than several milliseconds, that performance discrepancy can mean lost money. Examples of applications that require deterministic functionality include:

- Automated traded systems that compete against each other for stock; normally the fastest system will be able to obtain the stock. If the speed is anywhere less than the normal response time, the financial institution is more than likely to lose the bid.

- Command and control systems for defence or facilities such as water or nuclear power, where any abnormal delay in response could result in the loss of life.

- Travel systems that require a guaranteed response time or else the travel agency could lose the reservation. Eg if a response time of 4 seconds is required and the reservation takes 7 seconds, then there will be loss of business.

- Tracking satellite data and keeping data synchronised with the other satellites and client systems.

The Realtime component of MRG has several benefits in the above areas:

- It guarantees a response time such as in the travel system reservation software and in command and control systems.

- It can prioritise applications to the point where even if the system is overloaded and prone to hanging, the application itself will still process.

Speaking with Red Hat’s, Bryan Che, Product Manager of the MRG offering;

“The Realtime component uses an updated Linux kernel which means that MRG is essentially working as part of the operating system and so allows Red Hat Enterprise Linux to work in realtime without latency and to take advantage of deterministic features.”

Messaging Functionality

Messaging functionality is the heart of all distributed computing applications. Applications that use messaging technology include: automated trading systems, billing information and reservation systems and defence communication systems to send data to send data to one another.

- In a Travel Reservation software, messaging would be used by the travel agents systems or users’ systems to communicate a reservation and then to automatically return a pop up confirmation message and an email confirmation.

- In an automatic trading application, messaging would be used by the systems to communicate bids and return whether the bids have been successful.

“We also have clustering and federation technologies directly in the messaging broker to enable robust deployments of messaging across disperse environments,” says Bryan Che

Messaging functionality is also not new, many of the large software vendors such as IBM’s Websphere MQ series and Microsoft’s MSMQ are aimed at delivering reliable messaging for distributed computing platforms. However the problem is that many of these messaging based systems cannot communicate with one another. For example, if you purchase a system from a vendor it won’t communicate with another vendor’s application due to the use of proprietary messaging protocols.

Two components make Red Hat’s MRG Messaging offering unique:

1. Its integration with and optimization for Linux at the kernel enables performance of up to 100 times better than other messaging products

2. Its support of the Advanced Message Queuing Protocol (AMQP).

Messaging with Realtime and Grid Integration

Messaging and Grid integration benefits include:

- Data distribution across the grid

- Leveraging messaging for the management and monitoring of all of MRG, including the grid

- Low-latency scheduling of jobs onto the grid. By leveraging Messaging and Realtime, MRG can schedule jobs in the millisecond range—much faster than other grid platforms. This opens up new classes of work for grids to compute

Advanced Message Queueing Protocol (AMQP)

One major advantage that Red Hat MRG has over other messaging software is its default support of AMQP. AMQP is a user driven messaging protocol standard designed to allow different distributed applications to natively talk with each other.

For example Vendor A’s trading application can use this protocol to talk with Vendor B’s trading application. As AMQP has risen in popularity over the last few years, companies such as Cisco have also built devices to allow you to bridge between AMQP and other messaging.

If you use AMQP as the messaging protocol for your distributed applications you won’t have to budget for developers to work on methods to integrate your distributed applications.

In other words, using AMQP will save on the tremendous complexity and administrative overhead of having to integrate communication between different applications whilst enabling optimal architectures and performance. A protocol such as AMQP can enable future applications and architectures to take advantage of the full power of distributed deployments as it eliminates the headaches of integration.

Red Hat MRG or Bust?

Due to the limited size of this report, we can only give you a brief overview of the benefits of MRG and it would be worth your time to investigate further benefits by reading the white papers on the MRG offering.

Over the last decade Red Hat has shown that the Open source model for computing is a viable and sustainable business model realising hundreds of thousands of dollars in savings for companies. From this standpoint alone, apart from the benefits already covered in this article the product is worth an in-depth look.

If you would like more technical information and white papers visit http://www.redhat.com/f/pdf/MRG_WhitePaper.pdf