MRG 1.3 is Released

I'm pleased to announce the release of Red Hat Enterprise MRG 1.3.  MRG 1.3 is a significant update that features many new enhancements, including:

Messaging

MRG 1.3 offers updated clients with improved performance, new protocol version independent C++ and Python clients, Windows C++ client and additional QMF APIs.  Clustering enhancements include durable stores to provide optimized performance in clustered environments and stability improvements.

The update also enables MRG Messaging to be recognized as a supported messaging transport for JBoss Enterprise SOA Platform, and offers security improvements such as SASL support for Python, including Kerberos support.


Realtime
With MRG 1.3, Realtime will move to a 2.6.33-based kernel, and will include the new Performance Counter subsystem in the kernel and the new associated perf performance tool to enable greater performance capabilities for customers.  The update also features new hardware enablement and certifications, which will incorporate a MRG Realtime hardware certification program that is expected to be introduced shortly for 1.3.


Grid
MRG Grid is a key component of Red Hat's Cloud Foundations and also for HPC markets.  The release includes new user tools, Windows Execute Node support, enhanced workflow management, resource restriction capabilities, update configuration management and new admin tools.  With these updates, customers will gain the ability to scale to tens of thousands of devices, be able to provision virtual machines, centralize configuration and management and utilize cloud spill-over capabilities.


For additional information on MRG 1.3, see the official release announcement.

Condor Week 2010 Presentations

This week is Condor Week, and this is always a big event for the Condor project, from which we build MRG Grid.  Red Hat has a strong partnership with the University of Wisconsin, and we are happy to participate in this year's event with two presentations today.

In case you're not able to attend the sessions in person, here are links to them:

• Red Hat MRG by Matthew Farrellee
• Improving Condor Configuration Management with  Wallaby by Will Benton and Rob Rati

Open Source Energy Savings with Condor

Forbes has an article today on Open Source Energy Savings using Condor.  In addition to highlighting how Condor can help with saving energy in a data center, the article features Red Hat  and our work around Condor with Red Hat Enterprise MRG a couple times:

Two years ago Red Hat  worked out a partnership deal with the university to make Condor open source using the Apache Foundation license.

and

Paul Cormier, president of products and technologies at Red Hat, is working on combining a large collection of open source projects into a cloud provisioning and management suite. "The move to cloud computing as the next generation architecture has only been possible by integrating many of these open source projects, such as Condor," says Cormier. "It is only natural that the software for creating and managing these virtual environments come from the world of open source as well."

The energy savings policies you can implement with Condor are nice, but we see these features as truly beneficial for most enterprises as part of a cloud solution.  For example, we have added virtualization support to Condor, which can further improve power management in a cloud deployment.  Let's say you had two servers each running at 50% capacity with one virtual machine job on each of them.  You could have a Condor job which moved one of the virtual machine jobs to the other server, consolidating all work on one machine.   Then, you could have a Condor job turn off the other machine.

Red Hat Virtual Experience 2009

This Wednesday December 9, 2009, Red Hat will be hosting its Red Hat Virtual Experience event focused on cloud computing.  I have a session there in the afternoon regarding MRG:

Building and Leveraging Compute Clouds with Red Hat Enterprise MRG

This presentation will cover the current state of Red Hat's cloud efforts and provide technical information that details how to build a cloud infrastructure based on Red Hat's technologies and blueprints. The presentation will also provide an overview of the capabilities and features Red Hat Enterprise MRG and Red Hat Enterprise Linux virtualization jointly provide and how workload scheduling and virtualization can be used as an enterprise management platform.

There's also many other great sessions.  You can see the agenda here.

The Red Hat Virtual Experience 2009 is free to attend, and you can register here.

MRG Presentations and Videos from the Summit

I'm back from Chicago from another great Red Hat Summit.  I have uploaded my two presentations from the 2009 Red Hat Summit online:

• Red Hat Enterprise MRG Update
• Building and Leveraging Compute Clouds with Red Hat Enterprise MRG

As an additional treat, you can also view videos of one of my sessions, as well as our CTO Brian Stevens' keynote, which highlighted cloud computing with MRG and other technologies.  I can't link to the videos directly, but you can find them at

• http://www.redhat.com/promo/summit/2009/highlights/ 

Brian Stevens' keynote is on the first tab.  My MRG presentation is on the Summit Sessions tab.

Finally, you can watch a demo video of cloud computing with Red Hat Enterprise MRG.  In this video, I bridge and aggregate three different clouds together (a local render cloud, an internal cloud provisioned by Red Hat Enterprise Virtualization, and Amazon EC2) into one seamless render cloud for film rendering.  This video created quite a buzz at the Summit, so enjoy!

MRG In The Open Source Cloud Computing Forum

Red Hat is hosting an online event, the Open Source Cloud Computing Forum, on July 22, 2009.  Matt Farrellee from the MRG team will be presenting on how Condor, the Grid component in MRG, helps with building and adopting clouds during the 5th session at 11:30am.

The event is free also features lots of other great sessions.  Read more about it here and then register to attend!

Cloud, Utility, Grid and Other Mixed Metaphors

I've written a new blog post for press.redhat.com: Cloud, Utility, Grid and Other Mixed Metaphors.  It discusses three terms that people typically use together but are often unclear about how they fit: cloud computing, utility computing, and grid computing.  The post illustrates the differences between the concepts and also describes how they fit together by using Red Hat Enterprise MRG as an example of how to achieve all three. 

Check out the post at http://press.redhat.com/2009/06/18/cloud-utility-grid-and-other-mixed-metaphors/.

Corporate, Customer, and Academic Open Source Communities for Next Generation Software

I submitted a paper for the NSF's Cyberinfrastructure Software Sustainability & Reusability Workshop.  I discuss some of the innovative open source models we've used to develop Red Hat Enterprise MRG:

The NSF is looking both at models of how to build sustainable cyberinfrastructure software as well as specific software that will benefit its goals like providing communities with access to a “world class high performance computing (HPC) environment.” Red Hat Enterprise MRG, a high performance distributed computing platform which integrates Messaging, Realtime, and Grid capabilities, provides both an open source model of how academic researchers, customers, and corporations can collaborate as well as powerful software infrastructure which can help the NSF meet its next-generation cyberinfrastructure goals.

For example, I discuss the partnership we have with the University of Wisconsin around Condor for Grid scheduling:

This partnership between UW and Red Hat is adding many innovative capabilities to Condor and also expanding significantly Condor's reach from research environments to enterprises. For example, Red Hat has focused on adding many capabilities which enterprises require for deployment but which are not paramount for academia. These enhancements range from new graphical management tools to enterprise maintainability to concurrency limits on scarce resources like software licenses. Furthermore, Red Hat has also focused on advancing Condor towards utility and cloud models of computing by adding capabilities like libvirt virtualization support and Amazon EC2 integration. Many enterprises are now looking at MRG and Condor for building private clouds and moving to cloud computing.

The paper also discusses how we're feeding back developments from the enterprise into academia, how we've collaborated with customers and users around AMQP and messaging, how the technologies in MRG can help the NSF meet its software goals around sustainable HPC, and so on.

You can see the submission at the NSF's site for papers.

You can also directly download a pdf of the paper from the NSF's site: Corporate, Customer, and Academic Open Source Communities for Next Generation Software.

Upcoming Events: JBoss Virtual Experience and SCALE 2009

I'll be at a pair of events in the next couple weeks: the JBoss Virtual Experience and the Southern California Linux Expo (SCALE) 2009. 

At the JBoss Virtual Experience, I'll be manning a virtual booth to talk about Red Hat Enterprise MRG and Cloud Computing.  MRG Grid's support for virtualization and also integration with Amazon EC2 makes it a powerful tool for companies that either want to leverage the cloud or build their own cloud.  If you're attending the JBoss Virtual Experience, stop by the booth!

At SCALE, I'll be giving a talk, Introduction to Realtime Linux.    If you've wondered about Realtime Linux's benefits, performance characetristics, state, or just what it is, this will be a useful session to attend.  You can read about the rest of Red Hat and Fedora's presence at SCALE as well.

Red Hat Enterprise MRG 1.1 is Released

I'm pleased to announce that we released Red Hat Enterprise MRG 1.1 today.  This is a significant release that adds many new capabilities and performance enhancements.  It also introduces formal support around
the Grid component and entire MRG platform for the first time (Grid was Technology Preview in v1.0).  Some of the highlights of MRG 1.1 include:

• Messaging
• Native infiniband and RDMA driver for dramatically better
  latency
• Active/Active Clustering
• Enhanced security
• Queue semantics like Last Value Queue and Ring Queue
• Native .NET client
• Improved management tools
• Increased performance
• Realtime
• Improved performance, especially on boxes with higher CPU-counts
• Improved performance tools. For example, Tuna now has the ability to write tunings to an init script once you've found an optimal tuning for your system
• Grid
• New GUI management tools
• Low latency scheduling via MRG's messaging bus
  
• Amazon EC2 support for adding capacity on-the-fly in the cloud
• Concurrency limits on any scarce resource like software licenses or database handles
• Dynamic provisioning, which enables you to mark slots as partitionable and sub-divide them dynamically so that more than one job can occupy a slot at once

You can find out more about MRG at http://redhat.com/mrg.  Also, you can read the press announcement for MRG 1.1 at http://www.press.redhat.com/2009/02/04/red-hat-debuts-enterprise-mrg-11/.

Fedora Nightlife Article on lwn.net

There's a nice, detailed article about Fedora Nightlife on lwn.net: http://lwn.net/SubscriberLink/284887/b05744ca15f41a52/.

Fedora Nightlife and Energy Usage

Wow, lots of response to my blog post about Nightlife! It's great to see so much interest right at the start. There are a lot of questions, but many of the conversations around these should happen on the Fedora Nightlife mailing lists as they're not just for me to answer. Also, I've now created an initial Wiki page for Nightlife (https://fedoraproject.org/wiki/Nightlife), so a lot of information will ultimately go over there. I will, however, blog about some of the topics that have stirred more discussion.

I'll start with one of the questions that always seems to arise when people talk about harvesting idle computing capacity: energy usage. Specifically, isn't it a waste of energy to leave your computer running when you're not using it so that others can leverage it for distributed computation? This is a complex issue with a complicated answer: it sometimes is a waste of energy, but it doesn't have to be a waste and can even save energy in the long run.

Cycle harvesting is sometimes a waste of energy
Let's start with the obvious: harvesting idle computer capacity across many--perhaps millions--of computers can definitely waste energy. Computers that might otherwise have been turned off are now running at full power crunching data for projects that may not be useful. Furthermore, these computers won't all be fully utilized 100% of the time, so there will be many instances of computers running with nothing to do but waste energy. Yes, unfortunately, cycle harvesting can and often does waste energy.

Cycle harvesting doesn't have to be a waste of energy
Cycle harvesting can waste energy, but it doesn't have to do so. I hope that as we work on Nightlife, this will prove to be the case.

There are many worthwhile tasks which can only be accomplished by heavy computation. A lot of fundamental research today in biology or healthcare, for example, requires access to large computer grids. If you believe that this type of research is a worthy use of energy, then the issue of wasting energy becomes an engineering problem of utilization and efficiency. That is, there are certain tasks for which it is worthwhile to let others use your available computing power. As long as these tasks fully utilize your computer when it is idle and they do so in the most efficient manner, then they aren't really wasting energy. More concretely, what if finding a cure to cancer required a lot of computational modeling? Would it be a waste of energy if there was a good project devoted to harvesting idle capacity to find such a cure, and it did so in a way that fully utilized all the computers which were donating capacity in an efficient manner?

If the keys to preventing energy waste in cycle harvesting are utilization and efficiency for worthwhile projects, then this is a problem we can address for Nightlife in a variety of ways. For example, the Condor scheduler is highly adept at maximizing resources efficiently. Given enough tasks or projects, we should be able to use all the resources available to Nightlife efficiently. The challenges will come from finding enough good projects and work for which people can donate their computing capacity. As long as we've got a good queue of work, we should be able to ensure that all the computers donating to Nightlife are doing something worthwhile and not just sitting around.

There are also many things that we can do at Fedora to increase further our ability to utilize resources efficiently. For example, we could explore waking computers to execute tasks that the owners of those computers deem worthwhile; otherwise, those computers will be in a suspended or low/no-power mode. At Fedora, we can drive the Linux operating system to be much more efficient in how it uses power while doing computations. And, Fedora's patron, Red Hat, has strong relationships with and influence over major hardware manufacturers and customers of grids. As commercial enterprises also look at how to save energy while doing their own grid computations, we have an opportunity to lead the way in developing and demonstrating the best techniques for doing this in an earth-friendly way.

Cycle Harvesting Can Save Energy
Much of the work that would run on Fedora Nightlife is going to be computed one way or another. A project like Nightlife, however, can not only help speed these computations by providing additional processing power, it can also help save total energy usage in the long run.

If you've ever visited a large data center, then you know that the energy usage of of the individual computers in that data center is only a fraction of the total energy the data center consumes. When you put tens of thousands of computers together in a single room, then many other energy hogs come into play. Foremost is cooling--large data centers require massive amounts of redundant air conditioning systems to prevent the computers from overheating as they process in close proximity to each other. There are also many other devices that draw power: the numerous network switches and routers connecting the computers, all the devices that monitor the data center's health and security, the redundant power supplies that keep the data center operating in the event of a power failure, and so on.

If a project were to distribute its computations over many individual, geographically dispersed computers and didn't need to build out a large data center for all its work, then it would no longer have to use as large a cooling center or provide as much backup power or do any of the other energy-consuming things that putting so many computers close together requires. Instead, by distributing its work over a number of individual machines through Nightlife, a project could cut down on its total energy required per computation.

Another way that Nightlife can provide energy benefits over a dedicated data center is by avoiding a concentrated usage of power in a single geographic location. I once visited a large Internet company in a power crisis because its host city's power grid could provide it with no additional electricity to grow--the company had totally maxed out the available electricity to it. Even if Nightlife didn't save total energy usage but increased the amount of energy required per calculation (which, as I've argued above, it doesn't have to do), this could still provide a better overall environmental impact. Rather than concentrating all its energy use in one place, a project could distribute and amortize its energy impact across a much larger area by leveraging Nightlife.

You don't have to participate, but you can contribute
Finally, maybe you fundamentally believe that there is nothing worthwhile for running on a large computer grid--no matter how noble the task--because of the energy required for running a grid. That's fine--you don't have to donate capacity to Nightlife. But, pragmatically, you have to agree that people are going to compute certain things one way or another. At Fedora, we have a tremendous opportunity to improve the power usage of grid computing in general. So, even if you don't donate idle capacity to Nightlife, please consider helping Fedora as a whole become the most energy-efficient platform for computation.

Introducing Fedora Nightlife

I've recently started a new project at Fedora called Nightlife. Here's the text of the e-mail I sent to the newly-created Fedora Nightlife mailing list:

Fedora Nightlife is a new project for creating a Fedora community grid. People will be able to donate idle capacity from their own computers to an open, general-purpose Fedora-run grid for processing socially beneficial work and scientific research that requires access to large amounts of computing power. Given the large number of Fedora users, I hope that we will eventually be able to build a community grid of over a million nodes at Fedora. This will be a great example of the power of the Fedora community, give people new and meaningful ways to contribute to Fedora, advance the development of large-scale grid software, and lead to real benefits for the world.

Fedora Nightlife will leverage the Condor project, which was (http://www.cs.wisc.edu/condor/) created and hosted by the University of Wisconsin Madison, for scheduling and harnessing donated computing power. Last year, Red Hat and the University of Wisconsin signed a strategic partnership around Condor. Part of this partnership entailed releasing Condor's source code under an OSI-approved open source license. As a result, we now have Condor packaged at Fedora, and upstream development continues to happen at the University of Wisconsin repository in an open manner.

Some of the immediate next steps for Fedora Nightlife are:
-create a Wiki page for this project
-get a Condor scheduler hosted at Fedora up and running

-work out what are the requirements for a project to be able to run on Fedora Nightlife (e.g. its software must be open source, it must be safe, it should have some kind of open policy around its results, etc)

We've already started working on getting a scheduler up and running. I should have a wiki setup relatively soon so that we can start mapping out more plans there.

Then, we can focus on growing the Nightlife community of projects and solicit Fedora users to donate capacity. Hopefully, enabling donation of compute power to Nightlife can eventually become a first-boot option for Fedora installs.

I welcome everyone to contribute to and participate in the Fedora Nightlife project!

If you'd like to subscribe to the Fedora Nightlife mailing list, you can do so at https://www.redhat.com/mailman/listinfo/fedora-nightlife-list

Fedora Nightlife is going to build on the work I do in my day job at Red Hat around Red Hat Enterprise MRG--it'll be based on the same technology we use for MRG's grid capabilities. This is great for me for a couple reasons: Fedora Nightlife will be a powerful and public example of the scale that's possible with Red Hat Enterprise MRG, and the work we do to drive Nightlife/Condor to the 1 million node count will directly benefit MRG. Also, it's great to be at a place like Red Hat where I can work on my product management job and do some good in the world at the same time.

Red Hat Enterprise MRG: Red Hat, Customer-Driven Innovation, and Open Source Leadership

Red Hat has shown that open source is one of the best ways to bring customer-driven innovation and leadership to the market. Today’s announcement of Red Hat Enterprise MRG provides a perfect example of this in many respects.

Spreading the Message of Open Source and Open Standards

Red Hat Enterprise MRG includes Red Hat’s implementation of AMQP-based ( Advanced Message Queuing Protocol) enterprise messaging. Both the MRG Messaging implementation and AMQP itself highlight Red Hat’s leadership and customer-driven innovation.

Red Hat is developing its AMQP messaging implementation in various open source projects and communities. One of the most notable aspects of these communities is that there are many messaging users from financial services and other industries contributing major pieces of code. These users are working to make sure that this messaging implementation will meet their specific needs when they ultimately consume it as customers. The results of this collaboration are noteworthy: MRG Messaging provides breakthrough features and performance and can reach durable messaging throughputs two orders of magnitude higher than other solutions.

Open source developers are not alone in recognizing the value of collaborating with others. The AMQP working group, of which Red Hat is a founding member, is developing the AMQP specification to be an open, interoperable standard for messaging. This particular working group is especially effective because its membership contains not only technology companies but also many end-users of messaging technology—including several investment banking giants. Of course, all contributions in the AMQP working group are valuable, no matter who provides them. But, AMQP is developing into a broadly accepted standard in many ways because there are so many end-users working to ensure that AMQP meets their own needs. Truly, this is customer-driven innovation.

Deterministic Success

In 2005, Red Hat began working on its realtime kernel technology in response to a request by the US Navy for the DDG 1000 Zumwalt Class Destroyer project. Red Hat engineer, Ingo Molnar, developed a realtime patch set which brought highly deterministic response times to the Linux kernel. However, Red Hat did not just release a product around this work. Instead, Red Hat has been working and continues to work with the Linux community to bring this realtime technology into the upstream Linux kernel. To date, Red Hat has incorporated about two-thirds of its realtime code base upstream and is working to push the rest of this code upstream. One notable recent achievement was the acceptance of Ingo’s Completely Fair Scheduler (CFS) into the mainline kernel this summer.

Why is Red Hat working so hard to push its realtime work into the mainline kernel? By having features implemented upstream, these capabilities “carry forward” into future versions of the kernel, so MRG Realtime has the product longevity that proprietary realtime extensions do not.

Trying to support extensions to Linux that are not accepted upstream is a losing battle. Red Hat recognized this long ago and thus pursued the long task of writing realtime extensions and pushing them upstream. Sure, this is hard work. But, at the end of the day, Red Hat will be in an optimal position to support this technology for the long term, since Red Hat wrote and led the work upstream.

Broad-Scale Innovation

Red Hat Enterprise MRG’s High Throughput Computing and grid capabilities are based on the Condor project created by and hosted at the University of Wisconsin, Madison. First developed in the late 1980’s, Condor has been under continuous active research and use and possess features and capabilities that far exceed those of any commercial, proprietary grid product. However, Condor has not seen significant industry usage to date because it does not provide all the enterprise features, manageability and supportability that customers require. For example, one of the first pieces of work Red Hat performed on Condor was to break it up from one large, statically linked program into separate RPM packages that are robust, manageable, upgradeable and can be discreetly patched.

Red Hat and the University of Wisconsin have signed a unique partnership around Condor. Under this agreement, the University of Wisconsin will release Condor’s source code under an OSI-approved open source license so that Red Hat may include Condor in its open source distributions, and Red Hat will jointly fund and staff Condor development on-campus at the University of Wisconsin.

Condor has a large community of users and researchers in the academic space. Through its agreement with the University of Wisconsin, Red Hat will be able to bring this innovation from academia to the enterprise. Furthermore, Red Hat and the University of Wisconsin will work to strengthen Condor with additional features and enterprise strength and also enhance Linux for High Throughput Computing to the benefit of both scientists and enterprises. Red Hat believes that this will lead to great advances in infrastructure technology and a great partnership between industry and academia. This is the best kind of customer-driven, open source innovation of all: one that not only advances technology but improves the way we do things.

For more information on Red Hat Enterprise MRG, see here.