In any event, the platform wasn't suited to NASA's needs. And when NASA engineers attempted to contribute additional code to improve its scaling, this didn't work either. "I don't think the average user of Eucalyptus is going to build multiple global data centers on 10 Gigabit networks and manage exa-byte file systems. [Eucalyptus'] target customer is different, and we're not their target customer. We were constantly working around issues and patching things that they didn't want to include back into their code base because it wasn't ideal for their target customer."
As Kemp points out, NASA recently launched a project with Microsoft Research that provides a real-time, high-resolution view of Mars. It draws on over 15,000 tera-pixel photos that are then mosaic-ed into a half billion PNG images via the Nebula cloud. Indeed, the average corporate user isn't likely to require that sort of power.
"NASA, to a certain extent, exists in a class of its own," Kemp says. "We have the fastest peaceful supercomputer here, and certainly, the largest Intel-based supercomputer on the planet, with about 100,000 cores. I think we're up to about a petaflop. We can generate petabytes of data in hours."
And so, about six months ago, NASA engineers went to work on Nova. Pieces of Nebula are still running Eucalyptus, but the entire cloud will eventually move to the new platform. "Nebula is pursuing Nova in the future," Kemp says. "We're doing no further R&D on Eucalyptus...We will continue to run [Eucalyptus] for a few of the projects that are on it. But we're putting all of our investment in Nova. The future roadmap of Nebula is Nova as a compute engine and fabric controller." Kemp adds, however, that Eucalyptus may be used elsewhere at NASA.
In open sourcing Nova under an Apache license and adding it to the OpenStack project, NASA hopes to both foster development from outside coders and create a thriving "ecosystem" of infrastructure cloud platforms. "Now that Nova is included in OpenStack, we are anticipating a much larger community of developers running code that we'll continue to consume in the Nebula project."
OpenStack also includes code open sourced by Rackspace. The hosting provider has donated the compute engine that drives its Cloud Servers service and the on-demand storage platform behind its Cloud Files service. Some Rackspace code will be mixed with Nova, and Rackspace intends to use this, well, super Nova on its production services as well. Meanwhile, NASA will adopt Rackspace's Cloud Files software.
"In the Linux world, there are a lot of different kernels," Kemp says. "I think that the OpenStack will create cloud kernels, if you will. We will take a lot of the code. We will evangelize the development of that code. And we'll add some special sauce to it for Nebula. Rackspace will check the same code out and optimize it for their service, and that'll become a kernel. The exciting part of this project is we're creating a Linux-type ecosystem at that high-level of abstraction at the data center level for the cloud."
The OpenStack project already has the backing of several other big cloud names — including Cloud.com, Cloudkick, Dell, Opscode, and RightScale — as well as hardware names such as Intel, AMD, and Dell. Last week, Rackspace gathered the lot at its Austin, Texas headquarters to discuss the project.
Where does this leave Eucalyptus? Kemp makes it clear that he believes the platform still has its uses in the enterprise. But as Rich Wolski said, it's not a project suited to Amazon-like scale — for many reasons. "There's a difference between open core and open source," says Alex Polvi, the CEO of Cloudkick, an outfit offering a service for overseeing virtual servers running on Amazon-like public clouds as well as in private data centers.
"NASA and Rackspace have no business benefit directly from the code itself...OpenStack is a true open source project, where all the features will be given away for free because this benefits everyone." ®