NASA and Rackspace open source cloud fluffer

OpenStack targets one million machine Nebula


NASA and Rackspace have joined forces to open source a new platform for building so-called infrastructure clouds. Known as OpenStack, the platform is available under an Apache license, and when completed – possibly by the end of the year – NASA and Rackspace will ditch their current infrastructure cloud platforms, which don't scale as they would like.

NASA's Nebula infrastructure cloud is installed installed at NASA's Ames Research Center and is in the process of being and extended to the Goddard Spaceflight Center outside of Washington, DC. The Nebula cloud was spearheaded by Chris Kemp, who is now NASA's chief technology officer, and in an interview with El Reg, Kemp had nice things to say about the Eucalyptus cloud fabric controller that Nebula was currently using, which is an open source clone of Amazon's EC2 compute cloud and S3 storage cloud.

But Kemp also said that the scalability of the product and other issues with Eucalyptus (including the inability by NASA to get some of its enhancements into the Eucalyptus code base) compelled Kemp to take the entire Nebula team and dedicate it – for the past six months – to creating a new fabric controller, called Nova, from scratch.

It was serendipity that at just the time when NASA was working on the guts of a new compute engine that Rackspace had decided to contact NASA about its plan to open source the "Ozone" code behind its Cloud Servers compute cloud and the Cloud Files storage cloud (formerly known as CloudFS). NASA had released the Nova cloud fabric controller under an Apache 2 license at http://novacc.org/ (the code is currently at the 0.42 release level and admittedly buggy), and according to Jim Curry, vice president of corporate development at Rackspace, decided not only to open source its Cloud Servers code but to start the OpenStack project to mix-and-match the best bits of its Ozone code base with NASA's Nova code base to forge a better cloud fabric controller.

Nova is written in C, C++, and Python and Ozone is written in Python, and at this point, the exact nature of the merged project is being hammered out by NASA, Rackspace, and 25 other partners who are lining up to support the OpenStack Compute fabric controller.

Like the Eucalyptus tools NASA is currently using in its Nebula cloud, the Nova cloud fabric controller is built to mimic Amazon's EC2 and S3 APIs. Most of the code is written in Python, using the Tornado and Twisted frameworks. It uses the open source Advanced Message Queuing Protocol (AMQP) that Red Hat has championed for its Enterprise MRG cloudy Linux stack as well as a Redis distributed key value store. The code was released at the end of May.

NASA will be adopting the OpenStack Object Storage as its default object-based storage, and this code is literally Rackspace's Cloud Files opened up under an Apache 2 license to match that of NASA's Nova fabric controller.

Sometime in the fourth quarter, after the OpenStack project gets fully organized and programmers from inside NASA and Rackspace as well as from the OpenStack community decide what features and standards to support, both NASA and Rackspace intend to put the OpenStack Compute and Object Storage code into production. At that point, Eucalyptus and VMware's vSphere will be relegated to jobs where NASA needs to support a cloud that has third party application certification, something that Kemp never envisioned doing on the Nebula cloud in the first place and still doesn't see happening. "There is a place in the enterprise for all of these platforms," says Kemp.

The problem with the tools being created and sold by Eucalyptus Systems, VMware, and others trying to provide tools to fluff up clouds is that the scale that NASA and Rackspace are wrestling with is beyond the capability of these products.

"We are not the target customer," says Kemp. "We are constantly working around issues and creating code because of this." The difference is between a platform, which you can pull apart in a modular way – perhaps changing the networking engine or the queuing engine to boost performance – and a product, which is tends to be a single, integrated thing bound by the needs of an average of customers, not those pushing the scalability limits.

What kind of scalability are NASA and Rackspace talking about with the OpenStack software? The initial design goals agreed to by the two parties (who do not have any kind of formal arrangement except to contribute code to the OpenStack project and work in the community) is for OpenStack to span 1 million host machines and control up to 60 million virtual machines.

"VMware has not built something that can scale up to Amazon-level scalability of hundreds of thousands to millions of servers," says Curry at Rackspace. And as for Eucalyptus, Kemp over at NASA said "our team's analysis is that it could not get anywhere near it," meaning the design goals for OpenStack. Curry joked that if anyone wants to provide Rackspace with the iron, it would be happy to test it across a million boxes. (What do you say, Google?)

The exact feeds and speeds of the OpenStack Compute fabric controller have not been hammered out, but Rackspace hosted a design summit in Austin, Texas from July 13 through 16 where 100 technical advisors from 25 companies were on hard to help hammer out the initial details. These include AMD, Autonomic Resources, Citrix Systems, Cloud.com, Cloudkick, CloudSwitch, Dell, enStratus, FathomDB, Intel, iomart Group, Limelight, Nicira, NTT Data, Opscode, Peer 1, Puppet Labs, RightScale, Riptano, Scalr, Sonian, Spiceworks, ThoughtWorks, Zenoss, and Zuora. The initial cloud fabric will support KVM virtual machines as well as Xen virtual machines based on the open source Xen as well as Citrix Systems' XenServer implementation. (It is not clear yet how the Xen community, which created its own cloud stack last summer, and Citrix, which sells a commercial cloud fluffer called Citrix Cloud Center (or C3 for short), will participate in the CloudStack project.)


Other stories you might like

  • Employers in denial over success of digital skills training, say exasperated staffers

    Large disparities in views from bosses vs workers on 'talent transformation initiatives,' says survey

    Digital transformation projects are being held back by a lack of skills, according to a new survey, which finds that while many employers believe they are doing well at training up existing staff to meet the requirements, their employees beg to differ.

    Skills shortages are nothing new, but the Talent Transformation Global Impact report from research firm Ipsos on behalf of online learning provider Udacity indicates that although digital transformation initiatives are stalling due to a lack of digital talent, enterprises are becoming increasingly out of touch with what their employees need to fill the skills gap.

    The report is the result of two surveys taking in over 2,000 managers and more than 4,000 employees across the US, UK, France, and Germany. It found that 59 per cent of employers state that not having enough skilled employees is having a major or moderate impact on their business.

    Continue reading
  • Saved by the Bill: What if... Microsoft had killed Windows 95?

    Now this looks like a job for me, 'cos we need a little, controversy... 'Cos it feels so NT, without me

    Former Microsoft veep Brad Silverberg has paid tribute to Bill Gates for saving Windows 95.

    Silverberg posted his comment in a Twitter exchange started by Fast co-founder Allison Barr Allen regarding somebody who'd changed your life. Silverberg responded "Bill Gates" and, in response to a question from Microsoft cybersecurity pro Ashanka Iddya, explained Gates's role in Windows 95's survival.

    Continue reading
  • UK government opens consultation on medic-style register for Brit infosec pros

    Are you competent? Ethical? Welcome to UKCSC's new list

    Frustrated at lack of activity from the "standard setting" UK Cyber Security Council, the government wants to pass new laws making it into the statutory regulator of the UK infosec trade.

    Government plans, quietly announced in a consultation document issued last week, include a formal register of infosec practitioners – meaning security specialists could be struck off or barred from working if they don't meet "competence and ethical requirements."

    The proposed setup sounds very similar to the General Medical Council and its register of doctors allowed to practice medicine in the UK.

    Continue reading
  • Microsoft's do-it-all IDE Visual Studio 2022 came out late last year. How good is it really?

    Top request from devs? A Linux version

    Review Visual Studio goes back a long way. Microsoft always had its own programming languages and tools, beginning with Microsoft Basic in 1975 and Microsoft C 1.0 in 1983.

    The Visual Studio idea came from two main sources. In the early days, Windows applications were coded and compiled using MS-DOS, and there was a MS-DOS IDE called Programmer's Workbench (PWB, first released 1989). The company also came up Visual Basic (VB, first released 1991), which unlike Microsoft C++ had a Windows IDE. Perhaps inspired by VB, Microsoft delivered Visual C++ 1.0 in 1993, replacing the little-used PWB. Visual Studio itself was introduced in 1997, though it was more of a bundle of different Windows development tools initially. The first Visual Studio to integrate C++ and Visual Basic (in .NET guise) development into the same IDE was Visual Studio .NET in 2002, 20 years ago, and this perhaps is the true ancestor of today's IDE.

    A big change in VS 2022, released November, is that it is the first version where the IDE itself runs as a 64-bit process. The advantage is that it has access to more than 4GB memory in the devenv process, this being the shell of the IDE, though of course it is still possible to compile 32-bit applications. The main benefit is for large solutions comprising hundreds of projects. Although a substantial change, it is transparent to developers and from what we can tell, has been a beneficial change.

    Continue reading
  • James Webb Space Telescope has arrived at its new home – an orbit almost a million miles from Earth

    Funnily enough, that's where we want to be right now, too

    The James Webb Space Telescope, the largest and most complex space observatory built by NASA, has reached its final destination: L2, the second Sun-Earth Lagrange point, an orbit located about a million miles away.

    Mission control sent instructions to fire the telescope's thrusters at 1400 EST (1900 UTC) on Monday. The small boost increased its speed by about 3.6 miles per hour to send it to L2, where it will orbit the Sun in line with Earth for the foreseeable future. It takes about 180 days to complete an L2 orbit, Amber Straughn, deputy project scientist for Webb Science Communications at NASA's Goddard Space Flight Center, said during a live briefing.

    "Webb, welcome home!" blurted NASA's Administrator Bill Nelson. "Congratulations to the team for all of their hard work ensuring Webb's safe arrival at L2 today. We're one step closer to uncovering the mysteries of the universe. And I can't wait to see Webb's first new views of the universe this summer."

    Continue reading

Biting the hand that feeds IT © 1998–2022