More on cloud computing
Cloud hosting vendor Rackspace is teaming up with NASA to launch a new open source effort for building large enterprise clouds. Officially called the OpenStack project, the effort will involve both compute and storage elements, enabling users to build large distributed pools of connected resources.
The OpenStack project builds on efforts already underway by both Rackspace and the space agency. Rackspace had been developing its own cloud storage technology, while NASA, by way of its Nebula project, was building out a distributed compute fabric.
The OpenStack effort will hit the market as a new challenger to Amazon's cloud and S3 services, while also providing another option for open source users.
"From NASA's perspective, having the broader community of developers and organizations using this technology means that NASA will have more choices too," Chris Kemp, NASAs chief technology officer for IT, told InternetNews.com. "We haven't had the resources to add VMware, Xen or Hyper-V support to the Nebula computing engine, but now that this is going to be worked on by a larger community of developers, we'll have those options now and taxpayers won't have to give us a penny to have that flexibility."
The NASA Nebula project is currently built with the Ubuntu Linux distribution, and uses the KVM virtualization hypervisor. Kemp explained that the decision to partner with Rackspace in the OpenStack effort has to do with the flexibility of storage choices, since NASA's large images and datasets make its storage requirements immense.
"The Nebula code orchestrates the interface between end users and the actual underlying infrastructure," Kemp said. "There are hypervisors, virtual networks and filesystems and the computing engine is orchestrating all of that."
Kemp added that an end user could request a new virtual machine and then the Nebula code in turn figures out which physical hardware it should run on, how much memory to use, how the network endpoints are set up and how the filesystems are set up and located. He added that the system is hypervisor agnostic at this point, as are the choices for filesystems.
"We want to see standards emerge in the stack so that components are interchangeable and that gives us the flexibility to solve different problems," Kemp said.
NASA did look at other open source and commercial technology, including Eucalyptus, to help solve its cloud-computing needs. The open source Eucalyptus project is part of the Ubuntu Enterprise Cloud (UEC) offering.
"We did look at Eucalyptus [and] tried to make it work, but it wasn't built for very large infrastructure," Kemp said. "So we ended up rolling our own and now we want to share what we've developed and build a robust community around it."
The storage perspective is where Rackspace's open source contributions come into play. The OpenStack object-storage component is derived from software that runs Rackspace's Cloud Files storage offering.
"Cloud Files is roughly equivalent to Amazon's S3 storage system, and it's meant for storing objects or files in a redundant way so there is low risk of data loss," Jonathan Bryce, cofounder of the Rackspace Cloud, told InternetNews.com.
The OpenStack object storage is not a filesystem itself, but rather a complement to the filesystems that an enterprise user already has in place.
"It uses filesystems that are available on Linux. This is not a separate filesystem -- it sits on top of a filesystem," Bryce said. "It uses the filesystem to store the blocks of data."
He explained that each device inside of a server is managed independently. When a new object is uploaded to the OpenStack object store, it gets logged, tracked and copied across the cloud network resources. He added that the system does the work to ensure that the data is always replicated and available.
The OpenStack components are being licensed through the Apache open source license, and they aren't going to be directly monetized by either Rackspace or NASA. Both groups are aiming to grow an ecosystem of developers and users for the technology.
"We as a hosting company don't have a desire to build a software business and we're not pursuing a dual-licensing strategy," Jim Curry, vice president of corporate development at Rackspace, told InternetNews.com. "We want to make this technology widely available and hopefully have it emerge as a standard. We then hope to win customers based on our support and value proposition for it."