First there were containers. Then came container management systems to help keep track of them. What's coming over the horizon now is management for specific containerized apps — and in particular, big data apps.
The one we're going to take a look at today is the rather dully named ClusterGX, from a company with a much more extravagant moniker: Galactic Exchange. The company is led by Robert Mustarde, formerly of startups Riverbed and Ruckus Wireless, and Stan Kladko, an early member of the Java development team at Sun Microsystems and recently a co-founder of Cloudessa, a cloud-based security company that was acquired by Global Reach.
ClusterGX is what Galactic Exchange calls an enterprise-class, container-virtualized cluster. So what does that actually mean? "ClusterGX provides a fast and powerful solution for Hadoop/Spark clustering that can be deployed in minutes both on premise or on the cloud provider of your choice," is how the company describes it.
Let's take a closer look at that. The process of setting up a Hadoop cluster with ClusterGX involves signing up for an account, and then downloading the ClusterGX software onto each hardware node that you want to add to the cluster.
This automatically adds each node to the cluster, and installs containerized versions of Hadoop, Spark and other big data software tools as needed. It can be installed on bare metal, Ubuntu or CentOS Linux, Windows, or MacOS servers.
Once the cluster is set up, you can then install containerized enterprise and Hadoop/Spark-powered big data apps with a single click via Galactic Exchange's AppStore — or at least you will be able to when the AppStore goes live later this quarter.
The AppStore will be linked to containerized application repositories such as GitHub, and it will also be possible to install selected commercial applications being integrated into the AppStore.
Easing the Burden of Managing Big Data Clusters on Your Own
What's really clever about Galactic Exchange's approach is that regardless of whether your server nodes are on premises or running in the cloud, it relieves you of the burden of managing the cluster on your own by instead managing it from the cloud.
It also hosts the master node functions required by big data platforms such as Hadoop and Spark, and since these master nodes don't actually do application processing, there are no security worries about data leaving your premises or cloud provider.
Whenever you want to launch an application, ClusterGX spins up a new container or containers to run the application. "If your application has big data dependencies such as Hadoop or Spark or other dependencies such as Impala, Kafka or Hive, ClusterGX automatically spins those up in parallel so your application is ready to run," the company says.
The potential advantages of this type of approach are impressive. Aside from making it possible to deploy Hadoop or Spark in a matter of minutes or hours instead of days or weeks, the use of containers allows you to support different versions of Hadoop/Spark on the same cluster concurrently and to share access to data, cluster compute power, or applications. It also means remote users can bring their applications to your data rather than sending your data to them.
The company claims that by using ClusterGX it's possible to cut cloud compute costs by up to 80% by running multiple containers on the same VM or bare metal server and removing the need for separate VMs or separate clusters for different applications, as mentioned earlier.
So how much does Galactic Exchange's product cost? The software is still in beta, and the good news is that there is a Community Edition which is completely free to use and allows unlimited cluster size and scaling.
The Community Edition will be made available under an open source license in the near future, the company says. The cloud management service is also free for 12 months, and then $99 per node per year thereafter.
A ClusterGX Full-Service version will also be offered, and this will include pre-deployment services and consulting and post deployment monitoring and support. Pricing for this offering has not yet been announced.
Containers are becoming increasingly popular for big data applications implementations, and if Galactic Exchange can make those implementations as easy to set up and manage as it claims, then its increasingly likely that they'll soon be the implementation method of choice.
Paul Rubens is a technology journalist and contributor to ServerWatch, EnterpriseNetworkingPlanet and EnterpriseMobileToday. He has also covered technology for international newspapers and magazines including The Economist and The Financial Times since 1991.