Aug
23

VMware and Nutanix Give Wings to Hadoop

Nutanix has the fortune and misfortune of being thought as strictly a platform for VDI. I am glad that they serve the VDI market or else I probably wouldn’t have heard of them. Nutanix has many benefits, one of which is that their value proposition is based on software. Local controller VM’s on each node give commodity local storage(if you can call Fusion-IO commodity) SAN like abilities in the form of their file system NDFS. Where I am going with this? Hadoop.

Hadoop was built around using local storage using a file system called HDFS (Hadoop Distributed File System). The problem with HDFS and Hadoop is not all the components are spared from the pitfalls of bare-metal installations. The NameNode and JobTracker are not designed to handle hardware failures and therefore it can bring your Hadoop cluster to your knees. With virtualizing Hadoop on NDFS you get the best of both worlds. You can use commodity hardware that is proven and still get the benefits of virtualization on local storage.

VMware comes into the picture with Project Serengeti. Serengeti is an open source project that can be used to package and spit out Hadoop VM’s. Serengeti offers scripts and templates to speed up deployment and I think it will have close tie in’s with Spring family of products. It’s still a mystery to me on how you actually get something like a twitter feeds into Hadoop cluster to get something useful out of it but I am learning!

Performance with Hadoop on Nutanix

Terasort – standard benchmark for Hadoop. Simulated real-world with 4.5 TB working set. Data is from Nutanix webinar.

As you add more nodes, it scaled in linear fashion.

“Nutanix Complete Cluster software-based storage capabilities enable a flexible deployment model ideal for running Hadoop workloads on VMware vSphere virtualized workloads,” said Jerry Chen, vice president, Cloud and Application Services, VMware

The biggest value I can see so far is using your servers 24 by 7. I can foresee VDI servers getting shutdown in the night and those same servers running Hadoop at night. The really cool thing is that if the VDI workload was too high at night there is good chance you could move your Hadoop workload to a Public Cloud. CEO of Nutanix, Dheeraj Pandey mentioned in an online Video that the Nutanix software could be used in Amzaon’s cloud. With Nutanix now offering replication it opens the doors to a lot of possibilities.

The Hadoop on Nutanix Reference Architecture http://www2.nutanix.com/l/8112/2012-08-22/2ypd56

Comments

  1. Jeff Brown says:

    Nutanix and Hadoop enable new horizons for Big Data and bring the value of Big Data analytics to the Enterprise

Trackbacks

  1. […] to working at Nutanix I wrote an article just before VMworld 2012 VMware and Nutanix Give Wings to Hadoop. Now with my feet grounded in the company I think I deliver to you some technical content on how to […]

Speak Your Mind

*