Archives for January 2016

Jan
29

Docker, Docker, Docker!

docker

Well it really has been too long since I looked at Docker. I posted on great course and podcast that I did then haven’t really done much sense other than read about it in the news.

I found myself looking at this post because of the length of time away to get started again -> Docker on CentOS. The reason for me using CentOS is because Nutanix CVM which has three STIGS is based off CentOS. I have grand hopes we can use this same image to provide containers sometime in the future. Currently its just wishful thinking but it sounds like a great idea.

I think its important as an infrastructure person to pay attention to the developers of the world. Developers really do dictate hardware demands in one form or another. My plan is write about Docker for infrastructure folks. My eyes are going crossed from lack of sleep so until next time check out the new Best Practice Guide for Docker from Nutanix.

Jan
29

Commvault Intellisnap Hypervisor Native Backup and Restore For Nutanix

As of today Nutanix and Commvault Intellisnap support both ESXi and Hyper-V. IntelliSnap backup enables you to create a point-in-time snapshot of the data used for backups. An effective way to back up live data is to quiesce it temporarily, take a snapshot, and then resume live operations. This avoids the pain of hypervisor based snapshot getting in the way of your backup. Why is this big deal? Extra IO would have to used collapsing the hypervisor based snapshot while the full backup would take place. The hypervisor snapshot still happens but it brief.

How does it work?

commvault

1) CommServer Requests the Proxy Agent to backup a VM.
2) VSA Proxy Agent requests the host to create the snapshot of the VM. (The Media agent is also installed with the VSA but your don’t need to size for it.)
3) Host uses VMware Tools to take an App-consistent snapshot of the selected VMs
4) Proxy Agent requests the Nutanix to take the storage snapshot of the associated containers. (IntelliSnap). VMware’s VM Snapshot is removed and consolidated.
5) Proxy Agents mounts the container snapshot, performs the indexing. Metadata relationship between commvault snapshot and Nutanix snapshot/PD is maintained.
6) Commvault backup policies determines to backup the VMs from the snapshot to media, and/or retain the snapshot without backing up.
Intellisnap is VADP snapshot aware.

Notes for Step 2 – you need to install the Media Agent component where the proxy agent (VSA) is. That doesn’t need to be the same media agent that’s storing the data, but it’s the package that needs to be installed to facilitate the snapshot functionality for the agent. This is because the media agent contains the code and add-ons that facilitate the snapshot management.

Notes for Step 5 – its optional to do the indexing, but you don’t lose the ability to do a granular restore of files within the guest. You can choose to index the data from the hardware snapshot, OR you can perform the indexing “live” when you need to perform a restore. The tradeoff for the latter is waiting a few minutes to mount the VM and open the disk to browse the contents rather than looking at it from a pre-built index. The recommendation (and default) is the latter option, especially with Nutanix since NFS makes this a much quicker process, and you save some time during the backup by not having to do the indexing of the files within the virtual machine. One great benefit of both indexing options is that there is no need for any agents – not even temporary ones, which means less headaches administering VM credentials etc.

Thanks to Damian Andre from Commvault and Jerome Joseph from Nutanix in helping me understand some of the nuances.

Check back here because this story will get even better in a couple of weeks, wink wink.

Jan
10

Hadoop For Free: Skip Flash

I see more and more people looking at getting started with Hadoop but it can be risky if you don’t have the skills and time in your organization. To top that off you have to buy new equipment that will mostly likely only be used 20% of the time.

It takes more resources and time when deploying Hadoop for the first time.

It takes more resources and time when deploying Hadoop for the first time.

Nutanix has always supported mixed workloads but Hadoop can be blunt force trauma on storage for a variety reasons:

1) Hadoop was never built for shared storage, it was architect around data locality which is core architectural design feature of Nutanix.

2) Large ingest and working sets can destroy the storage cache for your remaining workloads. With Nutanix you can bypass the Flash tier for sequential traffic. If the workloads was sized properly the HDDs are usually relative inactive as they are meant to be cold storage. Using the idle disk for Hadoop will give infrastructure and hadoop teams the ability to test the waters before carrying on.

In the case of customers running NX-8150, they might never need to buy extra nodes for compute. With 20 HDDs at your disposable the raw disk gives great performance with out flash. If your performance is fine running just from HDD you can save additional cost by adding storage only nodes. The storage only nodes don’t require additional licensing from Cloudera or Hortonworks.

While the HDD are idle, the, hadoop admins will play.

While the HDD are idle, the, hadoop admins will play.

DSFIO Test
Performance on Cloudera with 4 nodes of NX-8150 using no flash

Green = Writes Blue = Reads

Green = Writes
Blue = Reads

In the above case CPU was only at 50% so you could run additional workloads even while Hadoop was running. If your goal is just Test/Dev you can also turn HDSF replication factor to 1 since Nutanix provides enterprise class redundancy already. When you add in erasure encoding the effective capacity will be less than 2X compared to 3X with traditional hadoop.

hdfstestdev

Please hit me up on twitter @dlink7 if you have any questions.

-DL