Sep
25

Maximize Your ROI and Agility with Big Data #Nutanix #Docker #BlueData

Separate out your data from your compute for more agility.

The datanode is what is used to build out the HDFS. Typically the the dataNode and the nodeManager are co-located on the same host whether its physical or virtual. The NodeManager is responsible for launching and managing containers that are scheduled from the Resource Manager. On Nutanix if you virtualize the dataNode and the nodeManager on separate virtual machines you have the opportunity to increase your agility. The agility comes from the ability to use your resources to the max of your capacity at all times. When the the cluster isn’t in use or as busy, other systems have the opportunity to use the resources. You can shut down the NodeManager since they’re not responsible for persisting data and make the the CPU and memory available for another project like Spark or maybe a new machine-learning program someone wants to test out.

Hold the phone! What about data locality? You are correct performance is going to take a hit. Performance may drop from up to 15% from the standard way but if your system is only busy 30% of time it might be more that worth it. Let’s say a job takes 60 minutes to complete. Using this new model of separating out compute and storage, the job may now take 70 minutes to complete. Is the extra 10 minutes worth the agility to use your hardware for other projects? I think so but that is going to depend on your business requirements of course.

On the data locality side, the datenode still gets to benefit from reading locally. It’s data path on the network isn’t going to cause more stress so that’s a plus. Also the nodeManager is busy writing all of the temporary and shuffle data locally so that is also not going to cause any additional stress compared to having the nodeManager write to a remote shared storage device. Also in some cases the NodeManager will still talk to the local datanode over the local hypervisor switch.

If your after some real flexibility you could look at using BlueData to run Docker containers along side the dataNodes. BlueData will take over for the nodeManager essentially. Install some CentOS VMs that fit inside the hosts NUMA node and install BlueData. BlueData can help with QofS for different tenants, allow you to run different versions of Hadoop distros, Spark, Kafka and son on without blowing out your data requirements. BlueData also helps to maximize the remote connection between the containers and HDFS distro of choice.

If your after more agility, avoiding separate hardware for projects, getting better ROI for systems that run only weekly, monthly, quarterly or better testing methodologies this may be the right architecture for you to try out.

Feb
27

Commvault IntelliSnap & Metro Availability with Nutanix

I was asked if Commvault IntelliSnap works with Metro Availability on Nutanix and I wasn’t 100% certain due to the snapshots that Metro production domain would have to take. So after giving it a quick test, it turns out it works just fine.

Setting your retention policy

Setting your retention policy

I would just take into account that if you have separate jobs running that each time it will take a snapshot of the container and then your retention policy will come into affect for the production domain.

Below is quick video of the process in action.

Nov
27

Leading Canadian Service Provider Agilis Networks Runs Nutanix XCP

agilisIt’s great to hear success stories in the service provider space other than AWS. Agilis Networks needed a new more resilient storage solution that was powerful and responsive enough to support all of their external b2b platforms. Making money for them was also not spending countless hours fixing problems, the promise of hyper-converged with web-scale properties made Nutanix the right choice.

In the Agilis Networks business model, customers can buy cycles all the way up to the VMs, according to Frescura. “We provide data center hosting, power, air conditioning, server cycles, and VMs to our customers,” he explained. “But unlike a much larger service, we don’t serve hundreds or thousands of customers. We aren’t a commodity provider, we only serve clients with very rigorous up time requirements. We o er a very high level of reliability and redundancy,so our solution is the perfect fit for customers that need a robust service.

Using Nutanix Prism
Agilis Networks is now using the Nutanix Prism interface to manage the environment. “We keep an eye on the messages and alerts, and our storage admin checks in about once a day just to make sure everything is good,” Frescura said. “There haven’t been any problems since we deployed the systems, but it’s always reassuring to see the statistics. It really is a set it and forget it platform.”

Working with Nutanix Channel Partner of the year for 2014 and 2015, Zycom Technologies, Nutanix was selected as the best solution to meet their needs. As well as obtaining high availability for all of their applications, they have reduced their datacenter footprint by 3x and seen significant savings in power costs.

Check out an on-demand webinar in your area of interest:

http://www.nutanix.com/resources/webinars/

Nov
16

Nutanix Top Of Rack Integration: Video

Quick demo of the Top of Rack integration inside of Prism.

Oct
01

McAfee MOVE For The Acropolis Hypervisor

McAfee MOVE AntiVirus Multi-Platform lives up to it’s name. Move MP is hypervisor agnostic so it fits perfectly into what Nutanix is doing from a VM mobility standpoint. If you want to do DR or migrate VM’s between different hyperviors you don’t have to worry about your AV solution getting in the away as-long as your VM’s have access to the Offload Scan Servers. This solution removes the need to install an anti-virus application on every VM, and it is the original agent-based deployment option.

The solution has been around for awhile and has been steadily approved upon. I actually run this years ago for a VDI View environment and back then a limitation was that it couldn’t do on-demand scanning which is no longer a limitation. The targeted on-demand scan feature allows the administrator to select a system or a group of systems from the System Tree in McAfee ePO and assign a client task to initiate the on-demand scan immediately. Pretty handy if you just want to target one area if you’re checking for something specifically related to a threat.

The Multi-Platform deployment option offloads all scanning to a dedicated VM — an offload scan server — that runs VirusScan Enterprise software. Guest VMs are no longer required to run anti-virus
software locally, which improves performance for anti-virus scanning, and increases VM density per hypervisor.

move1

The Multi-Platform deployment option:
• Uses McAfee ePO to manage the MOVE configuration on the client systems, offload scan server, and SVA Manager (OSS Manager).
• Leverages the McAfee Agent for policy and event handling.
• Uses McAfee ePO for reports on viruses that are discovered on the VMs

RAM disk for scanning

RAM disk is used by the OSS for file scanning and it significantly reduces the disk I/O on the offload scan server. You can enable the RAM disk option in the ePolicy Orchestrator server. RAM disk is created by the OSS and it improves the OSS performance by enhancing the scan time. This way you can save your flash on Nutanix for your workloads.

If you’re looking for an AV option for AHV, McAfee Move Multi-Platform is an option you should look at.

Mar
03

W5 of Using VMware View Planner: Complete Setup

by Tony Holland Senior SE Nutanix @tonyholland00

Have you ever wanted to test a VDI workload to see what it looks like? If you have, then I would highly recommend looking at VMware View Planner.

I have been working with VMware View Planner since its inception with View Planner 1.0. It was a very primitive tool at the time, but having a tool to simulate what an actual VDI user typically does throughout the day is a very hard thing to come by. Who doesn’t have hundreds or thousands of users at your beck and call to test with? VMware View Planner provides exactly that. The primary users of VDI workload generation tools are hardware vendors. Hardware vendors do not have 100-1000’s of users to test with so they are forced into using synthetic tools to simulate the workload. The workload generated will not match your companies user workload, but it does provide us (Nutanix) and all other hardware players the exact same test to run over and over to find the breaking points / bottlenecks in our solutions. It is not a sizing tool!!

To acquire VMware View Planner you need to talk with a VMware partner. The download for the version of VMware View Planner 3.0 is only available though VMware Partner Central.

One key point that many IT administrators don’t understand about View Planner is that it should not be used as a sizing guideline for how many VDI users you can run on existing or new hardware. View Planner does a great job of simulating the workload of a VDI user but I have yet to see a single company that has a VDI image / template exactly like the one used by View Planner. I also have never seen a company that all users do the exact same thing for hours on end. If you want to size how many actual users you will be able to run, you should use a real user. Pick a handful of people to deploy a VDI desktop to and run a very small pilot. Use your corporate image/template to build the VDI desktops and have actual users work off those VDI desktops. Collect Read-Write IOPS, CPU and Memory data used per user during the pilot and extrapolate that data to figure out the correct amount of users your environment can handle. Don’t forget about N+1. View Planner should be utilized to simulate a synthetic VDI workload that can help find the bottlenecks in your hardware.

Another great thing about View Planner is that you can use it with other VDI technologies not just VMware View. If you follow my directions below you can utilize any VDI servicing apps such as XenDesktop or VMware View to create the desktop pools. I love this because I can test all the features, for example…VMware Storage accelerator, VMware VCAI, Citrix MC or PVS. The only things required for VMware View Planner are the ESXi hypervisor and vCenter. A simple Powershell script could also spin up the required number of desktops to do a View Planner run.

While View Planner does a nice job simulating VDI workloads, it does not take breaks. Think about your office, how often do you see people flirting with co-workers, chit-chatting on phone, going on a smoke breaks and impromptu meetings. With View Planner testing 100 or 1000 desktops in those 3-5 hours while you run the 5 iterations, those 100-1000 VDI desktops will be working/running 100% of the time – no potty breaks. This phenomenon of 100% of your users working simultaneously, not only will never happen but it also will cause the user density number to be skewed. Planning for 80% concurrency is probably more realistic.

Why use VMware View Planner in local only mode? To me the local only mode is very simple, it uses the least amount of infrastructure, resources, and gets IT administrators the data that they are yearning for. Again, it is not a sizing tool, but it is a great load (IOPS and CPU/Memory) generation tool. Utilizing remote mode you are testing the PCoIP protocol. PCoIP typically is not the issue when dealing in a LAN environment IOPS, CPU and memory are. So in my opinion adding extra complexity and hardware to have virtual client desktops launching a bunch of View sessions is unnecessary to get the data that most IT admin’s are looking for.

If View Planner looks like a tool that would be beneficial to you here are some prerequisites :

• View Planner 3.0 (Talk to your VMware partner)
• Microsoft Office with Volume Licensing 2007 or 2010 with no service packs and only (32-bit)
• Microsoft Internet Explorer version 8.x or 9.x.
• Mozilla Firefox version 3.6.x, version 4.0.x, version 5.0.x, version 6.0.x, or version 7.0.x
• Adobe Reader version 9.x or version 10.x (also called Adobe Reader X) on each desktop virtual machine.
You will find all of the instructions on how to install the applications in the View Planner User Guide page 78
• ESXi hosts 5.0, 5.1 or 5.5 to run View Planner appliance and the VDI Desktop VM’s
• VMware vCenter

Steps to setup View Planner

• Download the View Planner OVF from VMware Partner Central
• Download Planner Documentation from Partner Central
• Deploy OVF of View Planner on an ESXi host… Follow View Planner User Guide p.35
• Login to View Planner appliance web page (Username: root Password: abc123) and configure your vCenter IP, and credentials and
vCenter VDI Cluster Datacenter… Follow View Planner User Guide p.38
• Create a new VDI Desktop template Windows XP, 7 or 8 VM named “GoldTemplate”…Follow the View Planner Users Guide p.65
• Install template desktop OS using a VL MAK or KMS key KMS is preferred (XP, Windows 7 or 8)… Follow the View Planner Users Guide p.68 instructions for “Installing Applications in Desktop Virtual Machine”

Complete View Planner User Guide Documentation Sections
• Create the Desktop Template Virtual Machines…p. 65
• Install Microsoft Windows in the Template Virtual Machine…p. 68
• Install the VMware Tools Suite in Microsoft Windows…p. 72
• Add a Data Disk to the Desktop Virtual Machine…p. 75
TIME-OUT!!
• Go into Computer Management and Manage the local user accounts. You will need to enable the Administrator account and assign the password of Q1w2e3r4!
• Reboot computer and login as Administrator and password Q1w2e3r4!
• Continue to follow the View Planner guide on installing applications in Desktop Virtual Machine…. p. 78

Complete View Planner Documentation Sections
• Install Applications in the Desktop Template Virtual Machine…. p. 78
• Enable Pseudo Perfcounter for the Virtual Machine…. p. 79
• Install Windows Updates…. p. 80
• Add the Desktop Template Virtual Machine to the Dedicated Domain…. p. 82
• Install the View Agent…. p. 85
• Take a Snapshot of the Virtual Machine…. p. 85
• Download and Install the View Planner Workload Support File…. p. 86
• Shutdown

Now you can start to follow my directions as View Planner documentation and I go about things a little differently.

• Startup your View Planner Desktop template it should auto -login and you should notice a cmd window bat file running in the background.

Viewpanner1

• Browse C:\ drive of the View Planner Desktop template. Notice the IP.txt file. The only line needed is the harnessIP=.
** – If you do not see IP.txt your installation of the View Planner desktop files did not complete successfully. Manually create the file if need be, but their could be
other problems with the template install. I would attempt to uninstall and reinstall the View Planner desktop files. If all else fails go back to snapshot taken before this step.

vp2

Pre View Planner Template Checkup
• Check to make sure that the Windows Firewall is disabled
• Startup all apps, Word, Excel, Powerpoint, Adobe, Firefox, IE and make sure that no extra windows pop up when the app is running.
• Shutdown the GoldTemplate VM.
• Login to View Planner Appliance web page (username: root password: abc123)
• Under Run & Reports tab click on New under Run Profiles
• Fill out Profile Name: Test-1VM
• Number of VM’s: 1
• Desktop Name Prefix: GoldTemplate
• RampupTime:5
• Test Type: Local
• AD Group Settings: Name: Test | Workload Profile: Choose the StandardBenchmarkProfile_1i | % VMs: 100
• Click Save
• Login to the View Planner Appliance via SSH via Putty or a SSH client (username: root password: vmware)
• Change directory to /ViewPlanner
• Tail -f viewplanner.log
• Go back to the View Planner Appliance webpage
• Under Run & Reports tab click on the run profiles and click on the newly created Test-1VM
• Click on Run
• This will start the View Planner process on the VM call GoldTemplate
• Monitor the viewplanner.log via the SSH session. You should see View Planner scan all VM’s to find a match to “GoldTemplate”

viewpanner5

• View Planner appliance will tell vCenter to start the VM “GoldTemplate” and View Planner will wait for the guest to register or (check in) with the View Planner appliance.
• Open the VM Console view of “GoldTemplate” from vCenter and watch View Planner startup and run the applications.

viewpanner6

• Make sure that all apps open correctly and that no extra pop up windows appear.
• After around 20-30 minutes you will see that the workload was successfully uploaded to View Planner
• Look at the viewplanner.log screen on the SSH session. You should see a message indicating that the test was complete.
** Couple of things to watch for here…
If you used Office x64 View Planner will stall on MS Word and not work
If you don’t have the correct HarnessIP in the IP.txt file you will never connect to the View Planner appliance and the workload will not start.
Windows Firewall will also get in the way. Be sure that firewall is disabled after you added the computer to the domain.

viewplanner7

• Properly Shutdown GoldTemplate VM
• Create a final snapshot of the GoldTemplate VM called TestedViewPlannerSetup (Use this snapshot with VMware View Linked Clones)
• You can also now delete the snapshot created before as you have now validated that your desktop is ready to go!

Creating Desktops via VMware View 5.3

I utilize VMware View to create the desktops for my View Planner runs. You can use whatever technology you prefer when creating these pools. Even Xendeskop will work!!! In this use case I am going to create non-persistent Dedicated pool with VMware View Linked Clones.

• In VMware View add a new Pool
• Type “Automated Pool”
• User assignment “Dedicated”
• View Composer Linked Clones
• Pool ID: Pool1
• Pool Settings: I leave the defaults PCoIP will not be used so these settings are not really relevant
• Provisioning Settings: Enable Provisioning; Disable Stop provisioning on error
• Virtual Machine Naming: Use Naming Pattern: VDI-P1-
• Pool Sizing: X amount of desktops you want to create
• Provisioning Timing: Provision All Desktops up front
• View Composer Disks: Redirect disks if you would like I typically choose not to for this test
• Storage Optimization: For Nutanix you should never separate Replica and OS disk other technologies you might want to choose this option
• vCenter Settings: Choose your Parent VM: GoldTemplate
Choose Snapshot: TestedViewPlannerSetup
VM Folder: Your Choice
Host or Cluster: Hosts running Guests
Resource Pool: Your Choice
Datastore: Choose Datastore to provision Pool desktops onto. Nutanix should typically only have one Datastore.
• Advanced Storage Settings
• Use View Storage Accelerator: Optional (Adds provisioning time)
• Use NFS Snapshots: Optional again speeds up provisioning
• Click Finish to create the pool!!

• Once provisioning has completed and all desktops are fully provisioned clicking on desktops in VMware View should like out all of the newly created VM’s. All desktops should have a status of “Provisioned” and a DNS name.

viewplanner8

• After all desktops are successfully provisioned and DNS assigned Disable the VMware Pool.
By disabling the pool you remove VMware View from controlling the desktops. If you leave the pool enabled and shutdown all VM’s to prepare for the View Planner run, View will automatically power on the VM’s which in production is a great thing, but in this use case having vCenter control all of the power on and power off’s. From this stage forward we will use View Planner and vCenter to power on and off the desktop’s. Thanks for doing all that work View but we are done with you for now!

Prepare Desktops for View Planner Run

• Using vCenter go to the cluster object and click on the Virtual Machine tab. Sort VM’s by name. You should be able to highlight all powered on VM’s and select the Shut Down Guest OS option. This will cleanly shut down all of your desktop VM’s to prepare for View Planner run. I typically create a few powershell scripts that will do this by Name if you are working with more than 100 or so guests.

• Login to the View Planner appliance via SSH via Putty or SSH client of choice.

• Change directoy to ViewPlanner
• Cd /ViewPlanner
• Edit the adminops.cfg file
• vi adminops.cfg
• Press the i key for insert mode.
• Find the CONCURRENT_POWERONS_ONE_MINUTE=64

viewplanner10

• I typically change this to 32 or a lesser number as this can create a fairly large bootstorm. If only a few hundred users you can probably leave at 64. This is also a fun parameter to play with to see how a boot storm can effect the environment.
• Edit to 32 or whatever value you would like to set
• Press esc key
• :wq! (this will write changes to the adminops.cfg file)

• Reboot View Planner appliance VM

Run a View Planner Test

• Using vCenter perform a proper shutdown of all VDI guests that you want to run View Planner on.
• Login to the web console of the View Planner Appliance
• Under the Run & Reports Tab create a New Run Profile.

viewplanner11

• Fill out Profile Name: Production-Run
• Number of VM’s: 100 or however many VM’s you want to test
• Desktop Name Prefix: VDI-P1- (use the name scheme that you used to create your VDI Pool in View)
• Ramp up Time:(will populate automatically based on # of VM’s)
• Test Type: Local
• AD Group Settings: Name: Test | Workload Profile: Choose the StandardBenchmarkProfile_5i | % VMs: 100 click Add Group
• Save
• Login via SSH to View Planner Appliance (username: root password: vmware)
• Change directory to /ViewPlanner
• tail -f viewplanner.log
• Switch back to the VMware View Planner webconsole
• Select you newly created Run Profile “Production-Run”
• Click on Run

• Notice vCenter starting to power on all of your desktopsin the viewplanner.log. This is a great place to watch what is exactly happening with the View Planner run.

viewplanner12

• After all desktops are powered on and all checked in View Planner will attempt to resolve IP Address for each desktop via VMware tools.

viewplanner13

• After successful IP address resolution View Planner will take a moment to rest(Ramp Up Time) before it starts to tell the VM’s to begin the workload.

viewPlanner14

• The workload will then be started on each desktop machine. You should see CPU and memory usage start to increase on the ESXi host(s).

The test will now run though 5 iterations. Which means it will do 5 loops of the test. If you want more iterations to see how your storage handles it simply adjust that in the View Planner run profile. I have set iterations to 5000 so it would start and run for days to see how everything handled it.

This is fun time for an IT administrator. Once the test is started utilize ESXTOP take a look at IOPS (cmds) and latency. Look in vCenter at CPU and RAM consumption. Look at Datastore latency from vCenter……It is like Christmas with data.

esxtop

viewplanner15

viewplanner16

• Once CPU, memory and IOPS drop off you can make a safe guess that the View Planner test is completed.
• Checkout the viewplanner.log via the SSH
• Also have a look in the Web Console. You should see the Run Successful

viewplanner17

• Now that you have completed a successful View Planner run you will have all sorts of great information.
• I typically will go and shutdown all of my guests as a first step after confirmation that the test was a success. Even if the test was not a success you will want to properly shutdown the guest VM’s to get them ready for another run.
• If you want to test something thing different, delete the Pool in View or simply recompose the desktops. Just remember to disable the pool after you finish the creation as View will continually startup VM’s.

I also wanted to give a shout out to Banit Agrawal from VMware who has helped me out with View Planner from the 1.0 days. Thank you for all of the assistance. I hope this post will help a few others learn from some of the mistakes and stumbles that I had during my time working with View Planner.

Thank you for reading, I hope that this will help understand the use cases for and against synthetic workload testing. If you have questions please don’t hesitate to reach out over twitter. @tonyholland00

Jan
23

EUC Tip 92: SuperFetch Windows 8 vs 7

SuperFetch in Windows 8 is enabled by default. It is VDI-aware and should not be disabled. SuperFetch can further reduce memory consumption through memory page sharing, which is beneficial for VDI. Non-Persistent virtual desktops running Windows 7, SuperFetch should be disabled, but for personal virtual desktops running Windows 7, it should be left on.

Jan
02

EUC TIP 91: Best Practices with Snapshots with RDS

All reference VMs must be generalized before they can be used to provision or recreate a virtualized desktop

As best practice, start with a snapshot of an ungeneralized Windows 7 or Windows 8 virtual machine

Generalize, shutdown, and create new snapshot
In the virtual machine, run the following command from Windows\System32\Sysprep directory
Sysprep /oobe /generalize /mode: vm

The switch Mode:vm only detects the enumerated and synthetic devices that are possible and skips on the physical layer checks. Only works with Windows 8 and 2012

The ‘now’ snapshot reflecting the generalized state will be used to (re)create the virtual desktops

When updating a VM, revert to ungeneralized snapshot, delete generalized snapshot, update VM, generalize, shutdown. and snapshot

I think this process can eliminate the need for a KMS server with RDS but not 100% sure, let me know if you have the answer.

Jan
01

EUC Tip 90: New Paint on a Old Truck, RDP 8.0 for Win 7

RDP 8.0 is disabled by default, change Group Policy settings on the Windows 7 system you’ll be connecting to:

“Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Session Host\Remote Session Environment\Enable Remote Desktop Protocol 8.0” should be set to “Enabled”

“Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Session Host\Connections\Select RDP Transport Protocols” should be set to “Use both UDP and TCP”
[Read more…]

Dec
27

Nutanix: Scale Out Doesn’t Mean Addtional Licensing

man-pantsIt’s that time of year where you can feel like you’re being squeezed from every direction. Hypervisor costs don’t have to be apart of the equation however. Being on lots of sales calls, a common misconception is that if you want to add more storage to Nutanix to also have to pay for more hypervisor licensing. This is not true.

The Nutanix virtual storage controllers don’t relay on any hypervisor management features for the features to work. No vCenter, no SCVMM, and WebVirtMgr(KVM) is needed to run the Nuntanix Operating System(NOS). NOS upgrades are handled without interaction from the hypervisor. The Nutanix AutoPath features allows for seamless upgrades in a non-disruptive fashion.

With vSphere the hypervisor can be upgraded in a rolling fashion using the virtual storage controllers as the focus point. The virtual storage controllers maintain a list of the hypervisors forming the cluster so you can easily run esxcli commands to update the hosts.

Hyper-V can be ran with the core edition and running Linux VM’s don’t require licensing.

KVM no licences to worry about to my knowledge.(I am not strong in this area so Linux folks please chime in)

The 6020 from Nutanix offers 20% more storage than the 6050(for running server workloads), 30% less compute capacity and can be ordered with only 32 GB of RAM. Nutanix has background processes to move the cold data to the new nodes. Flash on the 6020 can be used by remote nodes if their local flash is full, the 6020 can help participate in replication to reduce hot spots in data protection. Spare compute can be used by vSphere vDR or Veeam Backup/DR products for recovery.

If your up on your orchestration you can also add automatically add the Nutanix nodes to your hypervsisopr management platform of choice based using the Nutanix REST-API.

Any other questions about Nutanix and\or maintaining an environment please leave a comment.