Nutanix Cache Money: When to Spend, When to Save

Spend wisely

Spend wisely

Every Nutanix virtual storage controller has local cache to server the workloads running directly on the nodes. A question that comes up is if the local cache should be increased. No ever complained about having too much cache but being a hyper converged appliance we want to keep the RAM available for the running workloads if needed. I would never just recommend giving every controller virtual machine(CVM) 50 GB or 80 GB of RAM and see where that gets you.

The cache on the CVM is automatically adjusted when the RAM of the CVM is increased. I recommend increasing the CVM memory in 2 GB increments and track the effectiveness of the change. Even starting with 16 GB of RAM in a system that has 256 GB of RAM available is only ~6% of the RAM resources available.

Nutanix CVM Resources starting points




Inline Dedupe

Memory Size

Increase to 16 GB

Increase to 24 GB

Memory Reservation

Increase to 16 GB

Increase to 24 GB

Base (Non-Dedupe)

Go to any CVM IP address and check the startgate diagnostic page http::2009 and use the below guidelines before increasing your RAM on the CVM. You may need to allow access to the 2009 port if you’re accessing the page from a different subnet. This is covered in the setup guide.

Extent Cache

Amount of CVM RAM

Extent Cache Hits

Extent Cache Usage


16 GB

70% – 95%

> 3200 MB

Increase CVM RAM to 18 GB

18 GB

70% – 95%

> 4000 MB

Increase CVM RAM to 20 GB

NOTE: Going higher than 20 GB of RAM on a CVM will automatically start allowing RAM to be used for dedupe. If don’t enable dedupe past 20 GB of RAM you will be wasting RAM resources. You can prevent this from happening by the use of GFLAGs. It’s best to contact support on how to limit RAM being used for dedupe if you feel your workload won’t benefit from it.

Using the Prism UI you can assess if more RAM will help hit rate ratio. Cache from Dedupe is referred to as content cache. The content cache spans over RAM and flash. It is possible to have a high hit rate ratio and have little being served from RAM.

In the Analysis section of the UI check to see how much physical RAM is making up the content cache and what your return on it is.
Screen Shot 2014-02-24 at 4.57.10 PM

If the memory being saved is over 50% of the physical memory being used and the hit rate ratio is above 90%. You can bump up the CVM Memory.

NOTE: For both extent cache and content cache it is possible to have a low hit rate ratio and high usage of resource and still benefit from more RAM. In a really busy system the workload may be too large and might be getting cycled thru the cache before it can hit a consecutive time. It’s our recommendation to increase the CVM memory if you know your maximum limit for CPU on the host. Available memory can help the running workload instead of sitting idle.

Hopefully this helps in giving some context before changing settings in your environment.

Learn more about Nutanix with The Nutanix Bible

Speak Your Mind