Nutanix Native File Services (AFS) Now Supports AV Offload Scanning

With AFS 2.2 and AOS 5.1.2 now supports ICAP(Internet Content Adaptation Protocol), which is supported by a wide range of security vendors and products, is a standard protocol that allows file and web servers to be integrated with security products. Nutanix chose this method to give customers the ability to choose the antivirus solution that works best for their specific environment.

Following is the workflow for an ICAP-supported antivirus solution:
An SMB client submits a request to open or close a file.
The file server determines if the file needs to be scanned, based on the metadata and virus scan policies. If a scan is needed, the file server sends the file to the ICAP server and issues a scan request.
The ICAP server scans the file and reports the scan results back to the file server.
The file server takes an action based on the scan results:
If the file is infected, the file server quarantines it and returns an “access denied” message to the SMB client.
If the file is clean, it returns the file handle to the SMB client.

The ICAP service runs on each AFS file server and can interact with more than one ICAP server in parallel to support horizontal scale-out of the antivirus server. We recommend configuring two or more ICAP servers for production. The scale-out nature of AFS and one-click optimization greatly mitigates any antivirus scanning performance overhead. If the scanning affects AFS file server VM performance, one-click optimization recommends increasing the virtual CPU resources or scaling out the file server VMs. This feature also allows both the ICAP server and AFS to scale out, ensuring fast responses from the customer’s antivirus vendor.

AFS sets scanning defaults across the entire file server, but they are disabled by default per share when you enable file scanning. You can enable scan on write and scan on read. Scan on write begins when the file is closed, and scan on read occurs when the file is opened. You can also exclude certain file types and files over a certain size. Share scan polices can override any defaults set for the file server.

For each ICAP server, we spin up no more than 10 parallel connections per FSVM and randomly dispatch the file scanning among all the ICAP servers. With heavier workloads, which may encounter many scan requests and use all connections, the scan servers with more processing power scan more files. As soon as the current scan finishes, the next file is picked up from the queue, which keeps the number of active connections at 10.

Once AFS quarantines a file, the admin can rescan, unquarantine, or delete the file. Quarantined files can be searched if it is necessary to restore a file quickly.
If your antivirus vendor doesn’t support ICAP, you can scan the shares by installing an antivirus agent onto a Windows machine and then mounting all the shares from the file server. This approach allows you to schedule scans during periods of low usage. At the desktop or client level, you can set your antivirus solution to scan on write or scan only when files are modified. You can configure high-security environments to scan inline for both reads and writes.


#Nutanix #AFS – Cannot access folders/files created with a dot “.”

You may experience issues with third party application when accessing folders/files with a dot “.” prefix in the name such as “.profile” or “.build”.

One of the error messages may look like the one below.

Error mesessage
The “.profile” has different permissions.

This is pertaining folders like “.profile” or “.build”. If can create the folders and see them, but when trying to access them with programs, it throws access errors. The “dot files” have different permissions.

There is a configuration option called “hide dot files” in AFS. This “hide dot files” configuration option is enabled by default in AFS.

You can run the following command on one of the FSVMs to turn off the “hide dot files” option globally.

scli smbcli global "hide dot files" No

Example of the command run successfully below.
nutanix@NTNX-x-x-x-x-A-FSVM:~$ scli smbcli global "hide dot files" no
No existing config 'hide dot files' in smb.conf. Creating a new entry now..!
smb.conf update is successful


Recovery Points and Schedules with Near-Sync on Nutanix

Primer post on near-sync

For the GA release near-sync will be only offered with a telescopic schedule (time based retention). When you set the RPO <=15min to >=1 min you will have the option to save your snapshots for X number of weeks or months.

As example if you set the RPO to 1 min and schedule 1 month retention it would look like this:

X= is the RPO
Y = is the schedule

Every X min, create a snapshot retained for 15 mins (These are the Light-Weight Snaps. They appear as normal snap in Prism)
Every hour create a snapshot retained for 6 hours.
Every day, create a snapshot retained for 1 week
One weekly snapshot retained for 4 weeks (If you select a schedule to retain for 7 weeks, Y would be 7 weeks and no monthly snap would occur)
One Monthly snapshot retained for Y months.

Subject to change, as we’re still finalizing sizing and thresholds based on real-world testing but the user will have an option to change these retention values via NCLI.



Delete AFS Forcefully

There can be instances when graceful removal of file server does not work and you may see following errors, this can happen when the file server is not available and has been deleted without following the right process. Sometimes the FSVMs get deleted instead of using the delete workflow in file server section of Prism.

ncli fs delete uuid=________________________
Error: File server with uuid xxxxx-xxxxx-xxxxx is offline and unable to fetch file server VMs.

Use the following command to delete the file server permanently from the Minerva Database. This is run from any CVM.
minerva –fs_uuids _________________ force_fileserver_delete

file server UUID can be obtained from ncli fs ls command.


Powering Off and Starting Up AFS – Native File Services on Nutanix

There really isn’t a need to shut down AFS but moves and maintenance are a part of life. Here are the steps for a clean shutdown….


Shutting Down:
• Power off all guest VMs on the cluster, leaving only FSVM’s and CVM’s powered on.
• From any CVM run: minerva -a stop
• The stop command will stop AFS services and power off the FSVMs for all File Servers
• Once only the CVM’s remain powered on, run Cluster Stop from any CVM.
• Power Off the CVM’s and Hosts

Starting Up:
• Power on Hosts
• CVMs will auto-start once the Host is up
• Once all CVMs are up, run Cluster Start to initiate cluster services
• Verify all services are up with Cluster Status
• From any CVM run: minerva -a start
• The start command will power on the FSVMs for all File Servers and start AFS services
• Power on all remaining guest VMs