Nov
03

Acropolis Networking – Remove 1 Gb From The Bond

Every time a cluster gets destroyed for the next demo or next customer I find myself looking on the Nutanix Portal and searching how to remove 1 Gb nic from the default bond0. What alerts me to this call to action is the warning in Prism saying that the CVM is running on 1 Gb. This happens because IPMI use to be a requirement and there is a shared IPMI networking port. The default IPMI port is only 100 Mb on the older 3000 systems so you can use the shared port for IPMI which 1 Gb. The shared port will also serve other network from the hypervisor. I don’t really find to many customers using 1 Gb at all unless they are at robo site. A 10 Gb switch can be picked up for less than $5,000 bucks so price is usually not the issue but politics sure can be 🙂

If you log into anyone of the CVM’s you can quickly remove all of the 1 GB from the default bond.

Note: eth2 & eth3 are the 10 Gb links and eth1 & eth0 are the 1 Gb links.

Commands
manage_ovs show_uplinks -> shows which nics are in the bond for that host
manage_ovs –bond_name bond0 –interfaces 10g update_uplinks -> Only uses the 10 Gb uplinks for the bond
allssh manage_ovs –bond_name bond0 –interfaces 10g update_uplinks -> Adding the allssh to the command will run the command on all hosts in the cluster.


nutanix@NTNX-14SM15510075-A-CVM:10.20.27.29:~$ manage_ovs show_uplinks
Uplink ports: bond0
Uplink ifaces: eth3 eth2 eth1 eth0
nutanix@NTNX-14SM15510075-A-CVM:10.20.27.29:~$ manage_ovs --bond_name bond0 --interfaces 10g update_uplinks
2015-11-03 18:47:05 WARNING manage_ovs:298 Interface eth3 does not have link state
nutanix@NTNX-14SM15510075-A-CVM:10.20.27.29:~$ manage_ovs show_uplinks
Uplink ports: bond0
Uplink ifaces: eth3 eth2

allssh manage_ovs --bond_name bond0 --interfaces 10g update_uplinks
Executing manage_ovs --bond_name bond0 --interfaces 10g update_uplinks on the cluster
================== 10.20.27.29 =================
2015-11-03 18:49:27 WARNING manage_ovs:298 Interface eth3 does not have link state
================== 10.20.27.30 =================
2015-11-03 18:49:34 WARNING manage_ovs:298 Interface eth3 does not have link state
2015-11-03 18:49:34 INFO manage_ovs:341 Deleting OVS ports: bond0
2015-11-03 18:49:35 INFO manage_ovs:349 Adding bonded OVS ports: eth3 eth2
2015-11-03 18:49:37 INFO manage_ovs:411 Sending gratuitous ARPs for 10.20.27.26
================== 10.20.27.31 =================
2015-11-03 18:49:46 WARNING manage_ovs:298 Interface eth3 does not have link state
2015-11-03 18:49:46 INFO manage_ovs:341 Deleting OVS ports: bond0
2015-11-03 18:49:47 INFO manage_ovs:349 Adding bonded OVS ports: eth3 eth2
2015-11-03 18:49:49 INFO manage_ovs:411 Sending gratuitous ARPs for 10.20.27.27

Apr
10

What Would Tony Do? vSwitch0 on Nutanix

Tony Holland – Nutanix Sr SE with a common sense approach to infrastructure.

Q: What are the default vSwitch0 settings for a new install of 3.5.3.1. What physical adapters are included in vSwitch0? Also, what is the default teaming policy?

All NICS are part of vSwitch0

Default teaming is the 2 10GbE’s are active and 2 1Gb’s are standby. I use the 1 Gb’s for setup and then after simply unplug them.

I always create a new Port Group under vSwitch0 called Nutanix.

I then will change the NIC teaming to have VMnic0 be active and all the others standby, and for VM Network port group move VMNIC1 to active and the others standby. Has worked well for me doing this setup.

-Tony