At an implementation project we implemented 24 ESXi hosts and used the Nexus 1000V to have a consistent network configuration, feature set and provisioning throughout the physical and the virtual infrastructure. When I tried to add the last host to the Nexus it failed on me with the InstallerApp (Cisco provided java installer that adds the VEM and adds the ESXi host to the configured DVS and groups). Another option is to use Update Manager (the Nexus is a appliance that can be updated by update manager), but that one threw an error code at me. I will describe the symptoms a little bit later, first some quick Nexus product architecture so you will have a bit understanding how the components work, where they are and how they interact.
Nexus 1000V Switch Product Architecture
Cisco Nexus 1000V Series Switches have two major components: the Virtual Ethernet Module (VEM), which runs inside the hypervisor (or with other words on the ESXi host), and the external Virtual Supervisor Module (VSM), which manages the VEMs (see figure below). The VSM can be either a virtual appliance or an appliance in the hardware device (for example in the physical Nexus switch).
The Nexus 1000V replaces the VMware virtual switches and adds a Nexus Distributed Switch (Enterprise Plus). Uplink and portgroup definitions are bound to the Cisco ethernet profile configurations. The VEM and VSM use a control and data link to exchange configuration items.
Configuration is performed through the VSM and is automatically propagated to the VEMs. Virtualization admins can pick up these configuration to select the portgroups at VM provisioning.
Symptoms to a failing installation
The problem occurred as follows:
– Tried to install the VEM with the InstallerApp. The installer app finds the host and when the deployment is done, it stops when adding the hosts to the existing Nexus DVS. This happens somewhere from moving existing vSwitches to the DVS. Error presented is: got (vim.fault.PlatformConfigFault) exception.
– Checked the status of the host in update manager and this showed a green compliant Cisco Nexus Appliance. This probably delayed me a bit, because it really wasn’t.
– Tried to manually add a host to the Nexus DVS in the vSphere Webclient. This gave an error in the task. Further investigation let me to the line in the vmkernel.log: invalid Net_Create: class cisco_nexus_1000v not supported. Say what?
– With the Cisco support site and a network engineer I tried some VEM cli on the host. But hey wait vem isn’t there. An esxcli software vib list | grep cisco doesn’t show anything either (duh). While on an VEM installed ESXi host this shows the installed VEM software version. So Update Manager is screwing with me.
That leaves me with trying to manually install the VEM. With the Cisco Nexus 1000 installation guides the working sollution is as follows:
- The preferred Update Managers does not work. It fails with an error 99.
- copy the vib file containing the VEM software from the VSM homepage using the following url: http://Your_VSM_IP_Address/. Check an ESXi host that is installed for the running version (esxcli software vib list | grep cisco). Download this file (save as).
- Upload this file to a location where the host can access this. On the host or a datastore accessible from the host. I did the latter as the host did not have direct storage. Used WinSCP to transfer the files to a datastore directory ManualCiscoNexus.
- On the hosts I added this vib by issuing the following command:
esxcli software vib install -v /vmfs/volumes/<datastore>/ManualCiscoNexus/Cisco_bootbank_cisco-vem-v160-esx_18.104.22.168.2.1.0-3.1.1.vib
- vem status -v now gives output. Look for VEM Agent is running in the output of the vem status command.
- vemcmd show port vlans only shows the standard switches. Communication with the VSM is not yet there.
- I added the host manually to the Nexus DVS and success. When migrating the standard vmkernel management port to the DVS groups the hosts is also visible on the VSM. Communication is flowing and the host is part of the Nexus 1000v.
I hope this post will help when you experience the same problem, and also learns you a little about the Nexus 1000V Product Architecture.