VMware NSX Series – Data flow without control plane

In my last blog post (you can read it here: https://www.pascalswereld.nl/post/67365305981/nsx) I wrote about NSX architecture with the out of band components such as NSX manager and NSX controller cluster (management and control plane). But do they realy don’t interfere with the data IO?

Time to find out!

I am using the HOL NSX lab to show how this works. This is a preconfigured lab with a NSX manager, a NSX controller HA pair, edge router and some Linux based VM’s.

First we are setting up a logical switch, connecting this to the perimeter edge router and connecting VM’s to this switch.

The VM network Switch is creating.


Adding to the perimeter edge with an IP subnet declaration. And yes don’t forget to connect the port.


As you will notice the subnet is connected. The edge port is given the IP address.

Next up adding VM’s to this distributed logical network.


I am using two web servers that are currently at an other logical network. This action will move the VM’s from Web_Logical_Network to the created VM Network.

With SSH putty sessions to the VM’s we can verify that the VM’s have interfaces connected to this network.


We see both the VM’s in the configured subnet with web03 at address and web04 at address When we start a ICMP ping we can confirm data is flowing from one VM to the other, and we can confirm that traffic is flowing from one logical switch port to the other.


Okay now see how the traffic will flow after we shutdown the controller HA pair. We got to VM’s in the vCenter inventory.


Here you also notice the edge components.

With the shutdown guest OS operation we shut down both the NVP_Controller VM’s. This has the complete HA pair shutdown in effect.


After this we can retry our ICMP data flow.


And low and behold data IO is still flowing between web03 and web04. A ping back from web04 to web 03 show this way is also working.


This small example shows that the controller pair don’t interfere with already configured components in the data plane. You won’t even notice problems when wanting to add new VM’s to this logical switch. Let’s demonstrate with adding web02.



Network adapter 1 is connected to the VM network DVS. But why wouldn’t it? DVS is managed by vCenter and the host is already part of this DVS (for example web04 is running on the same host). At the IP address we can notice something wrong, the IP subnet of the guest (this is .30 instead of the .40). When opening /etc/sysconfig/network/ifcfg-eth0 there is a static IP configured, again elementary my dear Watson. Replacing the .30 with .40, and down and up the interface. Now ping is running.

But what will not work with the controllers down? For example creating a new logical switch will fail with a vCNS server error. There is no interaction from management plane to the hosts control plane components. There you need the controller as the work horse.

– This concludes this blog post.

VMware NSX Series – Introduction and components

This year VMware introduced some new solutions to the software defined data center (SDDC), namely Virtual SAN (or VSAN) for the storage and available solutions and NSX for the network and security layer. Or software defined storage resp. software defined networking.

Virtual SAN will be general available H1 2014. Beta has been released a while now, so there is plenty of opportunity to test this solution. I have done a little blog posting about the initial configuration at  https://www.pascalswereld.nl/post/62805854730/vsan-beta-part-what-install.

The other solution is NSX where I want to go in some deeper in this blog post. NSX is GA but you will have to contact VMware sales if you want something with NSX. But first a little SDDC.

Software Defined Data Center (SDDC)

So you have heard this SDDC term earlier. That is right, if you have been following the keynotes from this and last years VMworld you will have heard them. And if you are a regular visitor of vmware.com you will have seen even more of that. But what is meant with SDDC?


Software defined data center (SDDC) is an architectural model to IT infrastructures that extends traditional virtualization concepts to all of the data center’s resources and services. This started a decinia ago with the visualization of computing resources (CPU and memory) to provide server virtualization (the software server) as the base component of SDDC.
Software defined networking or network virtualization, is the process of merging networking resources and functionality into a software-based virtual network. This creates simplicity by creating virtual network component “free” of the underlying physical network and firewall architecture. Well free, you will still need some cabling and switching to go from you computing cluster to the edge and further. But these can be simplified by just providing hardware connectivity. Let the virtualization layer handle the connectivity of VM, tenants, routing and access control (just a few examples).
Software defined storage or storage virtualization, is simple shared storage specifically designed for virtual machines. by simple it is self tuning, easy provisioning, simple managed and dynamically scalaleble. It presents a single data store distributed across multiple hosts in a vSphere cluster (that is where VSAN is enabled)

If underlying hardware fails the virtualization layers automatically redirects workloads to other components in the data center as long as redundant paths exist.

A important reason for the SDDC is to simplify the provisioning of services and providers for application workloads. Yes, it adds more complexity to the virtualization layer, it is not just computing anymore. But it simplifies provisioning while not having to go from and to different IT service silo’s to get something done. Your expertise is there in the virtualization layers.

Well pretty clear isn’t…

Now for a little in about network virtualization via VMware NSX. Will try to keep it little as you can write a book about this subject. I don’t think I’m gonna be finished in one blog post, so I conveniently used series in my title. That is not a promise but a opening, as I am sure this subject will return.

VMware NSX Architecture

NSX is composed of the following components:


These bring components in the network/virtualization layers by means of virtual appliances, and components close to the hypervisor (on the host) components. As you will notice (or not) the switching supports the open vSwitch which allows NSX to be deployed with other hypervisors (and with other I mean other then VMware in this case). For example KVM, Xenserver can be supported/added to provide a true software defined data center, and not just a VMware software defined data center. For this you will have two flavouors of NSX, one optimized for vSphere and NSX for multi hypervisors.
But the question here is how many organizations use hybrid hypervisors in their environments. Often enough I will only see one flavor install base. But that is a case outside of the scope of this blog post. Back to NSX components.

An overview of the NSX components:

NSX Manager.  A web-based GUI management dashboard for user friendly interaction with the VMware NSX controller cluster. Via the NSX API. Primarily used for system setup, administration and troubleshooting. NSX Manager can take snapshots of the entire state of the virtual network for backup, restores, introspection, and archival. The services are provided via NSX API’s. The NSX manager works together with vCenter for managing cluster and host components.

NSX Controller. The NSX controller cluster is the highly available distributed system of virtual appliances responsible for the programmatic deployment of virtual networks across the entire architecture. The NSX controller cluster accepts API requests from cloud management platforms (e.g. vCloud, OpenStack), calculates the virtual network topology, and proactively programs the hypervisor NSX vswitches and NSX gateways with the appropriate configuration. While not handling packets directly, the controller cluster is the workhorse of the NSX infrastructure.

The NSX Manager and NSX Controller cluster are out of band and never handle data packets. Other way of definition are the NSX Manager is in the management pane (together with a vCenter system) and the NSX controllers are in the control pane of the network virtualization.

NSX Gateways/Edge Router. NSX edge services provide a secure path(s) in and out of the software defined data center. NSX Gateway nodes can be deployed in high available pairs, and offer services such as routing, firewalling, private tunneling, and load balancing services for securing and controlling traffic at the edge of one or more virtual networks. NSX gateways are managed by the controller cluster.

– NSX vSwitch. NSX vSwitch is an component that is added to the hypervisor and replaces the traditional switches. Well sort of, as there still is a distributed logical switch layer but now the NSX vSwitch or Open vSwitch. It can span multiple clusters and provide for example layer 2 and layer 3 logical switching.

– Host loadable modules. Most networking components use the host provided modules. For example to let a host understand the NSX switch and let traffic flow between NSX hosts they need to talk the same language. With the kernel modules your ESXi host is able. The installation of modules can be done using the UI or by bundling the vSphere image with proper VMware Installation Bundles (VIBs). These modules provide port Security, VXLAN, distributed firewall (DFW), distributed switching or distributed router (DR) functions on the host level.


Okay that is enough theory done for this blog post.

Would you like some hands on? VMware has some hands on lab (HOL) sessions on the NSX subject. Take these labs at at http://labs.hol.vmware.com/ (or www.projectnee.com). You can choose or do both the  HOL-SDC-1303 – VMware NSX: The Network Virtualization Platform and HOL-SDC-1319 – VMware NSX for Multi-Hypervisor Environments sessions.

– Interesting this network virtualization. To be continued for sure.