VMware NSX Cookbook

The four tiers of licenses are as follows:

The features available in each edition are as follows:

There are two ways to evaluate VMware products:

There are support and subscription plan options that you can purchase in addition to the product:

The production support plan is recommended for production and critical environments. If you need higher-level support above production grade, additional support options such as Business Critical Support (BCS) or Mission Critical Support (MCS) can be purchased on top of production support. For more information on VMware support offerings, see https://www.vmware.com/support/services.html.

VMware vRealize Log Insight is a log management engine that collects logs from a number of different sources and provides rich dashboards and search functionality.

Log Insight is available for NSX at no additional charge, you are entitled to one Log Insight CPU per NSX CPU license. The support and subscription is included with the NSX purchase. It is a fully functioning version of Log Insight but limited to vSphere and NSX data sources and content packs only. If you need more data sources and content packs, additional Log Insight licenses are required.

There are several tools for monitoring VMware NSX. Some of these tools are built directly into the NSX platform, and others are separate feature-rich VMware products. These tools are as follows:

In this section, we will check to make sure the version of NSX we are deploying is compatible with the other VMware solutions in our environment.

The following screenshot shows the official VMware product interoperability matrices that should be referenced before downloading NSX for vSphere:

In this section we will download the installation media from the VMware downloads website as follows:

In this section, we will download the VMware NSX installation media using the VMware Software Manager, in contrast to a manual download via the downloads website:

Importing PKCS#12 into the NSX Manager is used when the certificate signing was not completed using the CSR method outlined in the previous recipe. The PKCS#12 format is typically used in scripted installations of NSX Manager and other components. If a CSR was not generated by the NSX Manager itself, it is required that the PKCS#12 archive is imported into NSX Manager.

The PKCS#12 archive generally consists of the following:

The PKCS#12 is also password-protected, so it's important to have the password before attempting to import the PKCS#12 archive into NSX Manager.

In some cases, the received signed certificate may not be in the PCKS#12 format. In this event, you must convert the certificates into the PKCS#12 format for import into the NSX Manager. This can be achieved using openSSL (https://www.openssl.org/), and the command to achieve this is as follows:

openssl pkcs12 -export -out server.p12 -inkey server.key -in server.crt -certfile CACert.crt

The following are the steps to pair the NSX Manager with the vCenter server:

In this section we will register the NSX Manager with the Platform Services Controller for Single Sign-On services:

Before deploying the NSX controller cluster, an IP pool must be configured to reserve three IPv4 addresses on the network:

The IP Pool can be configured during deployment of the NSX controller cluster as well in the event it is not configured beforehand.

In this section, we will deploy each of three NSX Controllers on our vSphere cluster:

Once all the controllers have been deployed, you should see the following displayed under the Installation tab in Networking & Security, with the green boxes indicating healthy connectivity between each of the peers in the controller cluster:

DRS Anti-Affinity Rules

DRS anti-affinity rules are required to ensure that the NSX controllers do not reside on the same physical host and are kept separate on dedicated ESXi hosts. This is to ensure in the event a ESXi host goes down where all three controllers are potentially running as guest VMs, the entire control plane for logical networking is not lost. If two controllers are lost, then the remaining controller goes into read-only mode until a cluster majority is restored.

It's important to note that the underlying infrastructure should still be designed for HA and resiliency, which includes compute/network/storage.

Configuring DRS anti-affinity rules via the vSphere web client:

1. In the vCenter Web Client, navigate to Hosts and Clusters | Management Cluster | Manage | Settings | VM/Host Rules
2. Click on Add...
3. Choose Type as Separate Virtual Machines

4. Add NSX controller virtual machines and click on OK:

$nsx_controllers = get-vm | ? {$_.name -like "NSX_Controller*"}

New-DrsRule -Name nsx-controller-anti-affinity -Cluster RegionA01-MGMT01 -KeepTogether $false -VM $antiAffinityVMs

The controller cluster is deployed in a group of three. Each controller node can only be deployed onto a vSphere cluster that is part of the vCenter inventory that the NSX Manager you are configuring is paired with. In large environments with multiple vCenters, it is not uncommon for the vCenter server and NSX Manager to be deployed onto a dedicated vSphere cluster that is managed by an independent vCenter server that is deemed as management. In this scenario, the NSX controller cluster cannot be deployed onto the dedicated management vSphere cluster.

The NSX controller password must meet the following criteria:

In the event that the NSX controller password is forgotten, it can be easily changed using the following steps:

When enabling a vSphere cluster for NSX in a brownfield environment, it is important to be cognizant that any preconfigured DFW firewall rules have the potential to impact virtual machines on the newly-configured vSphere cluster. Therefore, it is extremely important to ensure that the default Distributed Firewall rule remains as allow any any. Changing to deny before defining rules for allowing legitimate traffic from/to virtual machines will cause traffic blackholing. 

As a best practice, vCenter server and virtual machines that require promiscuous mode should be excluded from the DFW if you are not planning to protect them. To learn how to exclude virtual machines from the DFW, refer to Chapter 6, Configuring VMware NSX Distributed Firewall (DFW) and SpoofGuard.

In this section we perform manual verification that the VIBs have been successfully installed.

esxcli software vib get --vibname esx-vxlan

[root@vSphere:~]    esxcli software vib get --vibname esx-vxlanVMware_bootbank_esx-vxlan_6.0.0-0.0.4987429   Name: esx-vxlan   Version: 6.0.0-0.0.4987429Type: bootbankVendor: VMwareAcceptance Level: VMwareCertifiedSummary: Vxlan and host toolDescription: This package loads module and configures firewall for vxlan networking. ReferenceURLs:Creation Date: 2017-01-27Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0Conflicts: nsx-api = 2Replaces: esx-traceflow, esx-dvfilter-switch-security, esx-bfdProvides: com.vmware.vxlan = 1.0.0.0-nsx, com.vmware.switchsecurity = 1.0.0.0, com.vmware.traceflow = 1.0.0.0, com.vmware.bfd = 1.0.0.0Maintenance Mode Required: TrueHardware Platforms Required:Live Install Allowed: TrueLive Remove Allowed: TrueStateless Ready: TrueOverlay: FalseTags:Payloads: esx-vxlan

esxcli network ip connection list | grep 1234

An example is included below that shows the connection as established to each of the three NSX controllers from the point of view of an ESXi host:

cat /etc/vmware/netcpa/config-by-vsm.xml

The following screenshot provides a truncated output of the command and its expected output.

[root@vSphere:~] esxcli software vib get --vibname esx-vsipVMware_bootbank_esx-vsip_6.0.0-0.0.4987429   Name: esx-vsipVersion: 6.0.0-0.0.4987429Type: bootbankVendor: VMwareAcceptance Level: VMwareCertifiedSummary: vsip moduleDescription: This package contains DFW and NetX data and control plane components.ReferenceURLs:Creation Date: 2017-01-27Depends: esx-base >= 6.0, esx-base <= 6.5.0, nsx-api <= 2.1, vmkapi_2_3_0_0Conflicts: nsx-api = 2Replaces: esx-vdpi Provides: vsip = 1.0.0-0Maintenance Mode Required: TrueHardware Platforms Required:Live Install Allowed: TrueLive Remove Allowed: TrueStateless Ready: TrueOverlay: FalseTags:Payloads: esx-vsip

esxcli network ip connection list | grep 5671

The following screenshot provides an example of the above command, the output shows a connection of established to the NSX Manager over TCP Port 5671:

esxcfg-advcfg -g /UserVars/RmqIpAddress

The following screenshot shows the results of the execution of the command above, and the expected configuration parameter is the IP address of the NSX Manager:

To check the communication channel between NSX Manager, NSX controller cluster, control plane agent, and Distributed Firewall agent, follow the following procedure:

The following screenshot displays the result of the preceding action and the ESXi communication health: