VMware: NSX: Logical Switch Objects as Source and Destination Firewall rules (DFW) not working and things to know.

One of the cool things w/ NSX is that you can define vCenter Objects as your source and destination targets in NSX's Distributed Firewall (DFW).  On the surface, great idea.  Under the hood, it is still based on IP source and targets.  Only when you decide upon a default deny/deny rule does this become apparent.

Quick and Easy:
  1. Make sure VM's are running VMware Tools.
  2. VM(s) in question is NOT in NSX Exclusion List
    1. ESX Agents (aka NSX Partner Integrations) appear to be in exclusion list by default.
    2. ESX Agents do NOT show in NSX Exclusion list UI.
  3. Use IP Sets if you are a masochist.
    1. or if a vendor provides an appliance w/o tools installed.

For my example, we decided to allow all L2 traffic.  To do so, we made a set of global rules that simply put a logical switch as a source and destination w/ any/any.  While this makes perfect sense, this only works when VM's on that logical switch are running VMware Tools.

The reason?  DFW still has to work w/ IP addresses to work properly.  So it does a clever, (clever being up for debate) little trick by simply populating an IP address set table as part of that logical switch object.  It does so by getting the IP address information provided by VMware tools.

No VMware tools, no IP address to populate a logical switch address set.

So what happens in this case is that a VM placed on this logical switch w/o VMware tools running would not 'match' this rule.  So even if this VM w/o VMware tools were to run on the same logical switch as a VM that is running VMware tools, it would not be able to communicate w/ its neighbor assuming a default deny rule.

Another interesting tidbit, VM's w/ VMware tools that are placed into NSX's Exclusion list would also NOT populate IP address sets in NSX Objects.

Meaning, if I were to place a VM object that had vmware tools but also part of NSX Exclusion list as a destination and another VM object w/ VMware tools as a destination the dvfilter (DFW) would not see a match because it would not know the IP address of the source.

Below commands are run from NSX Manager.

  • show dfw vm vm-####
    • Replace vm-#### w/ VM's ID in question.  You can use powerCLI to get a VM's ID info.
      • (get-vm nameofVMinQuestion).extensiondata.moref.value
    • This will return VM's host ID and associated dvfilter info needed for next command.
  • show dfw host host-## filter nic-#######-eth#-vmware-sfw.# addrsets
    • Replace host-## and nic-#######-eth#-vmware-sfw.# w/ info found from first command.
You'll get a list that you can scroll through.  If the source/destination were a logical switch, you'd be looking for something like this:
addrset ip-virtualwire-4 {,,
 If 1 of 3 VM's did not have VMware tools running or was part of NSX exclusion list, the above addrset would reflect behavior correctly.

No comments: