VMware: NSX: Logical Switch Objects as Source and Destination Firewall rules (DFW) not working and things to know.


Summary:
One of the cool things w/ NSX is that you can define vCenter Objects as your source and destination targets in NSX's Distributed Firewall (DFW).  On the surface, great idea.  Under the hood, it is still based on IP source and targets.  Only when you decide upon a default deny/deny rule does this become apparent.

Quick and Easy:
  1. Make sure VM's are running VMware Tools.
  2. VM(s) in question is NOT in NSX Exclusion List
    1. ESX Agents (aka NSX Partner Integrations) appear to be in exclusion list by default.
    2. ESX Agents do NOT show in NSX Exclusion list UI.
  3. Use IP Sets if you are a masochist.
    1. or if a vendor provides an appliance w/o tools installed.
    2. Honestly, IP Sets make the most sense rather than defining vSphere objects.  The reason?  If you deal w/ something like an appliance that can be deleted and recreated easily, a vsphere object rule would break because the original object no longer exists.
Details:


 For my example, we decided to allow all L2 traffic.  To do so, we made a set of global rules that simply put a logical switch as a source and destination w/ any/any.  While this makes perfect sense, this only works when VM's on that logical switch are running VMware Tools.

The reason?  DFW still has to work w/ IP addresses to work properly.  So it does a clever, (clever being up for debate) little trick by simply populating an IP address set table as part of that logical switch object.  It does so by getting the IP address information provided by VMware tools.

No VMware tools, no IP address to populate a logical switch address set.

So what happens in this case is that a VM placed on this logical switch w/o VMware tools running would not 'match' this rule.  So even if this VM w/o VMware tools were to run on the same logical switch as a VM that is running VMware tools, it would not be able to communicate w/ its neighbor assuming a default deny rule.

Another interesting tidbit, VM's w/ VMware tools that are placed into NSX's Exclusion list would also NOT populate IP address sets in NSX Objects.

Meaning, if I were to place a VM object that had vmware tools but also part of NSX Exclusion list as a destination and another VM object w/ VMware tools as a destination the dvfilter (DFW) would not see a match because it would not know the IP address of the source.

Below commands are run from NSX Manager.

  • show dfw vm vm-####
    • Replace vm-#### w/ VM's ID in question.  You can use powerCLI to get a VM's ID info.
      • (get-vm nameofVMinQuestion).extensiondata.moref.value
    • This will return VM's host ID and associated dvfilter info needed for next command.
  • show dfw host host-## filter nic-#######-eth#-vmware-sfw.# addrsets
    • Replace host-## and nic-#######-eth#-vmware-sfw.# w/ info found from first command.
You'll get a list that you can scroll through.  If the source/destination were a logical switch, you'd be looking for something like this:
addrset ip-virtualwire-4 {
192.168.2.4,
192.168.2.6,
}
 If 1 of 3 VM's did not have VMware tools running or was part of NSX exclusion list, the above addrset would reflect behavior correctly.


Comments

Post a Comment

Popular posts from this blog

NSX-T: Release associated invalid node ID from certificate

Image
Summary: Basically had an expiring certificate registered in NSX-T that was associated to a node_id that is no longer valid.  Long story short, there wasn't anything obvious in API to delete or disassociate a certificate from a node_id for 3.2.2.  Not sure how things got in this state, but annotating for future reference.  This may change in future revisions, so always check API for latest. Details: Effectively had a stale node associated w/ a certificate that was expiring.  Could not delete certificate until that node was disassociated from the certificate. To get certificate details and associated node_id's, you can use the following curl call (UI works too): curl -k -X GET -H "Content-Type: application/json" -u admin https://<manager ip>/api/v1/trust-management/certificates/<cert UUID> Above will return something like this: Below must be run from one of the manager nodes via elevation to root: ONLY RUN THIS IF YOU ARE ABSOLUTELY SURE OF WHAT YOU ARE DO...
Read more

NSX-T: vCenter and NSX-T Inventory out of Sync (Hosts in vSphere not showing up in NSX-T)

Image
Summary: NSX-T loses synch w/ vCenter inventory, but statuses don't appear to show an issue.  Basically, you add a host to a vCenter cluster, NSX-T bits should start to automatically installing on new host.  Assuming you've created a Transport Node Profile and associated w/ the cluster.  The problem is that NSX-T doesn't see the new host and its link to the compute manager (vCenter) looks fine. Looks fine, Y U NO WORK!? So what's going on here?  This appears to affect NSX-T 2.5 and 2.5.1.  Cause is unknown. Workaround: Restart the cm-inventory service on each NSX-T mgmt/controller node using API or CLI. Details: If you were to query the status of the cm-inventory via API or CLI, you could query all 3 manager/controller nodes and get a status of running.  Even if the primary node associated w/ the VIP, if configured, is not necessarily in charge of inventory.  So you could restart the cm-inventory service till you are blue in the face ...
Read more

MacOS: AnyConnect VPN client was unable to successfully verify the IP forwarding table modifications.

Image
The VPN client was unable to successfully verify the IP forwarding table modification.  A VPN connection will not be established. Summary: I started running into issue utilizing Cisco AnyConnect on my Mac basically complaining about not able to overwrite IP forwarding tables.  This was on 4.6.x.  Since my VPN endpoints were not providing me w/ an updated client and w/ no access to Cisco Anyconnect downloads, my only option was to try openconnect.  It was totally worth it, here is why and how to set it up. PreReqs: Homebrew Installing OpenConnect: Launch MacOS Terminal brew install openconnect Getting VPN IP's/DNS Endpoints from AnyConnect: The information is typically located in your profile xml files located here: /opt/cisco/anyconnect/profile/somethingsomething.xml In the xml file, you are looking for "<HostAddress>typicallyaDNSName.com</HostAddress>" entry.  These are your VPN endpoints that you would need to pass to openconnec...
Read more