NSX-T: vCenter and NSX-T Inventory out of Sync (Hosts in vSphere not showing up in NSX-T)


Summary:
NSX-T loses synch w/ vCenter inventory, but statuses don't appear to show an issue.  Basically, you add a host to a vCenter cluster, NSX-T bits should start to automatically installing on new host.  Assuming you've created a Transport Node Profile and associated w/ the cluster.  The problem is that NSX-T doesn't see the new host and its link to the compute manager (vCenter) looks fine.
Looks fine, Y U NO WORK!?

So what's going on here? 
This appears to affect NSX-T 2.5 and 2.5.1.  Cause is unknown.

Workaround:
Restart the cm-inventory service on each NSX-T mgmt/controller node using API or CLI.


Details:
If you were to query the status of the cm-inventory via API or CLI, you could query all 3 manager/controller nodes and get a status of running.  Even if the primary node associated w/ the VIP, if configured, is not necessarily in charge of inventory.  So you could restart the cm-inventory service till you are blue in the face and get nowhere because another node is actually responsible for maintaining the sync. 

Even so, with this particular problem, they would all look healthy.  You'd have to dig into the logs to find the issue.  Thankfully Nathan Pyle from VMware helped us find the issue and provided us w/ a workaround solution until the bug is addressed in a future version.

API Method:
GET /api/v1/node/services/cm-inventory/status
POST /api/v1/node/services/cm-inventory?action=restart

CLI Method:
get service cm-inventory
restart service cm-inventory 

Comments

Post a Comment

Popular posts from this blog

RacADM, Powershell, and Kerberos KeyTabs

Summary: I was looking to automate configuration of iDRAC's (Dell Remote Admin Console) synonymous w/ iLOM and many other acronyms.  I also wanted to enable domain authentication in a somewhat secure manner.  By using a kerberos keytag, I'm able to in the very least obfuscate the username and password into a file for upload into the iDRAC.  What follows is a journey into a ridiculous realm. PreReqs: RACADM ktpass Powershell 3.0 (2.0 probably works too, but I was using 3.0 @ the time) Your root CA's cert This is so it AD Authentication can be setup Solution: It's not a complete solution, but hopefully it'll help you get started in your own endeavors.  Below assumes the iDRAC has been configured and is available on the network in default config. To generate a kerbero's keytab , you simply need the name and password of a service account (this is simply a Active Directory user account) in your domain. ktpass /pass ReallySecurePas5word /ptype K
Read more

MacOS: AnyConnect VPN client was unable to successfully verify the IP forwarding table modifications.

Image
The VPN client was unable to successfully verify the IP forwarding table modification.  A VPN connection will not be established. Summary: I started running into issue utilizing Cisco AnyConnect on my Mac basically complaining about not able to overwrite IP forwarding tables.  This was on 4.6.x.  Since my VPN endpoints were not providing me w/ an updated client and w/ no access to Cisco Anyconnect downloads, my only option was to try openconnect.  It was totally worth it, here is why and how to set it up. PreReqs: Homebrew Installing OpenConnect: Launch MacOS Terminal brew install openconnect Getting VPN IP's/DNS Endpoints from AnyConnect: The information is typically located in your profile xml files located here: /opt/cisco/anyconnect/profile/somethingsomething.xml In the xml file, you are looking for "<HostAddress>typicallyaDNSName.com</HostAddress>" entry.  These are your VPN endpoints that you would need to pass to openconnect. Using
Read more