Showing posts from 2014

Error: Issues information is not available at this time. (vCenter Appliance)

The above was a wonderfully descriptive error I would receive in the web client.  This started appearing after I removed individual nodes from a cluster and readded them back to inventory.  (was trying to clear some bunk HA errors)

Long story short, I had to open the C# client to find out what 'warning/error' it was complaining about.  Quite simply, I had forgotten to re-add the hosts back into the vDS and migrate the vMotion/vSAN vmk connections.

Man-o-man, I see why so many people complain about the web client.

The upside to the story was I found out a couple of things about vSAN.

Having a 3-node cluster minimum, I was able to remove and re-add each host back into the cluster with no issues to VM's running on each host.Even w/ the vDS problem, vSAN kept chugging along w/ VM's running just fine.In other words, even without vCenter managing things, the ESXi hosts kept everything running just fine.This was on my mac mini vSAN cluster.

vCloud Air On Demand (Beta Impressions)

It's definitely beta and I'm not a big fan of some aspects of the implementation, specifically that I might find myself jumping between this and the provided vCloud director a lot.  It's not all bad though.  Click through if you'd like to read my overview/opinion of the service.

Outlook 2013 (Office 365) for Mac Clock Error

Summary: Ran into this wonderful error while traveling.  Thankfully, I still had Outlook 2011 still installed, so I had a workaround for a bit.

Details: This would occur for no obvious reason, although it does seem related to 'travel' related activities when switching time zones.  Outlook would go into offline mode and would require that I activate it first, activation would fail w/ the above error.  If I remember correctly, I had this happen to me two about a month ago as well, but the activation error went away on it's own after I flew back to my timezone.
This time, I screwed around w/ it, and well I probably broke it while trying to fix it before going back to my home time zone.  One of which was deleting the computer from my activated list as a troubleshooting step.  Anyway, Microsoft support on a communities board finally posted a solution, but it was difficult to figure out, so here is a 'translated' step by step.  Specifically step 5 wasn't that clear to…

vCenter Server Virtual Appliance and Native Active Directory, back to the basics...

I had a rather interesting issue w/ my vCenter Server Appliance (vCSA).  I had it natively joined to my Active Directory domain so I could use it as a native identity source in SSO.  I was running into a rather curious issue where I would add a user to the SSO Administrators group, everything would appear to happen correctly, but the table showing group membership would be blank.

Searching for users in any domain would work fine.  Just not the membership table (Group Members).

Line to capture LDAP port packets from within vCenter appliance (389 for LDAP, 636 for LDAPS):
tcpdump port 389 -v -w /tmp/pktcapturefile.log
tcpdump port 636 -v -w /tmp/pktcapturefile.log

LDAPS packet capture is somewhat not useful because the packets are encrypted.  The best you can really tell is whether packets are communicating to whom.

Come to found out, after working w/ VMware, looking @ a packet trace, it so happened that the user search dialog was querying a domain controller that had DNS PTR records.

Packt’s $5 eBonanza returns (Sponsored Post)

Following the success of last year’s festive offer, Packt Publishing will be celebrating the holiday season with an even bigger $5 offer.   From Thursday 18th December, every eBook and video will be available on the publisher’s website for just $5. Customers are invited to purchase as many as they like before the offer ends on Tuesday January 6th, making it the perfect opportunity to try something new or to take your skills to the next level as 2015 begins. With all $5 products available in a range of formats and DRM-free, customers will find great value content delivered exactly how they want it across Packt’s website this Xmas and New Year.

GeekTool: GeekWeather2 w/ auto geo-location...

If you haven't downloaded geektool for the Mac, I would definitely recommend doing so for the 'tinkerers' out there.  I first saw it on LifeHacker and have been somewhat infatuated with it.  Here is what one of my mac's looks like:
Since the bash command line outputs can be downloaded as geeklets and are relatively easy, I'm going to focus on how to setup the GeekWeather2 geeklet.  In addition to setting that up, I'll also show you how I was able to get it to update the image based on my current location rather than statically set Longitude, Latitude parameters.  (Geeklet is simply another term for a widget specific to geektool.)  Continue through to the page break to learn more.

JXplorer, ADSIEdit equivalent tool that is open source, usable on a Mac.

Was turned on to this tool by VMware's GSS for troubleshooting a relatively minor issue that I've been having w/ the VMware vCenter Server Appliance.  Specifically around the native Active Directory integration.

Anyway, I like the tool since it's just another thing I can use w/o having access to a Windows system to find Active Directory information.  Although, it may be difficult figuring out your DN w/o Active Directory Users and Computers (ADUC) or ADSIEdit to make the initial bind connection.

Here is a screenshot of how to connect:

vRealize Operations @ (Cox Automotive)

My customer testimonial for VMware's vRealize Operations and Log Insight offerings.  Feel free to e-mail over any questions to me:

Cox Automotive

Mac: Background flashes, Dock not showing up...

Not sure what caused this as I hadn't done anything unusual w/ my Mac for awhile.  Long story short, some preference in my user profile was causing this issue.

In particular, the thought it was my dock preferences, but I had to delete my everything under my user's ~/Library/Preferences to get my account back and working properly.  You should have only to do the following:

Delete ~/Library/Application Support/Dock
Delete ~/Library/Preferences/

Here is screenshot by screenshot on how to do this:

Interview w/ William Lam on Mac Mini vSAN (MacCloud)

Enjoyed my conversation w/ William talking about my ghetto MacCloud setup.  You can read the interview here:

Some things people have asked me for clarification:

We are using Mac Mini 'Server' versions.  These have two drives by default.You can get a kit to add a second drive to the standard mac mini version.Mgmt and VM traffic flows over Standard vSwitch0 (Onboard as Uplink)vSAN and vMotion flows over dVS (Thunderbolt as Uplink)Onboard 1Gb and Thunderbolt 1Gb adapterBooting to USB thumb drive plugged into the back of the Mac Mini w/ ESXi.vCenter is a vCSA built and running on another vCenter instance.  Here is what it looks like configured:

Converged Networking Perils...

Had a wonderful experience where a P2V VM w/ bonded NIC's brought down several of our ESXi hosts.  HA compounded the problem by powering up the VM on other hosts once the host w/ this VM was brought down.  The perils of converged networking and why it's important to keep your ESXi management/storage separate from your other physical ports.  If these were 'physically' separate, the problem would have isolated to one host and prevented the cascading HA events.

Here is the config in short:
Dell Blade two nPar'd 10Gb ports --> Internal Dell I/O aggregator ports --> External Dell I/O aggregator ports --> Nexus 5K

Management, vMotion, NFS, AND VM traffic go over these two ports.

One port goes over Fabric A, the other over Fabric B.  Two physically separate uplinks.

What happened:
VM w/ bonded NIC's comes online.  This seemed to cause a 'spanning-tree' like event which caused the Internal Dell I/O aggregator ports to go into an 'error-disab…

#freeITBM VMware ITBM Free? (Opinion)

So lately there has been more discussion around the office whether we should move workloads to the 'cloud'?  AWS being the obvious 800lb gorilla.  I recently attended an AWS Essentials training and came out of it really impressed w/ their offering.  So much so, I thought, 'yeah, it might be time to diversify out of my VMware only mindset.'

That being said, 'cost' is a huge factor.  Not to mention security and a slew of other things, but we'll focus on cost being the topic.  How in the world do you calculate cost?  VMware had Chargeback, but that tool was a pain and quite frankly useless.  Now they have ITBM which is a very simplified tool @ it's core, but has some pretty impressive capabilities.

Amazon has a calculator, but honestly, I feel like that it is more than likely skewed in favor of AWS.  So this leads me to the idea that VMware needs to take the "Progressive" approach of 'compare' our prices to our competitors and choose what&…

Amazon Terms translated to VMware (How I understand them anyway)

Feel free to comment.  I'm just taking a simplified view.

Amazon EC2 = VM or vSphere Amazon EBS (Elastic Block Store) Volume = vmdk / RDM Amazon S3 = NAS / Windows File Share / NFS Volume / Etc. Amazon AMI = vSphere Template/OVF (Basic OS w/ or w/o applications pre-installed) Amazon RDS = Managed RDS DB (No OS or SQL server to manage) just DB instance. Amazon Cloudwatch = vCenter Operations Manager / HP Insight / SiteScope / Monitoring / Etc. Amazon Elastic Beanstalk = vFabric / Pivotal? Amazon CloudFormation = vApp / VMware vCloud Automation Center / VMware Orchestrator (Granular Blueprint)

Useful Powershell .NET method to convert DNS name to IP

Just thought this was useful, so decided to post it as a reminder for myself or anyone else really.
The reverse being:
Here is how I used it w/ PowerCLI:
$VMHosts = Get-VMhost $VMHosts | select Name, @{Name="IP";Expression={[]::gethostaddresses($_.Name)}}
This information can be particularly useful if you need IP's for firewall rule tables. Source:

VMWorld 2014 - Are you attending?

If not, I highly recommend you register.  Not only is it a fun time, it's probably one of the best 'training' experiences you can get.  In combination w/ the hands-on labs, you'll have access to some of the industries best known experts for immediate answers to your pressing questions.  VMware is also offering 50% off their VCP/VCAP examinations if taken while onsite.

Registration Link:

Letter to convince your Boss:

VMUG Advantage Discount:
By joining the VMUG Advantage program, you can take off $100 of registration among other benefits.

For those who are attending:
VMUnderground -
VMWorld 2014 5k Fun Run -

Recommended Session:
INF1601 - Taking Reporting and Command Line Automation to the Next Level with PowerCLI
Always a hit!

Swingbench, OEWizard.bat, and Java fun...

I'll keep this short.  Using Java 7 (1.7) seems to cause this issue:
Error occurred during initialization of VM
Could not reserve enough space for object heap

Upgrading/Installing Java 1.8 "JDK" seems to fix the above issue.  Adding -Xmx<#>m would not fix the issue either.

Anyway, figured I'd post this since I really couldn't find an answer to this bizarre issue.

Get-VIEventsFaster PowerCLI function...

[Update: Come to find out LucD wrote a Get-VIEventPlus function that I did not find before making mine.  He adds some other switches that might be useful, same idea though.  Just tells me I really should keep up on his blog more often.]

I wrote this function so I could replace the Get-VIEvent cmdlet in my scripts.  The great part is that the output ends up being roughly the same as the cmdlet so just changing my calls to this custom function speeds everything up like 100-fold.  That might be an exaggeration, but yeah, it's WAY faster.

It's written so that most get-vievent common switches are included.  Syntax is 'slightly' different, but usable so you don't have to modify any get-vievent calls too heavily.  I added a 'type' switch which will help shave off even MORE time.  Anyway, feel free to give it a try or let me know if you have any feedback.

Finally, It's written like a cmdlet so you can import it as a module or dot source it.  I've included s…

Updating vCenter Plug-in Registration URL's (from IP address to DNS) using PowerCLI

I've found this post by Josh Perkins helpful in more ways than the one demonstrated.  It's allowed me to not only update my vcops plugin registration w/ vCenter, but also helped me to update my Dell vCenter plug-in so that it connects via its DNS address rather than its IP.

However, I decided to explore the possibility of using PowerCLI to fix these registrations.
Turns out you can and the change is immediate, so you don't have restart the vCenter service.
YAY!  No more cert errors!  You still have to ssh into the vCOPs UI vm and update the viClientConfig.xml file, but that's easy enough.

Here is my result (specifically for vCOPs):

$BaseURL="https://myvCOPsRegisteredDNS.Name.local"$ExtURL="/vcops-vsphere/"Connect-VIServermy1stvCenterServer, my2ndvCenterServerForeach ($DefaultVIServerin$global:DefaultVIServers) { $VIServerExtensionManager=Get-…

PernixData: What is it? Does vSAN render it obsolete?

What is it?:
Simply put, it's flash acceleration.  You install a VIB in ESXi and register the PernixData Management (virtual management appliance) plugin to vCenter.  You can then take flash drives from each of your ESXi servers and clump them together to make a flash tier across all your hosts.

Does vSAN render it obsolete?:
Not really, it's a good solution to augment any existing SAN you might have.  It also won't require much change.  The scenery maybe changes as vSAN increases its configs max and traditional SANs fade, but that probably won't be for a long while.

The cool things:

You can use any SSD you want.  You don't have to purchase high-end or vendor specific SSD's.Reads and Writes can be accelerated.If the SSD fails, if configured, replicates that write to another SSD in the cluster.Yes, you would use network bandwidth @ this point only until the failed SSD is replaced.Most flash caching will only do reads because write redundancy can be somewhat problem…

VMware vSphere Profile-driven Storage Service not starting/running...

Really a benign error especially if you don't use the function, but quite annoying if vCOPs is looking @ vCenter health.

In my case, it appeared as though the vSphere Web Client service was conflicting w/ this Profile-Driven storage service.

Stop vSphere Web Client serviceStart VMware vSphere Profile-driven Storage Service.Start vSphere Web Client service. For some reason this works and the web client seems to have a bit more intelligence in choosing ports  to connect to rather than the storage service which gives up the second a port it tried is in use.  Probably a hard-coded thing.
Other Notes: If you are using the vSphere web client (which I wouldn't even bother with until 5.5 or newer), then it might make sense to install it on a different server to mitigate this issue.  Another possibility is to set its service to a delayed start.

Connect-VIServer not connecting to vCSA 5.5 U1 using windows integrated authentication...

Connect-VIserver MyvCSAServer was not connecting using my service account's (for scheduled tasks) windows account.  It would always prompt for credentials.  Fairly odd since it has permissions and is able to connect to several other vCenters w/o inputting credentials.

By the way, this is very convenient since I don't have to insert passwords anywhere in clear text or come up w/ some crazy solution to encrypt the password.

Simply log into the web client using the service account once.  Once authenticated, powerCLI should not have prompt for credentials.

[This applied when the VCSA's default identity source is set to Active Directory (Windows Integrated Authentication) and is set as the default domain.  Active Directory as a LDAP server option will not work.]

I'm guessing this is some kind of weird SSO thing, where the account needs to get locally cached prior to allowing Windows integrated authentication.  It also makes me wonder if…

SQL Connection Delay!?

Just another standard maintenance, upgrading vCenter from 5.0 U2 to U3.  Nothing special to see here.  Oh wait, it failed?  CRAP.  Restore database, try again.  Can't build vCenter Repository?!  Crap, restore database, try again.  AGAIN!!?  Long story short, SQL Authentication specified in the ODBC connection was experiencing intermittent connection issues.  Windows integrated was much more stable.

Opened ODBC --> Configure ODBC connection --> Enter SQL Credentials -->  Next... wait --> Error (See below for errors) --> Click OK --> Select Next again --> successfully connects and tests successfully.  Repeat...

This problem was likely due to the firewall in-between the vCenter server and SQL server.  As to what that problem was exactly, I have no clue.

Changed ODBC connection from SQL Authenticated User to Windows Integrated.  For this to work, you have to change the following:

Change the following services to star…

Newer Processor = Faster Performance? Sometimes...

We were receiving reports of some developers experiencing a degradation in JVM compile performance and general compiled application performance.  We had just recently vMotion'd these VM's from HP G7 systems w/ Westmere processors to newer Dell M620 blades running Sandy Bridge Processors.

Long story short, applications compiled on Westmere don't take advantage of a new AVX CPU extension available on Sandy Bridge, which in turn is only made available in RHEL 6.2 and newer kernels.  I'm not sure if Windows would be affected in the same way, but this was a definite issue in RHEL 5.8.

The performance delta was not 'huge' but enough in terms of scale.

Powershell functions and referencing the script that calls that function...

Lately I've been putting together functions so I can reuse code in my other scripts.  As part of error checking/reporting, I wanted to add a way for my function to reference the script that was making use of it.  I put a question out on twitter and here is what I learned.
@Zsoldier did you check this forum ? /cc @ScriptingGuys
— Johan Bijnens (@alzdba) April 15, 2014Quick and Dirty:
functionTest-Function { Param ( [string]$Weird ) Write-Host"MyInvocation"-ForegroundColor:Green $MyInvocation [string]$test=$MyInvocation.ScriptName.split("\") | select-Last1Write-Host $test -ForegroundColor:Green }
Hit page break if you want more details and an example of how I used $MyInvocation.

How to null terminate object properties in Powershell

I was working on a script, which I'll post on later, to insert IPMI/iLO/iDRAC configs into my ESXi hosts.  I would get an error the following error consistently no matter what I put into my IPMI object:

Exception calling "UpdateIpmi" with "1" argument(s): "A specified parameter was not correct. 
At line:50 char:2
+     $_this.UpdateIpmi($ipmiInfo)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : VimException

The short answer, I had to 'null terminate' the properties in my IPMI object.  I don't know 'why' I have to do this, but the vSphere API docs state that this is a requirement.  Quite frankly, I had no idea what this meant.

Solution Example:
To null terminate an object's property, you can simply do like so:
$MyObject = "" | Select Usefulproperty
$MyObject.Usefulproperty =  ("Something" + $null)

Additional Notes:

Get iDRAC/ILO (aka Baseboard Management Controller) IP via PowerCLI

[Update: 1 report states issues w/ HP Gen9 systems.  Newer Dell, and older HP systems don't seem to have issues.  If you have a HP Gen9 system, feel free to contact me via google hangouts and/or twitter to start an interactive session and we can work through figuring out a workaround.]

Needed a way to figure out what IP my HP iLO's / Dell iDrac's were configured with.  Ended up using an oldie, but a goodie script put together by Carter Shanklin.


Powershell 2+PowerCLI 4+Must have Port 443 (https) access to your ESXi hosts.If firewalls are an issue you may have the option of running this from your vCenter if it's still running on Windows.  If it's the vApp, you'll need to open access to port 443.Download Carter's scriptShort and Sweet: $info = Get-VMHostWSManInstance -VMHost (Get-VMHost myESXiServer) -ignoreCertFailures -class OMC_IPMIIPProtocolEndpoint $info # You can remove the -ignoreCertFailures flags if your systems have trusted certs. # Th…

Learning PowerCLI by Robert van den Nieuwendijk

Don't ask me how to pronounce Robert's last name.

The great thing about Robert's approach is that he takes the time to point out some basic powershell syntax outside of PowerCLI. You are then soon driven into some very useful cmdlets to extract information from your vSphere environment using Powershell and vSphere's PowerCLI cmdlets. There were a couple of areas of concern where a lack of explanation on certain things lead to a hmmm? moment, but overall an excellent book for a vSphere admin looking to use PowerCLI. For me as a technical person, I like the 'straight to the point' approach. This book is filled w/ little text and more usable script examples for me to get my job done as an admin/engineer.

For full disclosure, I was given this book to review, but was not compensated beyond this. I also have met Robert before, but that in no way has an affect on my review of this book. The above review reflects my honest opinion. You can find th…

The server naming convention is dead, long live the server naming convention!


Get rid of conventions that can be cross-referenced from other data sources in a programmatic fashion.  Consolidate or present those cross-referenced data points into one pane of glass.Rant:
In times old, server's had cool names like Ferrari, Robotech, etc.  Although cool, this drove the need for a server naming convention which served useful purpose.  It could tell us a good many things about a server just by looking @ its name.  Some things could be:

LocationOSApplicationEtc. Because of limitations in previous technologies, you had to limit your server name to a certain number of characters (coming from a Windows background).  Each of these data points needed to be shortened to acronyms or numbers.  This essentially requires a decoder ring for each section of the server name to understand it's hidden meaning.  When dealing w/ a small environment, not a big deal, but scalability becomes a problem.
Using the examples given above: Location SourcesIP SpaceActive Directory …

Quirkiness of Excel 2011 for Mac, CSV's, and Powershell 3.0 import-csv

Work on Windows or a Mac, don't cross the streams unless you're up against Gozer.

I made a CSV file using Excel 2011 for Mac w/ that looked something like this:

HeaderName, HeaderName2, HeaderName3, HeaderName4
Meh, super.meh.local, space, heckling
Sigh, super.meh.local, space, bashing

Saving this as a "Comma Separated Values" file was my first mistake cause when I tried to import this in Powershell, I would get the following super helpful error:

import-csv : The member "super.meh.local" is already present

To boil it down, I needed to save the CSV as a "Windows Comma Separated" csv file type.  The other method was that I could have placed quotes around all comma separated values via a text editor.

vCenter Log Insight not receiving syslogs from ESXi hosts


This appears to occur under very specific circumstances. There is a KB and it is also noted Log Insight's troubleshooting section of their documentation.

Rather than regurgitate all the same information is here is the gist:
Occurs when log insight services are restartedAffects ESXi 5.0.x/5.1.x when using TCP and 5.0 U1 and earlier over UDP.Does not affect 5.0 U2 and newer when using UDP.Here is the table:

Communication ProtocolAffected ESXi VersionTCP

ESXi 5.0.xESXi 5.1.xUDPESXi 5.0 and 5.0 U1