Posts

Showing posts from 2010

Netbackup 7 Permissions needed to Backup VM’s in vCenter 4.x

Image
Summary: Symantec will tell you that full administrator is needed to backup systems properly.  It is the easiest way, but I’m here to tell you that is not required.  There are two things that need to be done: Create a proper security role Apply that role to the various different nodes in vCenter Details: (Update to Role Perms can be found here.) Role Permissions needed (we’ll call it Netbackup Role): Privilege Group Privilege(s) to Enable Datastore Allocate Space Browse Datastore Low level file operations Global Licenses Configuration Add existing disk Add new disk Change resource Disk lease Remove disk Settings Provisioning Allow disk access Allow read-only disk access Allow virtual machine download State Create Snapshot Remove Snapshot Next, you need to apply permissions to the correct nodes in vCenter: Apply the Netbackup service account w/ the Netbackup Role to the vCenter Node. Propagate down if you want everything under the vCenter node t

Create iOS like page using Powershell

Summary: I was simply bored w/ the standard directory listing provided by IIS and was inspired by a page I saw.  I decided to make one modeled after that one and automatically generated using PowerShell.  I added some tooltips and text below the images while I was at it.  I run this script now every 5 minutes on my server to have something pretty to look at before going to my ugly reports . Anyway you can view a live example here:  http://techexamples.zsoldier.com I filled up the directory w/ copies of one html file just to show what page looks like when there are lots of html files. Here is the Powershell script: http://techexamples.zsoldier.com/Create-HTMLDefaultDirectoryPage.ps1 I utilized CSS for all images.  My next steps when I get time are to: Extract <title> tags from the html files in the directory and have them inserted into the tooltip. Preview the target report via a screenshot and place into the tooltip. Extract the apple-touch-icon tag to use the images

PowerCLI weirdness around Get-SCSILun and where statement

Summary: Using a where –eq statement against the objects provided by the Get-SCSILun either the vendor or model properties ends w/ 0 objects returned.  The following is an example: Example: 1: Get-VMhost myESXHost | Get-SCSILun | where {$_.Model -eq "SYMMETRIX" } 2: Get-VMhost myESXHost | Get-SCSILun | where {$_.Vendor -eq "EMC" } These two examples will likely return 0 results because the Model property is ALWAYS 16 characters and the Vendor property is ALWAYS 8 characters.  Each property are padded w/ spaces.  How did I figure this out?  Like this: 1: $Test = Get-VMhost myESXHost | Get-SCSILun | where {$_.Model -eq "SYMMETRIX" } 2: $Test[0].Model.Length <-- This returns 16 3: $Test[0].Vendor.Length <-- This returns 8 Resolution: I suggest using –like or –match when querying against these properties.  I prefer match, but to each their own.  Like so: 1: Get-VMhost myESXHost | Get-SCSILun | where {$_.Model -match &quo

View Composer not deleting or spinning up VM’s…

Summary: When deleting VM’s from inventory, View Composer usually starts the job in vCenter within a few minutes.  In this case, it was not and restarting the View Composer service on vCenter didn’t fix it. Config: vCenter 4.1 View 4.5 Resolution: In this particular case, one of my View Connection Broker Replica servers had shutdown and the following logs were seen on my primary connection broker: Event Type:    Information Event Source:    VMware View Event Category:    VMware View Event ID:    102 Date:        10/25/2010 Time:        11:24:51 AM User:        NT AUTHORITY\SYSTEM Computer:    Primary Broker Name Description: The failed LDAP replication attempt to Secondary-Replica Broker Name can be caused by a firewall blocking communication between this server and Secondary-Replica Broker Name or if the server is down. By simply bringing back the secondary replica broker, the primary seems to have been able to continue w/ the jobs I had assigned it.  It seems that it was stu

Error upgrading VMware Tools.

Summary: When attempting an automatic upgrade of tools on a RHEL VM, the above error shows up w/ no additional details in vCenter tasks & events.  This is a possible bug as the following entries show up in the VM’s vmware.log:   vmx| TOOLS INSTALL Copying upgrader files from /usr/lib/vmware/tools-upgraders into guest at /tmp/vmware-root vcpu-0| TOOLS INSTALL Error copying upgrader binary into guest. success = 0, HgfsStatus = 1 Resolution: Confirm and/or create the path /tmp/vmware-root on the RHEL guest.  In my case, the /tmp/ mount was there, but the sub-directory vmware-root was not.  After creating it, the vmware tools were able to upgrade successfully.  VMWare has a KB that is somewhat related, but only mentions Windows.  I would expect VMWare to create the directory if it didn’t find it, but maybe there is some reason behind this.

Provisioning error occurred for Machine vmname01: View Composer agent initialization failed.

This error may also appear in View’s Event Log: “View Composer agent initialization state error (6): Unknown failure (waited 0 seconds)” View 4.5 Agent apparently still has an issues with the App-V agent or vice versa.  The “Application Virtualization Client” (sftlist) should be disabled or placed into manual mode on your Parent VM.  Rest of instructions on how to combat this can be found here: http://tech.zsoldier.com/2010/03/view-composer-agent-initialization.html

EMC Virtual Storage Integrator (VSI) 3.0.1.5 the, what I hope to be, easy explanation

Image
Summary: Everywhere I read and hear “oh it’s easy”.  Yes, it is, but it is so poorly implemented and explained.  The reason is that you need two pieces to make it work, Solutions Enabler and VSI itself.  Here are the steps in nutshell: Recommended Install Steps: Install Solutions Enabler on Windows Server (Linux Server and running this locally is also an option) Can be a VM, but there are some caveats if your ESX hosts connect to vMax/Symmetrix arrays. Choose Complete and leave the rest of options default on install. Open a cmd prompt to the install directory. (Normally C:\Program Files\EMC\SYMCLI\BIN) Run the following commands: stordaemon install storsrvd –autostart stordaemon start storsrvd The VSI 3.1.0 install guide has the above command backwards. Install Solutions Enabler on a system that will be running the vSphere Client (normally your desktop/laptop) Choose Complete and leave the rest of options default on install. Install VSI on the same system running

View 4.5 Post-Synchronization Script not running?

Image
Summary: I have a post synchronization and power-off script that clears some McAfee reg keys, starts App-V services, and other misc. stuff (workaround from 4.0.1) that I’ve ported over to 4.5.  Essentially, I noticed when logging into my View 4.5 systems, this script was not being run for some reason.  Below is what I was seeing in my View pool settings: Notice the Yen symbols?  This likely happened when I recreated these pools and I was typing in “Romaji” instead of “US” on my Mac.  The Yen symbol is the equivalent of backslash(‘\’) on the keyboard and translates fine for most applications, not for View 4.5 apparently though.  This is likely a rare occurrence in which my Japa-cheesyness isn’t so cool. Resolution: Replace the Yen symbol w/ a good ‘ol US backslash(‘\’) as picture below: The good news is that after making this change any new VM’s that spin-up will run the batch files correctly.

API version '4' for vCenter Server '<vCenterName>' is not supported.

Image
Summary: This helpful message appears when you attempt to use a VUM 4.1 PowerCLI cmdlet against a 4.0 vCenter / Update Manager version. Resolution: Uninstall VUM 4.1 PowerCLI and install VUM 4.0U1 PowerCLI . Takeaway: Read the release notes and hope that VMWare has right download link posted on their site, which as of this posting is wrong.  The link posted above is correct.

Samsung Vibrant and 802.1x WEP…

If there is an easier or non-root required method to what I’ve outlined here, please I’m ALL EARS. Summary: I pretty much hate Samsung or Android, not sure where to direct my h8, but whatever.  I wasn’t going to let it beat me.  Essentially I needed to connect to an AP that uses 802.1x WEP, PEAP, and MSCHAPv2 radius authentication to get access.  Not that difficult on my iPhone w/ the use of profiles, but Android/Samsung wasn’t as easy.  Here it is in a nutshell: Root your Vibrant (SDK method preferred) Use the SDK to pull your current wireless config Modify the wireless config entry you have for the AP that you are trying to connect to. VOILA! successful connection. Nitty Gritty: Follow the instructions on how to root your Vibrant. Once your vibrant has been rooted, you need to download and install the SDK . Turn off your phone’s Wifi, and turn USB debugging on (Settings –> Applications –> Development). This is where it gets interesting (Windows method): Open a c

ScsiCore: 1181: Sync CR

Image
Summary: One VM out of several thousands became inaccessible via network and vCenter console access (sample pictured below).  Other symptoms included: Slow vMotion to vMotion just not occurring within the cluster that had that one VM failing. VMWare vMotion failure @ 10% ScsiCore: 1181: Sync CR (opcode ##) at ### (wid #) <—Error in vmkernel log Cannot browse datastore w/ questionable VM from vCenter VM Console shows below error when trying to access problem VM. Resolution: “Trespass” or “Failover” the LUN w/ the questionable VM to another storage processor (SP).  This immediately stopped all errors and all unaffected VM’s on that LUN continued to run while releasing whatever lock was placed on the LUN causing these ‘wonky’ issues. Other related reading from VMWare: http://kb.vmware.com/kb/1018675 http://kb.vmware.com/kb/1003734

VMWare View 4.0.1 no desktop resources exist, but admin console shows VM’s in Ready status…

Summary: Essentially the title sums it up.  A user tries to connect to a pool, but they get an error stating that no desktop resources are available to them.  A look @ the admin console shows either the VM is in “Ready” or “Disconnected” status. Possible Resolutions: Reset the “VMWare View Agent” service on all or a specific VM in a pool. Reset the VM from the vCenter management console. Check your DHCP pools, make sure all addresses are not reserved already. I believe the View admin console lacks the code to display something like ‘offline’ when it loses communication w/ a VM’s View Agent.  It’s interesting because the View broker is aware that a VM’s View Agent is offline based on it’s debug logs (Usually found under C:\Documents and Settings\All Users\Application Data\VMWare\VDM\logs), but you would never be the wiser because the admin console shows everything to be fine.  It seems that status is only updated when it ‘expects’ something back from the Agent like after initial

Desktop Composer Error:VMware.Sim.Fault.AdAuthenticationFault

Image
Summary: This particular error occurred when I was migrating my vCenter / vComposer server to new hardware.  There were 2 successive errors that would appear in View Manager that look something as follows: Pool ::Unable to remove from inventory VM /DataCenter/vm/VDIFolderName/VMName - Failed to cleanup linked clone /DDTC/vm/VDIFolderName/ Desktop Composer Error:; Error Type:com.vmware.SviService.AdAuthenticationFault;Error FaultActor:null; Error code:{http://schemas.xmlsoap.org/soap/envelope/}Server.generalException; Error details:[Lorg.w3c.dom.Element;@1fc1ee1 Resolution: Essentially the system was complaining that the vComposer account that is specified under Configuration -> Server -> vCenter Servers was not part of the local administrator group (Windows side, not vCenter) of the vCenter server.  This was one part of the issue, the other half were the QuickPrep accounts, the accounts specified need to have AD rights to create/modify/delete computer accounts.  If it i

VMWare View (Desktop Composer Fault: Virtual Machine with input specifications already exists)

Summary This error is caused when View Composer has some kind of hiccup and ‘thinks’ it deployed a desktop and can’t get over the fact that it failed.  There are 3 event entries that will usually show up right next to each other: Desktop Composer Error:VMware.Sim.Fault.InvalidParameterFaultParam: cSpec Detail: AlreadyExist; Error Type:com.vmware.SviService.InvalidParameterFau Desktop Composer Fault: Virtual Machine with input specifications already exists PendingOperation-Path/that/has/been/defined/ VMName Config: View 4.0.1 SQL 2005 Resolution: I found this SQL script on the vmware communities board after I went through the manual hell of finding each related key.  Essentially, you need to delete all related SQL entries so View Composer can move on to start deploying VM’s again.  This script makes this job a WHOLE lot easier. 1: DECLARE @vmdelete varchar (20); 2: set @vmdelete = 'FAILED_VM_NAME' ; 3: delete from SVI_VM_NAME where NAME = @vmdelet

Powershell –replace method and my own frustration.

Summary: I added some code to my VMWare Infrastructure script for a new column of data to show a tooltip when clicked or hovered over.  Because I also have it export a csv file from this code, I use the –replace method to strip the html formatting.  Because of one section, more data than I wanted was being stripped.  Why?  Probably because I used this: .*> .  Resolution: Anyway here is the string I was using: 1: foreach { $ _ -replace(" < p style='font-weight:bold;color:orange' > ","") -replace(" < p style='font-weight:bold;color:red' > ","") -replace(" < a class='cluster' href= . /healthcheck . htm #.*> ","") -replace(" < /p > ","") -replace (" < /a > ","") -replace (" < a href=' #' class='tt'>","") -replace (" < span class='tooltip' > < span class='top&

Battered, bruised, but alive

Image
Grab this badge here! I only wish that I had the time to finish the rest of the games.  There is always next year I suppose.

Bug? Feature? VM Hardware 7 RHEL5 32-bit running 64-bit

Image
Summary: Under VM Settings --> Option Tab --> General Options –> Guest Operating System, Red Hat Enterprise Linux 5 (32-bit) is selected.  However, the installed OS is actually RHEL5 64-bit.  Under VM Hardware version 4, this combination was not possible as RHEL would detect that the processor was not 64-bit capable probably because of masking.  Not that I care all that much since it seems to work fine, but what would this be considered?  A feature or a bug?  Only reason I noticed was because our Linux admins were wondering why in a set of 6 VM’s running 32-bit, this one was running a 64-bit version… Answer: Because someone on their end screwed up and installed the wrong version, can’t completely blame them though since in version 4 they would have been denied. Example: Side Note: Interestingly, the Guest OS field now seems to update w/ the actual Guest OS information provided by the VMWare tools.

Rename a VM without shutting down…using storage vMotion

Summary When renaming a VM in vCenter, underlying files and folders remain named the same as when the VM name was first designated.  This can make for a difficult recovery scenario if file/folders and names in vCenter are out of sync. Config vCenter 4.0 U1 ESX 4.0 U1 Solution Rename the VM in vCenter Migrate the VM to another Datastore Done! If you check the VM’s settings and/or datastore, all related files/folders should now match the VM name in vCenter. Maybe this was obvious to some folks, but it wasn’t until I was doing maintenance @ the wonderful hour of 2am that this dawned on me as a possible solution.  I’m so used to having to go through this process.

There are errors during the remediation operation…updating esx host using update manager.

Image
Summary: One of the greatest features of Update Manager (4.0 U1 P1) is that you can click on a cluster and remediate the whole thing.  It places one host @ a time into maintenance mode and does it’s thing.  Unless of course Update Manager happens to be a VM that resides on one of those hosts in the cluster. Example of Error: WorkAround: Update one host in your cluster manually Migrate the Update Manager VM to that updated host Remediate the entire cluster. Rant: I like the fact that vCenter returns such an obvious error, but why can’t it move itself to another host like any other self-respecting VM.

(Storage vMotion) Source detected that destination failed to resume.

Summary: I had to script migrate 130+ VM’s off a clariion SAN array because of SP utilization issues to a vMAX array.  All but 3 succeeded.  1 of which exhibited the below symptoms.  The other two were a MSCS cluster, unrelated to this post. Symptoms: Above message appears when attempting a Storage vMotion on a VM. Gets to 5% to 10% before above error appears. When VMWare’s KB1010045 does not apply as evidenced in my Summary. Work Around: vMotion VM to another ESX host. Attempt Storage vMotion. I have no clue why this worked, but whatever.  If this helps someone make their day a little easier then I’m happy.

An error occurred during installation (Mac OS X 10.6.3)

Configuration: Mac OS X 10.6.3 (Originally installed w/ English, later changed to Japanese) Summary: Using Mac’s Software Update system, I was presented w/ the above error followed by “ Some of the selected updates couldn’t be installed, an unexpected exception occurred. ”  Deleting the contents /Library/Update did not help.  My system was running in Japanese mode. Workaround: Change the system back to English, run Software Update again.  Success.   No clue as to why I had to do this, I thought all underlying binaries were written in the same language, but I guess I might be mistaken.

PowerGUI Script Editor Menu Bar Items Grayed Out

Image
[Update: The config file, named  BarManagerLayoutFile.xml is  likely responsible for this weirdness is located in your user's application data folder.   Win7 - C:\Users\MyUserName\AppData\Quest Software\PowerGUI WinXP - C:\Documents and Settings\MyUserName\Application Data\Quest Software\PowerGUI Play w/ the settings changing them from False to True.  Looking @ my own, most are set to True. ] Summary: Weird issue, I open PowerGUI’s script editor one morning and my menu bar items are all grayed out.  This makes it so that I can't use my mouse to select File, Edit, View, Go, Debug, Tools, or Help.  Reinstallation doesn’t seem to help either. Solution: Right-click in the menu bar area and select “Customize:” Highlight “Menu Bar” and whatever other buttons/menus are grayed out and click the reset button. The Menu Bar and other toolbars should now be un-grayed out unless they are supposed to be. Configuration: Windows 7 64-bit PowerGUI Script Editor 2.0.0.1082

VM Infrastructure Powershell Summary Report v. 2.5

Image
I've completely rewritten my original summary script for much faster performance (From 20 min. to approx. 3 min. in an environment running 60 Hosts, 1300VMs, and 181 Datastores).  Nothing much has changed as far as how or what it reports on, but here is a list of changes: Much Faster Performance. CSS can be used to change mouse cursor to hand icon for table header.  ( Example CSS found here ) vCPU and Datastore Percent numbers to highlight ‘out of spec’ numbers in customizable variables now. Custom ico file specifiable. (icon to show up in supported browsers) Custom JScript specifiable.  ( Recommended JScript found here ) Added some versioning There is still plenty to be refined about this script, but it’s pretty fast and can give you an overall picture of how your environment is doing.  Plus, it’s a good thing to put in front of management to help them understand what’s going.  I run this script about every hour so I know w

VMWare Paravirtual SCSI Driver and gParted

Image
[Update:  Windows 2008 does not require gParted.  It can extend disks on the fly.] Summary gParted does not detect drives using VMWare’s paravirtual scsi controller.  Unable to extend a boot partition. Workaround Change the SCSI Controller to the LSI Logic Parallel controller. Boot into gParted and extend the disk. Shutdown the VM and change the SCSI Controller type back to VMWare Paravirtual . Easy Peasy!

Can't get to Google.com or YouTube.com from Japan...

Image
Summary: Ran into an interesting issue today.  Friend from Japan messaged me that they were having issues connecting to Google and YouTube.  I remoted into their computer and found something interesting.  Google and YouTube were resolving to IPv6 addresses. Hypothesis: I'm thinking Japan's ISP's are upgrading their networks to IPv6.  Still, I found it strange that even though Google and YouTube were resolving properly, the pages wouldn't render.  Only those people connected directly to their modem would likely see this issue, if you are behind a router, then this issue would likely not manifest. Resolution: Anyway, to resolve this issue just requires that IPv6 be unbound from the Network Adapter setting in Windows.  Below a sample screenshot from my Windows 7 VM.

Unable to apply DRS resource settings on host…

Summary: VMWare KB 1004667 addresses a different scenario than below. This error can also occur if a VM has issues w/ the vmxf file as evidenced by digging into the hostd.log file.  What’s interesting, the VM can be fully functional, vMotion’d, and not show a single error in vCenter.  Once you dig to find a VM is causing the issue, vMotion’ing this VM to another host, the new host begins having issues applying DRS settings. Unable to apply DRS resource settings on host (Reason: A general system error occurred: Invalid Fault). This can significantly reduce the effectiveness of DRS. Hostd.log (/var/log/vmware/hostd.log) contains messages similar to: [2010-03-22 15:13:05.285 F5CA0B90 info 'Libs'] Vix: [18420 foundryFile.c:2109]: FoundryFile_Save: Failed to open file /vmfs/volumes/volumeID/vmname/vmname.vmxf [2010-03-22 15:13:05.285 F5CA0B90 info 'Libs'] Vix: [18420 foundryFile.c:2130]: FoundryFile_Save: Failed to close file /vmfs/volumes/volumeID/vmname/vmname

View Composer agent initialization state error (6): Unknown failure (waited 0 seconds)

Image
Perhaps the most useless error message on the planet, right next to PC Load Letter. [UPDATE: The workaround detailed below refers to quickprep only.  Versions 4.5 and above now allow you to use customization specifications in vCenter.  I'd recommend using those for easier administration, especially when expanding your environment.] Summary: This error message can appear when deploying View Pool from a Parent VM (aka Gold Image) that has Microsoft’s App-V 4.5 client installed: View Composer agent initialization state error (6): Unknown failure (waited 0 seconds) Workaround: On the Parent VM, change the “Application Virtualization Client” service to manual. Next steps are completely voluntary.  The only reason I went further was so that as soon as a user logs into a View session, their App-V applications would show up immediately. Create a batch/cmd file and place it on the root directory of the VM. (Commonly C:\) Open Notepad Paste the following two lines: sc con

VMWare Update Manager PowerCLI Released

All I can say, it’s about F’N time!  Now I can update my templates on a monthly basis. http://www.vmware.com/support/developer/ps-libs/vumps/ Requirements: Update Manager 4.0 Update 1 VMware vSphere PowerCLI 4.0 Update 1 .NET 2.0 Service Pack 1

vCenter Permissions/Roles overkill? maybe? maybe not… specifically with View 4.0.1

Image
Related Products: VMWare View 4.0.1 VMWare vCenter 4.0 U1 Problem Statement: When creating a View Desktop Pool, no folder path, or datastores are presented and a whole other slew of errors should you decide NOT to grant the administrator role to the View Composer Service Account . Resolution: View Composer Role permissions need to be granted in 1 or 6 places in vCenter depending on how restricted the account should be.  1 place would be to grant the administrator role to the View Composer Service Account @ the vCenter Node Level and propagate down.  If the aim to to restrict access to one cluster/host, then the View Composer Role must be applied to the following areas: Pictures are provided as an example and are not a reflection of how all vCenter servers are configured/designed. VM and Templates – Role added to Chosen Target Folder, Gold Image Folder, and the “ VMWareViewComposerReplicaFolder ” Hosts and Clusters – Role added to Host | Cluster Datast

vCenter 4.0 U1 Performance Statistics w/ mix of ESX 3.5 and 4.0 Hosts

Image
Configuration: ESX 4.0 U1 ESX 3.5 U4 vCenter 4.0 U1 Issue: Discrepancy between tracked metrics and “Statistics Level” set in vCenter.  For instance, Level 1 Statistics Level in 4.0U1 now includes “CPU Ready” in historical tracking.  In vCenter 2.5, this metric was only historically tracked @ Level 3.  However, this seems to only apply to VM’s hosted on ESX 4.0 AND vCenter 4.0.  If the VM is hosted on an ESX 3.5 system, then that metric is not tracked.  Another possible cause for lack of historical metric data may be that SQL jobs are not completing properly.  See SQL Job resolution section. Hypothesis: I’m guessing here, but this is likely due to the way ESX 3.5 identifies the metric to vCenter which is probably as Level 3.  Below is what you might see when looking @ a VM’s performance on an ESX 3.5 and 4.0 host. Examples: ESX 3.5 VM w/ vCenter 4.0 Stat Lvl 2: ESX 4.0 VM w/ vCenter 4.0 Stat Lvl 2: References: http://www.vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_admin_gui

iPhone 文字バケ(MojiBake) gMail

Summary: Sending and receiving emails from the iPhone mail app with Japanese characters are unreadable due to mojibake. Details: This seems to only occur if you use gMail's ActiveSync option by setting up your account on the iPhone by selecting Microsoft Exchange as the account type. Workaround: To remedy this, use either POP or IMAP when setting up your 'mail' account. You can still have your iPhone sync contacts and calendars seperately.