ESX 4.0 – Page 5

Script - ipmiConfig.pl

06.03.2010 by William Lam // 1 Comment

There was a question on the VMTN community asking about automating the configuration of IPMI/iLO Settings for Power Management via kickstart. Unfortunately, this cannot be configured directly on an ESX/ESXi host, it requires vCenter server. This configuration is a prerequisite to VMware's DPM (Distributed Power Management) which is only available with vCenter and DRS enabled cluster.

Although this configuration can't be added to kickstart, you can create a script that is executed as part of the "post configurations". Before starting, make sure that you have IPMI enabled over LAN which is generally found within your iDRAC/iLO or other out-of-band management console.

Here are two posts on popular hardware (Dell/HP) for configuring IPMI/iLO settings:

Dell configuration
HP configuration

In general, the actual configuration for the host is the same but will differ based on the server vendor and out-of-band management console.

Here is a vSphere SDK for Perl script that allows you to configure IPMI/iLO settings for an ESX or ESXi host:

Download: ipmiConfig.pl

You'll need to provide the following variables:

vihost = ESX or ESXi host
ipaddress = IP Address of your iDRAC/iLO/etc.
macaddress = MAC Address of your iDRAC/iLO/etc. interface
bmcusername = Username to login to your iDRAC/iLO/etc.
bmcpassword = You'll be prompted to provide the password

Here is an example run:

[vi-admin@scofield ~]$ ./ipmiSettings.pl --server reflex.primp-industries.com --username primp --vihost dev-esxi1.primp-industries.com --ipaddress 172.50.0.45 --macaddress 00:22:E8:71:C5:47 --bmcusername admin
Please enter your BMC Password:

Trying to configure IPMI/iLO Settings for Power Management on dev-esxi1.primp-industries.com ...

If you've entered everything correctly, the prompt should return right away. If it sits there for more than 20 seconds, then something went wrong and you may have entered something incorrectly. Due to the limitation of this API method, no tasks are returned back to properly handle these errors. You may want to manually configure this via the vSphere Client to double check your settings before re-using the script.

Here is an example of a bad run (entered wrong username + password):

Trying to configure IPMI/iLO Settings for Power Management on dev-esxi1.primp-industries.com ...

Error: Unable to configure IPMI/iLO Settings: SOAP request error - possibly a protocol issue: 500 SSL read timeout:

vSphere Tips and Tricks Part 1

05.30.2010 by William Lam // 3 Comments

Disclaimer: Ensure these changes have been thoroughly tested and confirmed with VMware support prior to rolling out into production. Most of these changes and tips may be undocumented by VMware, test at your own risk!

Map Service Console/COS VMDK to specific ESX host (useful if you boot from SAN):

grep -i "hostname" /vmfs/volumes/[datastore]/[esxconsole-datastore-uuid]/logs/sysboot-vmkernel-late.log
[root@everest ~]# grep -i "hostname" /vmfs/volumes/everest-local-storage/esxconsole-49a095a4-204d-c7ba-9bb8-001f29c7b048/logs/sysboot-vmkernel-late.log
0:00:00:06.289 cpu1:4105)Config: 363: "HostName" = "everest.primp-industries.com", Old value: "localhost" (Status: 0x0)

Retrieve VmId from vmware-cmd similiar to that in vimsh/vmware-vim-cmd (only available on classic ESX, not sure why it's not in vCLI):

vmware-cmd /vmfs/volumes/[datastore]/[path_to_vm]/vm.vmx getid
[root@everest ~]# vmware-cmd /vmfs/volumes/d32c5a97-e039bdbf/vCenter-ESX4.02172009/vCenter-ESX4.02172009.vmx getid
getid() = 32

VMware FT statistics and debugging utility
Available after some patch set, generally 4.0u1+

[root@esx-5 ~]# ft-stats
Usage: ft-stats [-hvf] [-c ] [-l] [-n ] [-d ] [-s ] [-w ]
-h, --help Show this message
-v, --verbose verbose output
-f, --force Continue even if version check fails
-c, --cache Use VSI cache file
-l, --list list FT VMs
-n, --iterations Print stats for n iterations
-d, --delay Print stats every n secs (default:5)
-s, --snapshot Collect stats for n secs
-w, --world Print stats for worldID

Note: This tool is intended to assist in troubleshooting and should only be used
with direction by VMware support. All other use of this tool is unsupported.

esxupdate - Query for installed/retired packages with undocumented "--vib-view" option

esxupdate --vib-view query
[root@himalaya ~]# esxupdate --vib-view query
--------------------------------------VIB ID------------------------------------- Package State -----------Timestamp------------
rpm_vmware-esx-drivers-scsi-fnic_400.1.1.0.109.2vmw-2vmw.1.9.208167@x86_64 installed 2010-01-06T12:24:38.100201-05:00
rpm_acpid_1.0.4-9.el5_4.2@x86_64 installed 2010-04-02T17:09:44.566014+00:00
rpm_samba-client_3.0.33-3.15.el5_4.1@x86_64 installed 2010-04-02T17:09:44.171847+00:00
rpm_nss_3.12.3.99.3-1.2157.vmw@x86_64 installed 2010-04-02T17:09:41.148547+00:00
rpm_initscripts_8.45.19.EL-1.2442.vmw@x86_64 installed 2010-04-02T17:09:36.911452+00:00

Change default COW (copy on write) disk growth value
Query (Default is 16MB):

[root@himalaya ~]# esxcfg-advcfg -g /COW/COWDiskSizeIncrement
Value of COWDiskSizeIncrement is 32768

Set (e.g. 256MB):

[root@himalaya ~]# esxcfg-advcfg -s 524288 /COW/COWDiskSizeIncrement
Value of COWDiskSizeIncrement is 524288

The input value for changing COWDiskSizeIncrement is in sectors

UNIT SECTORS	COW GROWTH SIZE
32768	16MB
65536	32MB
131072	64MB
262144	128MB
524288	256MB
2097152	1024MB
....	....

Easy formula is (SIZE_IN_MB)*(1024)*(2) = SECTOR VALUE

vSphere ESX 4.0 - Crash VM Bug?

05.29.2010 by William Lam // Leave a Comment

We recently discovered an anomaly while backing up one of our development VMs using ghettoVCB.sh. When attempting to back up this powered on VM, the backup was successful however oddly, we were left with a powered off VM immediately following the first VMDK clone operation. After some investigation, we found that the problematic VM contained virtual disks spread across two datastores with dissimilar blocksizes (1MB and 2MB).

The VM configuration alongside its main OS disk was stored on the datastore with a 1MB blocksize while it’s data disk (>256 GB) resided on the other datastore which was initialized with a 2MB blocksize. We came to the conclusion that this might have had something to do with the VM configuration residing on a datastore with a blocksize that was smaller than what is needed for the larger VMDK (which was on a datastore with an ample blocksize). Manually snapshotting this VM apparently fails however different behavior was experienced when the commands are executed from a script.

Believing that this was a corner case, we decided that it was best practice to keep all VMFS volume block sizes consistent. This was to be remediated at a later time.

Today we noticed a blog post http://www.yellow-bricks.com/2009/08/24/vsphere-vm-snapshots-and-block-size/ from Duncan Epping regarding the snapshot issue. This may not be a corner case as we thought so we wanted to share this finding with everyone.

If you have a similar configuration from above, it is guaranteed that the VM will crash if you run a script that tries to take a snapshot of the described VM and then subsequently exports the VMDK using vmkfstools.

Here is a video displaying the symptoms decribed from above:
http://engineering.ucsb.edu/~duonglt/vmware/crashVM

Output from script execution on VM: Quentin:

[root@himalaya ~]# ./crashVM.sh Quentin /vmfs/volumes/dlgCore-NFS-bigboi.VM-Backups/Quentin-clone.vmdk
thinking, give me a few ...
The power state of VM: "Quentin" is On
Extracted VmId and locating VM configured datastore (which should live on a smaller VMFS block size)
Located VMDK: "/vmfs/volumes/himalaya-local-SAS.Savvio/Quentin/Quentin.vmdk"
Trying to create snapshot ... (this should fail)
Create Snapshot:
Trying to vmkfstools copy "/vmfs/volumes/himalaya-local-SAS.Savvio/Quentin/Quentin.vmdk" (this should REALLY fail! right?)
Destination disk format: VMFS zeroedthick
Cloning disk '/vmfs/volumes/himalaya-local-SAS.Savvio/Quentin/Quentin.vmdk'...
Clone: 9% done.

Download crashVM.sh script