WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

Quick Tip - ESXCLI CSV --format-param options

04.03.2014 by William Lam // Leave a Comment

When using ESXCLI, the output is formatted using a "default" formatter based on the type of data being displayed. However, you can easily modify the output by using one of the three supported formatters: xml, csv and keyvalue or even leverage some internal ones mentioned by Steve Jin here. When working with some of the ESXCLI 'storage' namespaces, such as listing all the devices on an ESXi host, the output can be quite verbose as seen in the example below:

~ # esxcli storage core device list
t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__
   Display Name: Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__)
   Has Settable Display Name: true
   Size: 572325
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__
   Vendor: ATA
   Model: INTEL SSDSC2BB60
   Revision: D201
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: false
   Is Local: true
   Is Removable: false
   Is SSD: true
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: unknown
   Other UIDs: vml.01000000004254574c3333363330345a3636303054474e2020494e54454c20
   Is Local SAS Device: false
   Is Boot USB Device: false
   No of outstanding IOs with competing worlds: 32

t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__
   Display Name: Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__)
   Has Settable Display Name: true
   Size: 572325
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Devfs Path: /vmfs/devices/disks/t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__
   Vendor: ATA
   Model: INTEL SSDSC2BB60
   Revision: D201
   SCSI Level: 5
   Is Pseudo: false
   Status: on
   Is RDM Capable: false
   Is Local: true
   Is Removable: false
   Is SSD: true
   Is Offline: false
   Is Perennially Reserved: false
   Queue Full Sample Size: 0
   Queue Full Threshold: 0
   Thin Provisioning Status: yes
   Attached Filters:
   VAAI Status: unknown
   Other UIDs: vml.01000000004254574c3331383330314a4c36303054474e2020494e54454c20
   Is Local SAS Device: false
   Is Boot USB Device: false
   No of outstanding IOs with competing worlds: 32

Usually for such a command, you are interested in a couple of specific properties and I bet you are probably spend a good amount of time scrolling up and down, I know I do. One useful option that is not very well documented (will be filing a bug for this) is the --format-param options which goes in-conjunction with the csv formatter. I always forget the syntax and can never find it when I Google for it so I am documenting this for myself but I think this would also be useful for others to know about.

The --format-param option allows you to specify specific property fields you care about. If we use the our ESXCLI example above, what I really care about are the following for each device:

  • Display Name
  • Is Local
  • Is SSD

Using the following command, we can then extract only those fields we care about:

~ # esxcli --formatter=csv --format-param=fields="Display Name,Model, Is Local,Is SSD" storage core device list
DisplayName,Model,IsLocal,IsSSD,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304Z6600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL318301JL600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____WDC_WD4000FYYZ2D01UL1B0_______________________WD2DWMC130199689),WDC WD4000FYYZ-0,true,false,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL3183002H600TGN__),INTEL SSDSC2BB60,true,true,
Local ATA Disk (t10.ATA_____INTEL_SSDSC2BB600G4_____________________BTWL336304XL600TGN__),INTEL SSDSC2BB60,true,true,

If we now look at our output, we can easily see that we have 5 devices on our ESXi host and I can quickly see the Display Name of our device, whether it is a local device seen by ESXi and if it is an SSD. I find this filtering mechanism especially handy during troubleshooting or when you need to quickly identify a device for configuration.

Categories // ESXi, vSphere Tags // csv, esxcli, ESXi, vSphere

Simulating vSphere Performance Metrics using VCSIM

04.01.2014 by William Lam // 7 Comments

A really useful tool that I leverage from time to time is VCSIM (vCenter Simulator) which can be found within the VCSA (vCenter Server Appliance). VCSIM allows you to easily "simulate" a custom vSphere Inventory that can be used for a variety of use cases including custom reports using the vSphere API/CLIs. There are additional capabilities that VCSIM provides and one that I have not explored much is the simulation of vSphere Performance Metrics. Having received a couple of inquires regarding VCSIM and performance metrics, I figure this would be a good opportunity to explore this feature in a bit more detail.

Disclaimer: This is not officially supported by VMware, use at your own risk.

Before getting starting, you should familiarize yourself with VCSIM by reading these two articles here and here.

By default when running VCSIM, there are no Performance Metrics. If you wish to include it, you will need to ensure the following two lines are added to your VCSIM configuration file:

<perfCounterInfo>vcsim/model/PerfCounterInfo.xml</perfCounterInfo>
<metricMetadata>vcsim/model/metricMetadata.cfg</metricMetadata>

The first file contains the performance metric definitions that are supported and the second file will specify which metrics will be simulated. To demonstrate the performance metric capabilities of VCSIM, I will be using the following configuration files which you can just copy/paste:

vghetto-perf-vcsim.cfg - This will be our VCSIM configuration file

<simulator>
  <enabled>true</enabled>
  <initInventory>vcsim/model/vghetto-perf-inventory.cfg</initInventory>
  <hostConfigLocation>vcsim/model/hostConfig</hostConfigLocation>
  <perfCounterInfo>vcsim/model/PerfCounterInfo.xml</perfCounterInfo>
  <metricMetadata>vcsim/model/metricMetadata.cfg</metricMetadata>
  <datastore>
     <global>1</global>
     <cluster>2</cluster>
     <local>1</local>
     <prefix>vghetto</prefix>
  </datastore>
</simulator>

vghetto-perf-inventory.cfg - This will be our VCSIM inventory configuration file

<config>
  <inventory>
    <dc>1</dc>
    <host-per-dc>0</host-per-dc>
    <vm-per-host>0</vm-per-host>
    <poweron-vm-per-host>0</poweron-vm-per-host>
    <cluster-per-dc>1</cluster-per-dc>
    <host-per-cluster>2</host-per-cluster>
    <rp-per-cluster>1</rp-per-cluster>
    <vm-per-rp>3</vm-per-rp>
    <poweron-vm-per-rp>3</poweron-vm-per-rp>
    <dv-portgroups>0</dv-portgroups>
  </inventory>
  <prefix>vGhetto-</prefix>
  <worker-threads>1</worker-threads>
  <synchronous>true</synchronous>
</config>

To ensure everything is working, we can start VCIM by issuing the following command on the VCSA:

vmware-vcsim-start /etc/vmware-vpx/vcsim/model/vghetto-perf-vcsim.cfg

If everything is working, you should be able to login using the vSphere Web/C# Client to view the small inventory we just created. If you are able to see the inventory, then go ahead and stop VCSIM by issusing the following command:

vmware-vcim-stop false

Note: To ensure our inventory is not destroyed each time, you should pass in the 'false' flag else it will automatically be deleted each time. This is useful if you want to preserve your inventory on subsequent reboots.

For all Performance Metric configurations, you will only need to edit the metricMetadata.cfg file. VCSIM supports four different types of Stats model:

  1. Constant
  2. Linear
  3. Square
  4. Triangle

We will take a look at Stats model for 2-4 since the Constant is not all that interesting 🙂 For our examples, we will take a look at each Stats model for the datastore.datastoreIops.average metrics.

Linear Stats Model:

vcsim-perf-linear

<Metric id="datastore.datastoreIops.average">
  <Instance id="Default"/>
    <StatsModel>
      <Type>Triangle</Type>
      <Values>0,20,10,30,0</Values>
      <Periods>600,300,600,900</Periods>
    </StatsModel>
  </Instance>
  <Instance id="HostDatastore"/>
</Metric>

Square Stats Model:

vcsim-perf-square

<Metric id="datastore.datastoreIops.average">
  <Instance id="Default"/>
    <StatsModel>
      <Type>Square</Type>
      <Values>0,10,0,20,0</Values>
      <Periods>300,300,600,600,300</Periods>
    </StatsModel>
  </Instance>
  <Instance id="HostDatastore"/>
</Metric>

Triangle Stats Model:

vcsim-perf-triangle

<Metric id="datastore.datastoreIops.average">
  <Instance id="Default"/>
    <StatsModel>
      <Type>Triangle</Type>
      <Values>0,20,10,30,0</Values>
      <Periods>600,300,600,900</Periods>
    </StatsModel>
  </Instance>
  <Instance id="HostDatastore"/>
</Metric>

I thought this was pretty neat that the VCSIM developers included a couple of Stats Models that could be leveraged right out of the box! As you can see, it is pretty easy to enable various performance metrics simply by identifying the metric(s) you are interested in and specifying the Stats Model and then starting up VCSIM. The other neat thing that I have been asked about before is can VCSIM simulate performance metrics for specific vSphere entities? I originally thought the answer was no until I started to play with the performance metric simulator a bit more and realize there is a List property that you can use to specify the specific objects in which you want data to be displayed.

Here is an example of of the same performance metric we have been looking at but only enabling it for two Datastores:

vcsim-perf-list

<Metric id="datastore.datastoreIops.average">
  <Instance id="List">
    <List>vghettoDS_vGhetto-DC0_C0_0,vghettoDS_vGhetto-DC0_C0_1</List>
    <StatsModel>
      <Type>Triangle</Type>
      <Values>0,20,10,30,0</Values>
      <Periods>600,300,600,900</Periods>
    </StatsModel>
  </Instance>
  <Instance id="HostDatastore"/>
</Metric>

 Note: One thing I noticed while playing with the performance metric simulator is that some times the object in the UI is blank when using the vSphere Web Client. If I use the vSphere C# Client, it is perfectly fine and that is also true if you are using the vSphere API to query for these metrics.

Hopefully this was a good overview of how the VCSIM performance metrics feature works. I know there are a couple of internal folks who have used VCSIM injunction with vCenter Operations and I am also curious to see what other neat uses cases exists for the performance metrics feature. Also, if you have created a really cool metricMetadata.cfg configuration file, feel free to share with the rest of the community!

Categories // Not Supported, VCSA, vSphere Tags // performance, VCSA, vcsim, vcva

A kitten is also harmed, every time you edit a VMX file by hand

03.25.2014 by William Lam // 4 Comments

kitten-is-harm-when-editing-vmx-by-hand
Following up on a recent article that I wrote a kitten dies every time you query the VCDB, I wanted to raise another topic that could also bring harm to kittens as well as making me cringe (JK on the harming of kittens). The topic, as you might have guessed is about editing a Virtual Machine's VMX configuration file by hand which comes up some what frequently from customers. There are many valid reasons on why a customer would need to perform such an operation such as enabling Change Block Tracking for efficient Virtual Machine backups to the hardening of a Virtual Machine based on the vSphere Security Hardening Guideline, the list just goes on and on. Outside of extreme troubleshooting scenarios (debatable unless directed by VMware GSS), you should be leveraging the vSphere API to automate these changes and instead of manually editing the VMX file by hand.

Disclaimer: No kittens were harmed during the writing of this article

There are several disadvantages when editing a VMX file by hand:

  • Very error prone, you might get it right for the first VM but what about 500 VMs with couple of settings each?
  • VMX editing requires the VM to be powered off
  • Requires ESXi host to have SSH enabled unless you plan to manually download/re-upload the edited VMX file
  • Changes to VMX file must also be reloaded for changes to take affect

I think there are two main reasons customers are manually editing VMX files by hand. First, it might seem to be the quickest way but looking at the list of disadvantages, you will soon realize it really is not when you need to scale this change beyond a single Virtual machine. The second reason I think is the perceived intuitiveness of the change. I mean, all configuration changes made to a Virtual Machine whether it is using the vSphere Web/C# Client or vSphere API will need to eventually persist onto disk and those changes is what is saved into Virtual Machine's VMX configuration file. However, that is not the only way nor would I say the "correct" way of modifying your Virtual Machines when it comes to these and other settings. As I have already mentioned, you can already use the vSphere Web/C# Client to make these changes which in turn leverage the vSphere API. So why would it be any different if you were to perform these operations outside of a UI?

In fact, the vSphere API abstracts away the complexity of needing to edit the VMX file and provides nice programmatic/scripting interfaces to perform some of these Virtual Machine configurations. Taking our two existing examples, Change Block Tracking is actually exposed through a simple boolean property called changeTrackingEnabled and you can just call the ReconfigVM_Task() method to enable this functionality to ensure that you have efficient Virtual Machine backups. You do not need to manually tweak each Virtual Disk since that will automatically be handled by vSphere once this has been configured.

The second example is around hardening your Virtual Machines based on the vSphere Security Hardening Guide. Luckily, to implement those settings among other Virtual Machine Advanced Settings, it just simply an array property called extraConfig that accepts a list of key/value pairs. In fact, you can easily automate the configuration of one or dozen parameters using the scripts found in this blog article called Accessing Virtual machine Advanced Settings which I wrote a couple years back. If you want to automatically harden your Virtual Machines based on a set of pre-defined settings, you can also take a look at my Automate the hardening of your Virtual Machine VMX Configurations article.

The point that I am trying to make here is that though these changes may eventually end up in the Virtual Machines VMX configuration file, there are better, easier and more efficient ways of making these changes through the use of the vSphere API and that is one of the many benefits of using an API. If you have gotten this far in the article, then I would like to conclude with some of the benefits I mentioned earlier that you will not find if you were to manually edit a VMX file by hand. The first is that the Virtual Machine Advanced Settings can be applied while the Virtual Machine is up and running, yes the changes will not go into effect, but you can at least stage the changes. If really can not take down time of the Virtual Machine to completely power off and power on for the changes to reload, another trick that I learned from a friend in the VMware Community is to vMotion the Virtual Machine from one host to another. By doing a vMotion, a new VMX will be created on the destination host and the Virtual Machine Advanced Settings will then become active as a new VMX process is spawned. This is not something you can do if you manually edited the VMX file by hand 🙂

Categories // vSphere Tags // vmx, vSphere API

  • « Previous Page
  • 1
  • …
  • 93
  • 94
  • 95
  • 96
  • 97
  • …
  • 109
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Automating the vSAN Data Migration Pre-check using vSAN API 06/04/2025
  • VCF 9.0 Hardware Considerations 05/30/2025
  • VMware Flings is now available in Free Downloads of Broadcom Support Portal (BSP) 05/19/2025
  • VMUG Connect 2025 - Minimal VMware Cloud Foundation (VCF) 5.x in a Box  05/15/2025
  • Programmatically accessing the Broadcom Compatibility Guide (BCG) 05/06/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...