WilliamLam.com

  • About
    • About
    • Privacy
  • VMware Cloud Foundation
  • VKS
  • Homelab
    • Hardware Options
    • Hardware Reviews
    • Lab Deployment Scripts
    • Nested Virtualization
    • Homelab Podcasts
  • VMware Nostalgia
  • Apple

Search Results for: guest operations

vSphere 6.0 Update 2 hints at Nested ESXi support for Paravirtual SCSI (PVSCSI) in the future

03.14.2016 by William Lam // 6 Comments

Although Nested ESXi (running ESXi in a Virtual Machine) is not officially supported today, VMware Engineering continues to enhance this widely used feature by making it faster, more reliable and easier to consume for our customers. I still remember that it was not too long ago that if you wanted to run Nested ESXi, several non-trivial and manual tweaks to the VM's VMX file were required. This made the process of consuming Nested ESXi potentially very error prone and provide a less than ideal user experience.

Things have definitely been improved since the early days and here are just some of the visible improvements over the last few years:

  • Prior to vSphere 5.1, enabling Virtual Hardware Assisted Virtualization (VHV) required manual edits to the VMX file and even earlier versions required several VMX entries. VHV can now be easily enabled using either the vSphere Web Client or the vSphere API.
  • Prior to vSphere 5.1, only the e1000{e} networking driver was supported with Nested ESXi VMs and although it was functional, it also limited the types of use cases you might have for Nested ESXi. A Native Driver for VMXNET3 was added in vSphere 5.1 which not only increased the performance that came with using the optimized VMXNET3 driver but it also enabled new use cases such testing SMP-FT as it was now possible to get 10Gbe interface to Nested ESXi VM versus the traditional 1GBe with e1000{e} driver.
  • Prior to vSphere 6.0, selection of ESXi GuestOS was not available in the "Create VM" wizard which meant you had to resort to re-editing the VM after initial creation or using the vSphere API. You can now select the specific ESXi GuestOS type directly in the vSphere Web/C# Client.
  • Prior to vSphere 6.0, the only way to cleanly shutdown or power cycle a Nested ESXi VM was to perform the operation from within the system as there was no VMware Tools support. This changed with the development of a VMware Tools daemon specifically for Nested ESXi which started out as a VMware Fling. With vSphere 6.0, the VMware Tools for Nested ESXi was pre-installed by default and would automatically startup when it detected that it ran as a VM. In addition to power operations provided by VMware Tools, it also enabled the use of the Guest Operations API which was quite popular from an Automation standpoint.

Yesterday while working in my new vSphere 6.0 Update 2 home lab, I needed to create a new Nested ESXi VM and noticed something really interesting. I used the vSphere Web Client like I normally would and when I went to select the GuestOS type, I discovered an interesting new option which you can see from the screenshot below.

nested-esxi-changes-in-vsphere60u2-3
It is not uncommon to see VMware to add experimental support for potentially new Guest Operating Systems in vSphere. Of course, there are no guarantees that these OSes would ever be supported or even released for that matter.

What I found that was even more interesting was that when select this new ESXi GuestOS type (vmkernel65) is what was recommended as the default virtual hardware configuration for the VM. For the network adapter, it looks like the VMXNET3 driver is now recommended over the e1000e and for the storage adapter the VMware Paravirtual (PVSCSI) adapter is now recommended over the LSI Logic Parallel type. This is really interesting as it is currently not possible today to get the optimized and low overhead of the PVSCSI adapter working with Nested ESXi and this seems to indicate that PVSCSI might actually be possible in the future! 🙂

nested-esxi-changes-in-vsphere60u2-1
I of course tried to install the latest ESXi 6.0 Update 2 (not yet GA'ed) using this new ESXi GuestOS type and to no surprise, the ESXi installer was not able to detect any storage devices. I guess for now, we will just have to wait and see ...

Categories // ESXi, Nested Virtualization, Not Supported, vSphere 6.0 Tags // ESXi, nested, nested virtualization, pvscsi, vmxnet3, vSphere 6.0 Update 2

How to remotely run appliancesh & other shell commands on VCSA w/o requiring SSH?

02.25.2016 by William Lam // 13 Comments

In vSphere 6.0 Update 1, the vCenter Server Appliance (VCSA) has received a significant enhancement to its Virtual Machine Management Interface also known as VAMI for short. As the name suggests, this interface provides basic configuration, monitoring and management capabilities for the Virtual Appliance which can be consumed through either a UI using a web browser or from the appliancesh CLI running within the VCSA Shell.

vcenter-server-appliance-appliancesh-and-other-commands-without-ssh-0
When talking to customers, they love the fact that the VCSA is harden out of the box and things like SSH are disabled by default. However, one challenge today is that if you need to access the appliancesh interface, SSH still must be enabled or direct console access would be required which is not ideal from an automation as well as from a security standpoint. Although things like SNMP can be configured on the VCSA to help alleviate some of these challenges, it does not solve the problem of having programmatic and remote management access.

VMware Engineering is aware of this request and is working on exposing the VAMI capabilities as an API in a future release of vSphere. In the mean time, not all hope is lost and there is still a solution which does not require you to give up security to be able to operate and manage your VCSA. We can do so by leveraging one of my all time favorite features of the vSphere Platform which is the Guest Operations API which allows you perform guest operations (running commands, transferring files, etc) directly within the guestOS as if you were logged in. Valid guest credentials are still required and once authenticated, the operations are then proxied through VMware Tools. Networking is not even required which makes this a really handy feature for troubleshooting and can even extend into application level provisioning using a single API. I can not stress enough on how cool and underutilized this feature is and it still comes as a surprise when I tell customers that this is actually possible.

Customers can consume the Guest Operations API by consuming it through one of our many supported vSphere SDKs as I have shown here or you can also consume it through PowerCLI using the Invoke-VMSCript cmdlet. To demonstrate the power of the Guest Operations API with the VCSA, I will completely disable all remote access to the VCSA which includes Local Login, Bash Shell and SSH as shown in the screenshot below.

vcenter-server-appliance-appliancesh-and-other-commands-without-ssh-1
Here is an example of running a simple "echo" command using the vSphere SDK for Perl:

vcenter-server-appliance-appliancesh-and-other-commands-without-ssh-2
Note: You will notice that there is no output and that is because the standard output must be re-directed to a file and then downloaded back to your client. The PowerCLI's Invoke-VMScript does handle this for you and will return any standand output to the console. For more complex commands, I would recommend creating a script that contains the command and just running the script itself which you can then log locally or into a file.

Here is an example of running the "appliancesh" command using the Invoke-VMScript cmdlet:

Invoke-VMScript -ScriptText "echo 'VMware1!' | appliancesh help pi list
" -vm VCSA-No-SSH -GuestUser root -GuestPassword VMware1!

vcenter-server-appliance-appliancesh-and-other-commands-without-ssh-4
Here is an example of running the "cmsso-util" command using the Invoke-VMScript cmdlet:

Invoke-VMScript -ScriptText "export VMWARE_VAPI_HOME=/usr/lib/vmware-vapi
export VMWARE_RUN_FIRSTBOOTS=/bin/run-firstboot-scripts
export VMWARE_DATA_DIR=/storage
export VMWARE_INSTALL_PARAMETER=/bin/install-parameter
export VMWARE_LOG_DIR=/var/log
export VMWARE_OPENSSL_BIN=/usr/bin/openssl
export VMWARE_TOMCAT=/opt/vmware/vfabric-tc-server-standard/tomcat-7.0.55.A.RELEASE
export VMWARE_RUNTIME_DATA_DIR=/var
export VMWARE_PYTHON_PATH=/usr/lib/vmware/site-packages
export VMWARE_TMP_DIR=/var/tmp/vmware
export VMWARE_PERFCHARTS_COMPONENT=perfcharts
export VMWARE_PYTHON_MODULES_HOME=/usr/lib/vmware/site-packages/cis
export VMWARE_JAVA_WRAPPER=/bin/heapsize_wrapper.sh
export VMWARE_COMMON_JARS=/usr/lib/vmware/common-jars
export VMWARE_TCROOT=/opt/vmware/vfabric-tc-server-standard
export VMWARE_PYTHON_BIN=/opt/vmware/bin/python
export VMWARE_CLOUDVM_RAM_SIZE=/usr/sbin/cloudvm-ram-size
export VMWARE_VAPI_CFG_DIR=/etc/vmware/vmware-vapi
export VMWARE_CFG_DIR=/etc/vmware
cmsso-util --help
" -vm VCSA-No-SSH -GuestUser root -GuestPassword VMware1!

Note: The reason the additional "export" commands are required is that certain commands may rely on certain environmental variables to be setup. In the case of the cmsso-util command, there are several VMware environmental variables it uses. I decided to just export them all but you can selectively figure out which ones are truly needed.

vcenter-server-appliance-appliancesh-and-other-commands-without-ssh-4
As you can see from the examples above, I was able to successfully run both shell commands as well as the appliancesh without requiring SSH and even local login! This methods works whether you are connected to vCenter Server or ESXi host from vSphere API perspective.

UPDATE (06/06/19) - Example joining the VCSA to Active Directory using domainjoin-cli

Invoke-VMScript -ScriptText "echo 'VMware1!' | /opt/likewise/bin/domainjoin-cli join vmware.corp administrator
" -vm VCSA -GuestUser root -GuestPassword VMware1!

Categories // Automation, VCSA, vSphere 6.0 Tags // appliancesh, cmsso-util, invoke-vmscript, ssh, vcenter server appliance, VCSA, vcva, vSphere 6.0

Automating vRealize Automation 7 Minimal Install: Part 3 - vRA Appliance Configuration

02.16.2016 by William Lam // 3 Comments

In Part 3 of this blog series, we will now move onto configuring the vRA Appliance which includes setting up the VMware Identity Manager (Horizon SSO). There are two mandatory variables that you will need to edit prior to executing the configurevRA-Appliance.sh shell script. In addition, there are few optional variables that you can also configure which includes specifying a license key for vRA.

Variable Description Required
HORIZON_SSO_PASSWORD SSO Password Yes
NTP_SERVER NTP Server Yes
VRA_LICENSE_KEY vRA license key No
VRA_SSL_CERT_COUNTRY SSL cert No
VRA_SSL_CERT_STATE SSL cert State No
VRA_SSL_CERT_ORG SSL cert Org No
VRA_SSL_CERT_ORG_UNIT SSL cert OU No

Once you have saved the changes to the script, you will need to run the script directly on the vRA Appliance. You can do so by uploading the script (SCP) to the vRA Appliance and then running it locally. If you prefer to run it remotely, you can leverage any existing SSH tools or if you prefer a Windows solution, something like plink or leveraging the vSphere Guest Operations API by using PowerCLI's Invoke-Guest cmdlet. By default, the script outputs all the verbose logging into /var/log/vra-appliance-configuration.log if you would like to get more details or perform some troubleshooting.

Here is an example of running the script locally on the vRA Appliance:

automate-vra7-appliance-configuration-3
The script can take up to several minutes to configure and the high level steps are outputted to the screen console. Once the script has successfully completed, you can verify that everything is properly configured by logging into the Horizon SSO interface by opening a browser to the following URL: https://[VRA-APPLIANCE]/vcac which is also displayed in the output. You will login using "administrator" and the SSO password you had selected earlier. If you get a 404 when getting to the /vcac URL, you may just need to give it another 30 seconds and then refresh the page.

automate-vra7-appliance-configuration-0
If you did not specify a vRA license, once logged in, you should see an "Invalid License" message. If you did specify a license, then you should see the vRA web interface as shown in the screenshot below. In case you get some strange errors after successfully logging in, you may need to wait a few minutes while the system finish initializing and then re-log back in.

automate-vra7-appliance-configuration-2
In our fourth and final part of the blog series, we will tackle automating the the vRA IaaS Windows components from the vRA Appliance itself. This will include setting up the SSL certificates for both the Web/Manager Service and the installation of Web/Manager Service, Database, DEM Orchestrator, DEM Worker and vSphere Agent. Stay tuned!

  • Automating vRealize Automation 7 Minimal Install: Part 1 - vRA Appliance Deployment
  • Automating vRealize Automation 7 Minimal Install: Part 2 - vRA IaaS Agent Deployment
  • Automating vRealize Automation 7 Minimal Install: Part 3 - vRA Appliance Configuration
  • Automating vRealize Automation 7 Minimal Install: Part 4 - vRA IaaS Configuration

Categories // Automation Tags // powershell, vRA 7, vRealize Automation

  • « Previous Page
  • 1
  • …
  • 6
  • 7
  • 8
  • 9
  • 10
  • …
  • 21
  • Next Page »

Search

Thank Author

Author

William is Distinguished Platform Engineering Architect in the VMware Cloud Foundation (VCF) Division at Broadcom. His primary focus is helping customers and partners build, run and operate a modern Private Cloud using the VMware Cloud Foundation (VCF) platform.

Connect

  • Bluesky
  • Email
  • GitHub
  • LinkedIn
  • Mastodon
  • Reddit
  • RSS
  • Twitter
  • Vimeo

Recent

  • Ultimate Lab Resource for VCF 9.0 06/25/2025
  • VMware Cloud Foundation (VCF) on ASUS NUC 15 Pro (Cyber Canyon) 06/25/2025
  • VMware Cloud Foundation (VCF) on Minisforum MS-A2 06/25/2025
  • VCF 9.0 Offline Depot using Synology 06/25/2025
  • Deploying VCF 9.0 on a single ESXi host? 06/24/2025

Advertisment

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Copyright WilliamLam.com © 2025

 

Loading Comments...