William Lam

How to remotely run appliancesh & other shell commands on VCSA w/o requiring SSH?

02.25.2016 by William Lam // 13 Comments

In vSphere 6.0 Update 1, the vCenter Server Appliance (VCSA) has received a significant enhancement to its Virtual Machine Management Interface also known as VAMI for short. As the name suggests, this interface provides basic configuration, monitoring and management capabilities for the Virtual Appliance which can be consumed through either a UI using a web browser or from the appliancesh CLI running within the VCSA Shell.

When talking to customers, they love the fact that the VCSA is harden out of the box and things like SSH are disabled by default. However, one challenge today is that if you need to access the appliancesh interface, SSH still must be enabled or direct console access would be required which is not ideal from an automation as well as from a security standpoint. Although things like SNMP can be configured on the VCSA to help alleviate some of these challenges, it does not solve the problem of having programmatic and remote management access.

VMware Engineering is aware of this request and is working on exposing the VAMI capabilities as an API in a future release of vSphere. In the mean time, not all hope is lost and there is still a solution which does not require you to give up security to be able to operate and manage your VCSA. We can do so by leveraging one of my all time favorite features of the vSphere Platform which is the Guest Operations API which allows you perform guest operations (running commands, transferring files, etc) directly within the guestOS as if you were logged in. Valid guest credentials are still required and once authenticated, the operations are then proxied through VMware Tools. Networking is not even required which makes this a really handy feature for troubleshooting and can even extend into application level provisioning using a single API. I can not stress enough on how cool and underutilized this feature is and it still comes as a surprise when I tell customers that this is actually possible.

Customers can consume the Guest Operations API by consuming it through one of our many supported vSphere SDKs as I have shown here or you can also consume it through PowerCLI using the Invoke-VMSCript cmdlet. To demonstrate the power of the Guest Operations API with the VCSA, I will completely disable all remote access to the VCSA which includes Local Login, Bash Shell and SSH as shown in the screenshot below.

Here is an example of running a simple "echo" command using the vSphere SDK for Perl:

Note: You will notice that there is no output and that is because the standard output must be re-directed to a file and then downloaded back to your client. The PowerCLI's Invoke-VMScript does handle this for you and will return any standand output to the console. For more complex commands, I would recommend creating a script that contains the command and just running the script itself which you can then log locally or into a file.

Here is an example of running the "appliancesh" command using the Invoke-VMScript cmdlet:

Invoke-VMScript -ScriptText "echo 'VMware1!' | appliancesh help pi list
" -vm VCSA-No-SSH -GuestUser root -GuestPassword VMware1!

Here is an example of running the "cmsso-util" command using the Invoke-VMScript cmdlet:

Invoke-VMScript -ScriptText "export VMWARE_VAPI_HOME=/usr/lib/vmware-vapi
export VMWARE_RUN_FIRSTBOOTS=/bin/run-firstboot-scripts
export VMWARE_DATA_DIR=/storage
export VMWARE_INSTALL_PARAMETER=/bin/install-parameter
export VMWARE_LOG_DIR=/var/log
export VMWARE_OPENSSL_BIN=/usr/bin/openssl
export VMWARE_TOMCAT=/opt/vmware/vfabric-tc-server-standard/tomcat-7.0.55.A.RELEASE
export VMWARE_RUNTIME_DATA_DIR=/var
export VMWARE_PYTHON_PATH=/usr/lib/vmware/site-packages
export VMWARE_TMP_DIR=/var/tmp/vmware
export VMWARE_PERFCHARTS_COMPONENT=perfcharts
export VMWARE_PYTHON_MODULES_HOME=/usr/lib/vmware/site-packages/cis
export VMWARE_JAVA_WRAPPER=/bin/heapsize_wrapper.sh
export VMWARE_COMMON_JARS=/usr/lib/vmware/common-jars
export VMWARE_TCROOT=/opt/vmware/vfabric-tc-server-standard
export VMWARE_PYTHON_BIN=/opt/vmware/bin/python
export VMWARE_CLOUDVM_RAM_SIZE=/usr/sbin/cloudvm-ram-size
export VMWARE_VAPI_CFG_DIR=/etc/vmware/vmware-vapi
export VMWARE_CFG_DIR=/etc/vmware
cmsso-util --help
" -vm VCSA-No-SSH -GuestUser root -GuestPassword VMware1!

Note: The reason the additional "export" commands are required is that certain commands may rely on certain environmental variables to be setup. In the case of the cmsso-util command, there are several VMware environmental variables it uses. I decided to just export them all but you can selectively figure out which ones are truly needed.

As you can see from the examples above, I was able to successfully run both shell commands as well as the appliancesh without requiring SSH and even local login! This methods works whether you are connected to vCenter Server or ESXi host from vSphere API perspective.

UPDATE (06/06/19) - Example joining the VCSA to Active Directory using domainjoin-cli

Invoke-VMScript -ScriptText "echo 'VMware1!' | /opt/likewise/bin/domainjoin-cli join vmware.corp administrator
" -vm VCSA -GuestUser root -GuestPassword VMware1!

Automating vRealize Automation 7 Minimal Install: Part 4 - vRA IaaS Configuration

02.17.2016 by William Lam // 14 Comments

If you have been following the blog series thus far, we have covered deploying the vRA 7 Virtual Appliance, installing the vRA IaaS Management Agent on a Microsoft Windows system which will run the various IaaS components and then finally configuring the vRA Virtual Appliance which includes setting up the VMware Identity Manager (Horizon SSO). Before proceeding to the vRA IaaS installation, you will need to make sure that you have completed all three of steps above.

In addition, there are some other prerequisites (more details here) that are required on the Microsoft Windows system in which you plan to install the vRA IaaS components, namely an instance of Microsoft SQL Server running and various Windows configuration tweaks required by the installers. I will not be covering the installation of the DB, so this will be something you will need to either manually install or automate using a silent installer for SQL Server. For the pre-checks, although this is also not covered in the script, I will show you how you can run the same tool on the Windows system like you would using the new vRA 7 Guided Wizard. The tool will identify any configuration issues found and then also allow you to easily remediate them within the tool, which is an awesome feature in my opinion.

You can find the vRA IaaS Pre-req Checker tool on the vRA Appliance under the following path:

/opt/vmware/share/htdocs/service/iaas/download/PrereqChecker.zip

You will then need to SCP the PrereqChecker.zip file onto your Windows system that you plan to run the vRA IaaS components. Next, extract the contents of the zip and launch the PrereqChecker.exe. Before clicking on the "Run Checker" button, make sure you have enabled the IIS role, else the tool can not run all prechecks. The instructions are located to the right of the screen and once that has been completed, you can then run the precheck tool. For any issues that have been identified, you can remediate by clicking on the "Fix Issue" button. Once all prechecks have passed, you can then move onto the vRA IaaS installation.

Note: I was a bit surprised to see that the PrereqChecker.zip could not be downloaded directly from the vRA Appliance like the other installers by simply opening a browser to the following URL:

https://[VRA-HOSTNAME]:5480/installer/PrereqChecker.zip

It turns out there an index.py script which defines which files can be downloaded without authentication. If you wish to change this behavior, you can run the following snippet on the vRA Appliance and then you can download the zip file directly from the browser which can be useful from an automation standpoint.

sed -i "s/'DBUpgrade.zip'/'DBUpgrade.zip','PrereqChecker.zip'/g" /opt/vmware/share/htdocs/service/iaas/index.py

If you have made it to this point, we are now ready to get our automation on! We will be installing the following vRA IaaS components which are listed below onto the Windows system which has the vRA IaaS Management Agent running.

SSL Certificate for Web and Manager Service
Database
Web API (WAPI) Service
Manager Service
DEM Orchestrator
DEM Worker
vSphere Agent

The automation will be completely driven from within the vRA Virtual Appliance using the configurevRA-IaaS.sh shell script. This is possible because we had deployed the vRA IaaS Management Agent earlier which will act as a proxy for all component installations. There are 9 variables that you will need to edit prior to running the script and you can find their descriptions below.

Variable	Description
HORIZON_SSO_PASSWORD	SSO Password that you had configured earlier
VRA_IAAS_HOSTNAME	Hostname of the Windows system running vRA IaaS Components
VRA_IAAS_USERNAME	Username for the Windows system
VRA_IAAS_PASSWORD	Password for the Windows system (e.g. vra-iaas\administrator)
VRA_DATABASE_HOSTNAME	Hostname of the Windows system running SQL Server (should be same as vRA IaaS system)
VRA_DATABASE_NAME	Database name
VRA_DATABASE_USERNAME	Database username (assumes Windows Auth)
VRA_DATABASE_PASSWORD	Database password (assumes Windows Auth)
VRA_DATABASE_SECURITY_PASSPHRASE	Security passphrase

Once you have saved your changes, you can then run the script on the vRA Appliance as shown in the screenshot below. All verbose output is stored in /var/log/vra-iaas-configuration.log and you will be able to see the high level operations displayed in the console. The entire process can take anywhere from 10-20 minutes depending on your environment and what you will looking for are the "INSTALLED SUCCESSFULLY" messages which I have highlighted in green below. If you have met all prereqs, you should not run into any issues but if you do, the script will output the specific errors from each of the installers.

Once the script has completed, you will now have a fully functional vRA 7 deployment which includes both the vRA Appliance as well as the vRA IaaS components! I would also like to give a big thanks to both Kim Delgado for connecting me with some of the vRA Engineering folks as well as a huge thanks to Dora L. from the vRA Engineering for assisting me with parts of the IaaS installation.

Automating vRealize Automation 7 Minimal Install: Part 3 - vRA Appliance Configuration

02.16.2016 by William Lam // 3 Comments

In Part 3 of this blog series, we will now move onto configuring the vRA Appliance which includes setting up the VMware Identity Manager (Horizon SSO). There are two mandatory variables that you will need to edit prior to executing the configurevRA-Appliance.sh shell script. In addition, there are few optional variables that you can also configure which includes specifying a license key for vRA.

Variable	Description	Required
HORIZON_SSO_PASSWORD	SSO Password	Yes
NTP_SERVER	NTP Server	Yes
VRA_LICENSE_KEY	vRA license key	No
VRA_SSL_CERT_COUNTRY	SSL cert	No
VRA_SSL_CERT_STATE	SSL cert State	No
VRA_SSL_CERT_ORG	SSL cert Org	No
VRA_SSL_CERT_ORG_UNIT	SSL cert OU	No

Once you have saved the changes to the script, you will need to run the script directly on the vRA Appliance. You can do so by uploading the script (SCP) to the vRA Appliance and then running it locally. If you prefer to run it remotely, you can leverage any existing SSH tools or if you prefer a Windows solution, something like plink or leveraging the vSphere Guest Operations API by using PowerCLI's Invoke-Guest cmdlet. By default, the script outputs all the verbose logging into /var/log/vra-appliance-configuration.log if you would like to get more details or perform some troubleshooting.

Here is an example of running the script locally on the vRA Appliance:

The script can take up to several minutes to configure and the high level steps are outputted to the screen console. Once the script has successfully completed, you can verify that everything is properly configured by logging into the Horizon SSO interface by opening a browser to the following URL: https://[VRA-APPLIANCE]/vcac which is also displayed in the output. You will login using "administrator" and the SSO password you had selected earlier. If you get a 404 when getting to the /vcac URL, you may just need to give it another 30 seconds and then refresh the page.

If you did not specify a vRA license, once logged in, you should see an "Invalid License" message. If you did specify a license, then you should see the vRA web interface as shown in the screenshot below. In case you get some strange errors after successfully logging in, you may need to wait a few minutes while the system finish initializing and then re-log back in.

In our fourth and final part of the blog series, we will tackle automating the the vRA IaaS Windows components from the vRA Appliance itself. This will include setting up the SSL certificates for both the Web/Manager Service and the installation of Web/Manager Service, Database, DEM Orchestrator, DEM Worker and vSphere Agent. Stay tuned!