WilliamLam.com

Datastore File Management using vCLI vifs

by // Leave a Comment

There are many useful scripts that are bundled with the VMware vCLI, one such script, that is not very well known is the vifs utility which provides datastore file management. When you right click on a datastore and browse using the vSphere Client, you can create a new folder, download/upload, delete and move files.

Using the vCLI's vifs utility, you can perform the same set of operations via the command-line and behind the scenes it uses the vSphere API fileManager to perform these operations. You can also browse datastore by just having access to a web browser, just point it to the following address: https://[ESXI_HOSTNAME]/folder and you can access the datastores by clicking through the links.

To browse the datastore using vifs, you will need vCLI installed on either a Windows/Linux system or you may use VMware vMA.

To browse a specific datastore for an ESXi host, you will need to first list the available datastores by using the following command: vifs --server [SERVER] --username [USERNAME] --listds

Once you have identified the datastore you are interested in, you will then use the --dir flag to list the contents of the directory and their sub-directories by using the following command: vifs --server [SERVER] --username [USERNAME] --dir '[DATASTORENAME]'

Note: The format of the datastore name must be in brackets '[datastorename]' which is how a datastore path is identified in the vSphere API. To list sub-directories, you will need a space between the datastore name and the directory name and do not forget to quote the parameter

Let's say you would like to download the .vmx configuration file for in the directory, you can use --get flag to by using the following command:

vifs --server [SERVER] --username [USERNAME] --get '[DATASTORENAME] somedir/somefile.vmx'

Note: In the example above, we are downloading the file in the current working directory denoted by the "." (period). If you wish to download it somewhere else or even renaming the file, you will need to specify the full path to the destination


If you wanted to automate the downloading of say all .vmx configuration files, it might be pretty tedious to run through the directory discovery, so here is a quick shell script called getVMVMX.sh that is more user friendly that allows you to easily download all .vmx configurations for a given datastore.

To use the script, you will need vCLI installed on either a Linux system or use VMware vMA and be sure to set the executable permission on the shell script. You will need to specify the credentials to the ESX(i) host and the specific datastore you wish to either "list" or "download" all .vmx configuration files.

Using the --listds flag, you will need to identify the datastore you wish to use. Next you will use the following command to "list" all .vmx configuration file: ./getVMVMX.sh [ESXI_SERVER] [USERNAME] "[PASSWORD]" [DATASTORE] list

To download all .vmx configuration file you will use the following command:

./getVMVMX.sh [ESXI_SERVER] [USERNAME] "[PASSWORD]" [DATASTORE] download [FOLDER]

where FOLDER is a directory that will automatically be created for you to store all .vmx configuration files

Note: You can easily modify the script to add an additional "for loop" at the beginning to automatically download .vmx configurations for all datastores. I will leave that as an exercise for the reader.

So if you ever need to grab a vmware.log file for a specific VM or upload an ISO to datastore, you can do so from the command-line using the vifs utility that is bundled with the vCLI.

How to Run Windows 8 Consumer Preview & Windows 8 Server on vSphere 5

by // 15 Comments

A few weeks back I wrote an article on how to run Windows 8 Developer Preview on ESXi 5, which required a small trick to get working. Today, it looks like Microsoft just released the Windows 8 Consumer Preview and Windows 8 Server. For those looking to give these a try, you can do so by using vSphere 5.

Disclaimer: VMware still does not officially support Windows 8 on vSphere, this is mainly for educational and testing purposes. Do not use this in a production environment

UPDATE (06/02/12): The steps below are the same for the recent release of Windows 8 Release Preview and Windows Server 2008 (Windows 8 Server) RC.

Here are the three easy steps to run Windows 8 on ESXi 5

  1. Download and install ESXi500-201112001 (patch02) from VMware patch repository.
  2. Create either a Windows 7 or Windows 2008 R2 regular VM (You will need to use e1000 network driver, VMXNET3 does not work)
  3. Install Windows 8 Consume Preview or Server using the ISO (tested with 64bit) 

Note: Unlike the Developer Preview, no additional tweaks are needed to install Windows 8 Consumer Preview other than the ESXi 5.0 patch. 

    When you first boot up Windows 8, you should see this fish logo. Be patient and give it a few minutes to load:

     Shortly after, you will be prompted to start the installation:

    Here is a screenshot of Windows 8 Consumer Preview running on ESXi 5.0:

    Here is a screenshot of Windows 8 Server running on ESXi 5.0:

    Here is a screenshot of Windows 8 Server w/Hyper-V role enabled running on ESXi 5.0:

    To enable Hyper-V role, you will need to perform the following changes:

    • Update the guestOS to Windows 8 64bit using the vSphere Client
    • Add the following CPUID bits using the vSphere Client
    • Add the advanced setting using the vSphere Client hypervisor.cpuid.v0 = FALSE as described by this article as well as mce.enabled = TRUE (this is a new parameter introduced in ESXi 5) which is needed to run Hyper-V3 in Windows 8
    • Install Hyper-V role
    • Shutdown host
    • Remove advanced setting using the vSphere Client
    • Host should now boot and you should see the Hyper-V manager interface

    Note: You can install VMware Tools on either Windows 8 releases by right clicking on the VM->Guest->Install VMware Tools. If you install VMware Tools on Windows 8 Server, you may need to enable 3D support if you get a black screen after installation as noted in this article.

    So if you are itching to try out the new Windows 8 Consumer Preview or Server, you can install it and run it on vSphere 5!

    How to Access vCloud Director Remote Console using vCloud & VMRC API

    by // 17 Comments

    If you are familiar with the vCloud Director UI and have used the virtual console for a virtual machine, you may have noticed it is serviced by the VMRC (Virtual Machine Remote Console) vCD browser plugin which is only accessible in the vCD UI. If you are building a custom provisioning portal, you may want to provide similar functionality in your own portal for your users to access their VMs.

    You can do so by leveraging the vCloud API and the new VMRC API that was made available with the release of vCloud Director 1.5.

    In addition to the VMRC API documentation, there is an sample VMRC API Reference Implementation that you can download and experiment with. When you extract the contents of the zip file, you will find an index.html and console.html web page and you can use this locally on your system or host it on a web server.

    The index.html is the landing page that you will use to provide the vCloud VM's screen ticket using the vCloud API's AcquireTicket REST API method. The console.html will then load the VMRC for the requested VM assuming you provided a valid screen ticket.

    Here are the four simple steps to access a vCloud VM's remote console using the vCloud REST API & VMRC API using curl:

    Step 1 - Login

    You will need a system that has curl installed and you will need an account in vCD that has access to some powered on VMs. In the example below, I will be logging into a regular organization and not the System organization. There are a few parameters you need to specify to login to vCD and obtain an authorization token. You will need to specify the following parameters and the URL to your vCloud Director instance which should be in the form of https://vcd-fqdn/api/sessions:

    • -i = Include headers
    • -k = Performs an "insecure" SSL connection
    • -H = Setting the header for the version of vCloud Director (1.5 in this example)
    • -u = User credentials in the format of [username@org:password]
    • -X = Request type

    Note: For more details on the cURL flags, please refer to the cURL documentation.

    curl -i -k -H "Accept:application/*+xml;version=1.5" -u coke-admin@Coke:vmware -X POST https://vcd.primp-industries.com/api/sessions

    HTTP/1.1 200 OK
Date: Tue, 28 Feb 2012 22:20:54 GMT
x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=
Set-Cookie: vcloud-token=n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=; Secure; Path=/
Content-Type: application/vnd.vmware.vcloud.session+xml;version=1.5
Date: Tue, 28 Feb 2012 22:20:55 GMT
Content-Length: 894


<?xml version="1.0" encoding="UTF-8"?>
<Session xmlns="http://www.vmware.com/vcloud/v1.5" user="coke-admin" org="Coke" type="application/vnd.vmware.vcloud.session+xml" href="https://vcd.primp-industries.com/api/session/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
<Link rel="down" type="application/vnd.vmware.vcloud.orgList+xml" href="https://vcd.primp-industries.com/api/org/"/>
<Link rel="down" type="application/vnd.vmware.admin.vcloud+xml" href="https://vcd.primp-industries.com/api/admin/"/>
<Link rel="down" type="application/vnd.vmware.vcloud.query.queryList+xml" href="https://vcd.primp-industries.com/api/query"/>
<Link rel="entityResolver" type="application/vnd.vmware.vcloud.entity+xml" href="https://vcd.primp-industries.com/api/entity/"/>
</Session>

    If you have successfully logged in, you should get an HTTP 200 response and get similar output as above. You will need to make note of your authorization token which is located on the third line that starts with "x-vcloud-authorization". This will be needed throughout the remainder of the session

    Step 2 - Query All Powered On VMs

    Using the new Query Service API in vCloud 1.5, we will locate all VMs within the vCD instance with the vm type. If you are logged in as a System Administrator and using the System Org, use the adminVM type instead. You can see the command is very similar to the one above but instead of specifying the credentials, we are using the authorization token from step 1. We are also including specific fields in the XML output for readability (VM name + vApp name), if you remove '&fields=name,containerName', you will see the other fields in the XML response. We also want to filter for only powered-on VMs, we can do this by adding '&filter=status==POWERED_ON'.

    curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=" -X GET "https://vcd.primp-industries.com/api/query?type=vm&filter=status==POWERED_ON&fields=name,containerName"

    HTTP/1.1 200 OK
Date: Tue, 28 Feb 2012 22:22:09 GMT
Content-Type: application/*+xml;version=1.5
Date: Tue, 28 Feb 2012 22:22:09 GMT
Content-Length: 1356

<?xml version="1.0" encoding="UTF-8"?>
<QueryResultRecords xmlns="http://www.vmware.com/vcloud/v1.5" total="2" pageSize="25" page="1" name="vm" type="application/vnd.vmware.vcloud.query.records+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=records&filter=status==POWERED_ON&fields=name,containerName" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">
<Link rel="alternate" type="application/vnd.vmware.vcloud.query.references+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=references&filter=status==POWERED_ON&fields=name,containerName"/>
<Link rel="alternate" type="application/vnd.vmware.vcloud.query.idrecords+xml" href="https://vcd.primp-industries.com/api/query?type=vm&page=1&pageSize=25&format=idrecords&filter=status==POWERED_ON&fields=name,containerName"/>
<VMRecord name="WebServerVM" containerName="WebServer-Test-1" href="https://vcd.primp-industries.com/api/vApp/vm-25a4228e-1202-45a7-8f85-cafaeb1c3bc8"/>
<VMRecord name="ApplicationVM" containerName="AppTesting" href="https://vcd.primp-industries.com/api/vApp/vm-fadffc47-fa24-4ca0-9a90-f1048bc9f731"/>
</QueryResultRecords>

    If the operation was successful, you should see a HTTP 200 response and list of VMs in your vCD instance. You will need to make a note of the VM's href property that you wish to obtain the remote console screen ticket for which is needed in the next step.

    Step 3 - Obtain Screen Ticket for VM

    Next we need to obtain the screen ticket using the AcquireTicket vCloud REST API method. You will need to perform a POST operation and using the href property from the previous step, append "/screen/action/acquireTicket". This link is only available IF the VM is powered-on and must be used within 30seconds or it will be considered invalid.

    curl -i -k -H "Accept:application/*+xml;version=1.5" -H "x-vcloud-authorization: n/+ZglmIwJ2SbWlyz04XC5sHUDJqefU1mCMa9TFu9lk=" -X POST https://vcd.primp-industries.com/api/vApp/vm-25a4228e-1202-45a7-8f85-cafaeb1c3bc8/screen/action/acquireTicket

    HTTP/1.1 200 OK Date: Tue, 28 Feb 2012 22:23:21 GMT Content-Type: application/vnd.vmware.vcloud.screenticket+xml;version=1.5 Date: Tue, 28 Feb 2012 22:23:21 GMT Content-Length: 895 <?xml version="1.0" encoding="UTF-8"?> <ScreenTicket xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://172.30.0.139/api/v1.5/schema/master.xsd">mks://172.30.0.140/vm-180?ticket=cst-HtefHNvLNiLmfK1na3mx74rqmLmM%2FjhWu3UHHTFrRlnGZg%2FtgjfqwanE4HUvWgvaclaR5vRI3iJrOvJS4P%2Fq6vH%2FfU06v0kMJ4U1ngSCNnt2SmHpaazWp%2B7JPLMCPQz0f88bxnU3t2NIX3bn%2BrsIznRNaDkQeIEldti%2F7pWsH%2FWm1PGIAk%2Fh54IpBuaGmlwPtNjs40zGTLjSYzRtO9oII75jgWA%2F%2FVcRBXtIPgoQzL4GHpyS6PZ7vaTklxdjCUJklJnlAbpVjo7P5uUqzfhhFQa52GNxZK%2FCLS6nNJpVkmvV12J5aUwsWe3OXdecsckW%2By3VLdcKe9Fp8N44TEgGfqrjzvCsXOv5DbiChjrfhIBKlUhN2%2BHL2fHqbtrBbKvtteNERMl08cqExR7qyf4n1w%3D%3D-X6PBbcMUeRCaF5G6lkqOrrdPbjm%2FfLOj1Exk8w%3D%3D--tp-9B%3A2B%3A82%3A8D%3AA2%3AFD%3AF1%3A8F%3A42%3A20%3ABF%3A75%3AA9%3A4F%3AFA%3AA6%3AC4%3A09%3ACA%3A4F--</ScreenTicket>

    Now we just need to take the screen ticket which starts with mks://..... and copy that over to our VMRC sample web page and hit the submit button. If everything was successful, you should see the a remote console popup up for the VM you had just requested the screen ticket from. You will also notice the IP Address in the screen ticket is not the same address as your vCloud Director Cell, it is actually the console proxy address if you were wondering why the IP is different.

    So this is great, but you still need some interaction with the browser to actually submit the screen ticket request. What if we just wanted to provide a single URL that would automatically launch the remote console for a requested VM? You can definitely do so, but you will need to perform a URL encoding on the mks://... ticket and construct the proper URL.

    Here is a simple shell script called vcloud-vAppRemoteConsole.sh that uses the same vCloud REST API calls as above and allows a user to select the vCloud VM to to retrieve a screen ticket and returns back a single URL to access the remote console of the VM.

    The script requires three parameters: (username@organization, password and vcloud-host)

    Here is an sample execution:

    Once you have successfully logged in, you should see a list of VMs that are currently powered on (highlighted in blue). You will need to select a VM and provide the "href" property as input (highlighted in green). Then the AcquireTicket method will be called and the screen ticket is then encoded using a quick Perl one-liner. The encoded entry is then appended to the hosted URL of the VMRC sample https://air.primp-industries.com/vmrc/console.html? (you can replace the URL to fit your environment). The output of the script (highlighted in orange) is the URL you would then paste into a supported web browser and the remote console will automatically launch without any additional user input.

    With the above knowledge, you can easily translate this using any of the vCloud SDKs (Java, .NET, PHP), vCO or the new vCloud Director cmdlets and provide console access to your vCloud VMs.

    Note: For vSphere virtual machine remote console access instructions, please take a look at this article here.