datastore

How to move vSAN Datastore into a Folder?

06.14.2017 by William Lam // Leave a Comment

A question came up the other day from a customer that wanted to move a vSAN-based datastore into a vSphere Folder, but was having some trouble with this operation. vSphere Folders are commonly used by administrators for organizational purposes and/or permission delegation. When the customer tried to move their vSAN datastore into a folder using the vSphere Web Client (applies to HTML5 Web Client as well), what they found was that nothing happens even though the UI indicates the operation should be possible with the (+) symbol.

I also saw the same behavior described by the customer and was curious if this was a UI only issue or a general limitation. To quickly verify, I decided to perform the operation using the vSphere API instead of the UI. Behind the scenes, the UI simply calls the MoveIntoFolder_Task() vSphere API which allows you to move various vSphere Inventory objects into a vSphere Folder. As many of you know by now, the vSphere APIs can be consumed in variety of "SDKs" or programming/scripting languages which includes PowerCLI. For PowerCLI users, this functionality is further simplified and abstracted away using the Move-Datastore cmdlet which I will be using in our demonstration.

In my setup, I have two vSAN Datastores, one from a vSphere 6.0u3 environment and another from vSphere 6.5. Lets say I want to move the 60u3 datastore to HR folder and 66 datastore to Engineering folder. The following PowerCLI snippet below does exactly that:

Move-Datastore -Datastore (Get-Datastore "vsanDatastore-60u3") -Destination (Get-Folder "HR")
Move-Datastore -Datastore (Get-Datastore "vsanDatastore-66") -Destination (Get-Folder "Engineering")

Using the vSphere API/PowerCLI, the operation looks to have been successful. Lets now going back to our vSphere Web Client and see if the operation actually went through?

Look at that, both our vSAN Datastores is now part of a vSphere Folder! This looks like a UI (Flex/H5) only issue and I have also confirmed that this will be fixed in a future update of vSphere. For now, if you need to move vSAN-based datastores into a vSphere Folder, simply use the vSphere API as a workaround.

Note: I also found that if you need to move the vSAN Datastore back to the Datacenter level, you will also need to invoke that operation using the vSphere API as the UI also prevents this operation.

How to efficiently transfer files to Datastore in vCenter using the vSphere API?

06.18.2014 by William Lam // 19 Comments

A pretty common task for vSphere Administrators is to upload or download content from a vSphere Datastore which usually contains ISOs and floppy images. You can initiate the file transfer using the vSphere Web/C# Client, however this process can be quite tedious when having to manually upload several ISOs. Instead, you will probably want to automate this process and and there are a couple of ways in which you can accomplish this. One option, is to go directly to an ESXi host and upload your files but this is not ideal when you have vCenter Server to centrally manage your infrastructure. The second option is to go through vCenter Server, but depending on the implementation, you can potentially add unnecessary load to the vCenter Server if implemented incorrectly.

Let me explain this further with two diagrams and you can decide which implementation you prefer?

In this first implementation, I directly access the file management API which leverages a simple HTTP GET/PUT operation to upload files to a vSphere Datastore. What I found out while transferring the data was that the data actually traverses through the vCenter Server and then onto the ESXi host before writing to the vSphere Datastore. This of course made the data transfer very inefficient not to mention additional bandwidth and load being added to vCenter Server.

I created a sample vSphere SDK for Perl script that demonstrates this inefficent transfer called inefficent-upload-files-to-datastore.pl

Here is sample execution of the script which accepts the name of the vSphere Datacenter, vSphere Datastore, the source file to transfer and the destination path of where the file will be uploaded to:

./inefficent-upload.pl --config ~/vmware-dev/.vcenter55-1 --datacenter Datacenter --datastore vsanDatastore --sourcefile /Volumes/Storage/Images/CentOS-6.4-x86_64-netinstall.iso --destfile ISO/CentOS-6.4-x86_64-netinstall.iso

After talking to some folks about this problem, I learned about a more efficient method as shown in the diagram below.

As you can see, we can still initiate the transfer using the vCenter Server, but the actual data transfer is than sent to one of the ESXi hosts that has access to the vSphere Datastore. To accomplish this, we need to use the AcquireGenericServiceTicket() method which is part of the sessionManager. Using this method, we can request a ticket for a one time HTTP request to connect directly to an ESXi. To upload a file, the request must include the method which in this case will be a PUT operation and the local URL to an ESXi host that has access to the vSphere Datastore.

Here is an example of a URL: https://vesxi55-1.primp-industries.com/folder/ISO/CentOS-6.4-x86_64-netinstall.iso?dcPath=ha-datacenter&dsName=vsanDatastore

ESXi IP Address/Hostname - In the script, I select the first ESXi host that has access to the vSphere Datastore
vSphere Datastore Directory - Directory into which the contents of the file will be placed in. In this example, we just have one top-level directory called ISO which must already exist
Destination file name - The name of the file that should appear in the vSphere Datastore
Datacenter Name - This should always be ha-datacenter when connecting directly to an ESXi host
vSphere Datastore - The name of the vSphere Datastore

To demonstrate this functionality, I have created a vSphere SDK for Perl script called efficent-upload-files-to-datastore.pl which accepts the name of the vSPhere Datastore along with the source and destination of where the file will be placed:

./upload-files-to-datastore.pl --config ~/vmware-dev/.vcenter55-1 --datastore vsanDatastore --sourcefile /Volumes/Storage/Images/CentOS-6.4-x86_64-netinstall.iso --destfile ISO/CentOS-6.4-x86_64-netinstall.iso

Hopefully after looking at these two implementations, you will also agree that the second option is the best! One last thing that I would like to point out is that even though we are talking about transferring files to a vSphere Datastore, this method can also be used to efficiently transfer other supported files to an ESXi hosts through vCenter Server as described in this blog article.

Quick Tip - How to tell if a VMFS datastore is local or not using new vSphere 5.5 API?

05.27.2014 by William Lam // 3 Comments

I was recently working on a script for a friend that collects some basic information about VMFS based Datastores. While going through the vSphere API 5.5 Reference, I noticed a new property was introduced in vSphere 5.5 API called "local". It looks like we now have a simple way of checking whether a given VMFS Datastore is local or not. In the past, the only semi-reliable method for checking whether a VMFS datastore was local or not was to see if the "multipleHostAccess" property was set to true, which meant it was a shared VMFS. This was not very reliable as it could be a remote VMFS Datastore but only exported to one host so far and the other major caveat is that this property was only available when connecting to a vCenter Server.

To demonstrate this new API property, I have created an example vSphere SDK for Python (pyvmomi) sample called: list_datastore_info.py

Here is a screenshot running the script directly against an ESXi host (you can also connect to vCenter Server as well):

The script also supports a --json|-j output option, here is an example of that:

If you want to be able to format the JSON output in a more friendly manner, you can pipe the string output to python -mjson.tool