There are a number of vSphere Security enhancements that were introduced in vSphere 6.5 including the much anticipated VM Encryption feature. To be able to use the new VM Encryption feature, you will need to first setup a Key Management Interoperability Protocol (KMIP) Server if you do not already have one and associate it with your vCenter Server. There are plenty of 3rd party vendors who provide KMIP solutions that interoperate with the new VM Encryption feature, but it usually can take some time to get access to product evaluations.
During the vSphere Beta, VMware had provided a sample KMIP Server Virtual Appliance based on PyKMIP, which allowed customers to quickly try out the new VM Encryption feature. Many of you have expressed interest in getting access to this appliance for quick evaluational purposes and the team is currently working on providing an updated version of the appliance for customers to access. In the mean time, for those who can not wait for the appliance or would like an alternative way of quickly standing up a sample KMIP Server, I have created a tiny (163 MB) Docker Container which can be easily spun up to provide the KMIP services. I haver published the Docker Container on Docker Hub at lamw/vmwkmip. The beauty of the Docker Container is you do not need to deploy another VM and for resource constrained lab environments or quick demo purposes, you could even run it directly on the vCenter Server Appliance (VCSA) as shown here, obviously not recommended for production use.
The Docker Container bundles the exact same version of PyKMIP that will be included in the virtual appliance, this is just another consumption mechanism. It is also very important to note that you should NOT be using this for any production workloads or any VMs that you care about. For actual production deployments of VM Encryption, you should be leveraging a production grade KMIP Server as PyKMIP stores the encryption keys in memory and will be lost upon a restart. This will also be true even for the virtual appliance, so this is really for quick evaluational purposes.
UPDATE (10/08/22) - The KMIP Docker Container is now available for both x86 and Arm platforms. Simply run docker pull lamw/vmwkmip and the correct architecture will automatically be downloaded.
Note: The version of PyKMIP is a modified version and VMware plans to re-contribute their changes back to the PyKMIP open-source project so others can also benefit.
Below are the instructions on using the KMIP Server Docker Container and how to configure it with your vCenter Server. I will assume you have worked with Docker before, if you have not, please have a look at Docker online resources before continue further or wait for the virtual appliance to be posted.
Step 1 - On system that has a Docker Client, run the following command to pull down the Docker Container:
docker pull lamw/vmwkmip
Step 2 - Start the Docker Container by running the following command:
docker run --rm -it -p 5696:5696 lamw/vmwkmip
As you can see, the PyKMIP service has successfully started and by default, it has been configured to use the standard port which is 5696. If you do not want to run the Docker Container in interactive mode, you can run it in daemon mode by running the following command instead:
docker run -d -p 5696:5696 lamw/vmwkmip
Step 3 - Next, we need to associate the KMIP Server with our vCenter Server. Login to the vSphere Web Client and under the vCenter Server object, select Configure->Key Management Servers and add a new KMS. You will need to provide a name/alias, the IP Address of where the Docker Container is running and the default port number as shown in the screenshot below.
Step 4 - Once connected to the KMIP Server, you should be presented with a Trust Certificate dialog which you just need accept once.
Step 5 - If everything was configured correctly and the vCenter Server can communicate with the KMIP Server, you should see that both the Connection Status and Certificate Status display green. If you are not getting this, it means there is most likely a connection issue between your vCenter Server and the Docker Container, check to make sure you do not have any firewalls blocking the connection from where the Docker Contain is running.
At this point, you can now start encrypting your VMs. To do so, you simply apply the VM Encryption Policy on either the full VM (VM Home + VMDKs) or to individual VMDKs and let the Policy Engine do its magic.
After the VM Storage Policy has been applied successfully, you can view the Encryption status by clicking on the VM Hardware portlet for the VM as shown in the screenshot below.
Once you are done with your testing, you can remove the VM Encryption storage policy from the VMs and delete the KMS from the vCenter Server. If for whatever reason your KMIP server terminates, you can simply just remove the KMS from vCenter Server and relaunch a new instance by going through the setup instructions again. For more information about VM Encryption, please take a look at the official documentation which can be found here. Happy VM Encrypting 🙂
Hi William,
Thanks alot for publishing your docker container, it works like a charm. i just want to ask something, why my datastore is listed as incompatible storage in default VM encryption policy? i have 3 NFS datastore and 1 local VMFS datastore, i believe VM encryption is transparent to datastore type right?
Yes VM Encryption is transparent to datastore type. Was the default VM encryption policy modified by any chance? I suspect if the policy has some datastore specific elements added, then some datastore(s) could show up as incompatible.
Hello William, Thanks a lot for a shortcut to VM encryption testing. Have you had a chance to try also VSAN encryption with KMIP server you have prepared ?
I am getting an error:
///
General vSAN error.
The KMS cluster KMS1 does not have a client certificate or key configured
///
Could it be VSAN encryption layer is more strict during KMS cert/key validation ?
Best regards,
Roman.
Yes, it works 🙂 See http://www.virtuallyghetto.com/2017/04/easily-try-out-vsan-6-6-encryption-feature-using-kmip-docker-container.html
hey guys. thanks for this blog and William's. I get a "cannot establish trust connection" when I add William's vmwkmip docker image to the Key Management Server. Everything is self-signed and disconnected from the internet. The image is running on port 5696 and I'm able to ping it. any ideas?
Hi William.. thank you for the document.. Do you know in which directory the files for kms cluster that is created will be stored..
Created a KMS cluster and then deleted it, now created a new KMS cluster, however when setting the encryption on the host it still points towards the old kms cluster
RuntimeFault.summary
"Key 2/KMSCluster not found";
How do i delete the old stale entries?
I developed vapp for pykmip that can be deployed in vcenter environment.
https://github.com/gwojcieszczuk/kms4vsphere
Latest PyKMIP supports persistent storage (sqlite db) so all keys are preserved across reboots, etc.
Revisiting this in light of recent windows server announcements regarding TPM 2.0 needs might be good as a refresher for homelab people.
https://cloudblogs.microsoft.com/windowsserver/2020/06/11/microsoft-raises-the-security-standard-for-next-major-windows-server-release/
But, how would one go about dealing with this on standalone ESXi servers, as vTPM device addition seems to require both vCenter and a KSM, as the host client UI has no vTPM device option?
There is an additional step needed to fix the certificate issue error. Click on Make KMS Trust vCenter > KMS certificate and private key. Exec into the container and cd into "/root/cayman_pykmip/pykmip/src/kmip/demos/certs" and cat out the server.crt and server.key contents and copy them into their respective fields in vCenter.
Also if anyone needed a docker compose version of this:
version: '3.3'
services:
vmwkmip:
image: lamw/vmwkmip
container_name: vmwkmip
restart: unless-stopped
ports:
- '5696:5696'