Applying additional security hardening enhancements in ESXi 8.0

While responding to a few ESXi security configuration questions, I was referencing our ESXi Security documentation, which includes a lot of useful information and latest best practices. It is definitely worth re-reviewing this section from time to time to take advantage of all the ESXi security enhancements to help protect and secure your vSphere environment.

In certain areas of the ESXi security documentation, I noticed that it mentions CLI and API, but it does not always provide an example that customers can then reference and use in their Automation, which is really the only guaranteed method to ensure configurations are consistent across your vSphere environment. After answering some of the security related questions, especially on the Automation examples, I figure it would be useful to share this information more broadly so that folks are aware of some of the new and existing security enhancements along with some of their implications if you are not implementing them.

Speaking of new ESXi security enhancements, one of the new features that was introduced in ESXi 8.0 is the ability to disable ESXi Shell access for non-root users. While this might sound like a pretty basic feature, applying this towards the vCenter Server service account vpxuser can help add another layer of protection for your ESXi hosts against attackers. It turns out that users with ESXi Shell access can also modify other local users password on ESXi host including the root user. By restricting ESXi Shell access for the vpxuser, you prevent attackers, which can also be insiders who have access to vCenter Server the ability to just change the ESXi root password without knowing the original password. As a result, this can lock you out of your ESXi hosts or worse, enable an attacker to encrypt your workloads, especially as the rise ransomeware attacks has been increasing.

Another ESXi security configuration that is definitely worth applying and mentioned in the ESXi security documentation is to create a new ESXi local administrative user and then remove the administrative privileges for the default root account user, which most attackers will assume this account exists on your ESXi host. Not only will this make it more difficult for attackers but the result of removing administrative privileges from the root user will also disable SSH access for this user automatically and also prevent login access to the ESXi Host Client or ESXi API, so you actually get three benefits by simply applying this configuration in your environment.

To check whether local users on an ESXi host have ESXi Shell access, the existing ESXCLI system account command has been updated to include a new column called Shell access which can be viewed by running the following command:

esxcli system account list

To disable ESXi Shell access for non-root accounts like the vpxuser and dcui account, you can use the following ESXCLI command and the new -s or --shell-access argument by running the following:

esxcli system account set -i dcui -s false
esxcli system account set -i vpxuser -s false

Note: While you may want to disable vpxuser ESXi Shell Access prior to adding it to vCenter Server, you can still disable this after an ESXi host has been added to vCenter Server.

To create a new local administrative user, in my example I have named the account admin32 but you should select a name that is difficult to guess and use the following ESXCLI command:

esxcli system account add -i admin321 -p -c -s true

Note: Do not provide the credentials when creating a new account on the command-line as the password will be captured in the ESXi Shell logs (/var/log/shell.log). If you omit the password, you will be prompted and that will not be captured in the ESXi Shell logs. For Automation purposes, you may want to embed the password within a script, so that when you run the script, the only the running of the script is captured in the ESXi Shell log rather than the actual password.

To grant the new local administrative user with administrator role as the default root account, use the following ESXCLI command:

esxcli system permission set -i admin321 -r Admin

Note: Please make sure you do NOT remove administrative privileges from the default root user UNTIL you have successfully created a new local administrative user with the same privileges as the default root user. Once the new administrative user has been created, you can now use this to connect to vCenter Server and everything will continue to work as before but now the root user is no longer the account that is used to add the ESXi hosts. For further protection, you can also enable ESXi Lockdown Mode and for the strict configuration, you should also update the ESXi Exception list to include the new local administrative account so it can login for debugging or troubleshooting purposes. For more details, please refer to the ESXi Lockdown Mode documentation.

Once you have created a new local administrative user with the administrator role, we can now de-privilege the root user account with the NoAccess role using the following ESXCLI command:

esxcli system permission set -i root -r NoAccess

While these settings can be automated using a number of different options including vSphere Host Profiles, I will cover two common methods for automating ESXi configuration changes: ESXi Kickstart or through the use of the vSphere API via PowerCLI or any other vSphere SDK.

Automating using ESXi Kickstart

The following snippet should be saved into a shell script such as using heredoc, which is one to save script within the ESXi Kickstart, so that you can source control the credentials without exposing the actual password when the script runs. Instead of capturing the output of the ESXCLI command, it would simply show running /tmp/disable-admin-privledge.sh as an example

#!/bin/bash
PASSWORD='VMware1!'

# Disable Shell Access for dcui and vpxuser
esxcli system account set -i dcui -s false
esxcli system account set -i vpxuser -s false

# Create new local account
esxcli system account add -i admin321 -p ${PASSWORD} -c ${PASSWORD} -s true

# Give new local account administrative privileges
esxcli system permission set -i admin321 -r Admin

# Remove administrative privileges from root user
esxcli system permission set -i root -r NoAccess

Automating using vSphere API

If running local or remote ESXCLI command is not your cup of tea or you rather do this programmatically using the vSphere API, then here is an example doing the exact same thing using PowerCLI, which is just one of the many ways in which you can access the vSphere API.

The example below assumes you are using PowerCLI to connect directly to the ESXi host and before adding it to your vCenter Server, since we are not going to use the root account to add ESXi host to vCenter Server, you will need to create the new administrative local account prior.

The following PowerCLI snippet will disable ESXi Shell access for both dcui and vpxuser:

$disableShellAccessUsers = @("dcui", "vpxuser")

$accountManager = Get-View $global:DefaultVIServer.ExtensionData.Content.AccountManager

# Disable Shell Access for dcui and vpxuser
foreach ($disableShellAccessUser in $disableShellAccessUsers) {
    $spec = New-Object VMware.Vim.HostPosixAccountSpec
    $spec.id = $disableShellAccessUser
    $spec.ShellAccess = $false
    $accountManager.UpdateUser($spec)
}

The following PowerCLI snippet will create a new local administrative account:

$newUserName = "admin321"

# Create new local account
$newUserSpec = New-Object VMware.Vim.HostPosixAccountSpec
$newUserSpec.id = $newUserName
$newUserSpec.Description = "New Local Admin Account"
$newUserSpec.ShellAccess = $true
$newUserSpec.Password = "VMware1!"
$accountManager.CreateUser($newUserSpec)

The following PowerCLI snippet will add Administrator role to the new new local administrative account and de-privilege the root user with the NoAccess role:

$authManager = Get-View $global:DefaultVIServer.ExtensionData.Content.AuthorizationManager
$rootFolderMoRef = (Get-Folder -Type Datacenter -Name root).ExtensionData.MoRef

# Give new local account administrative privileges
$newAdminAuthSpec = New-Object VMware.Vim.Permission
$newAdminAuthSpec.RoleId = -1 # Administrator
$newAdminAuthSpec.Propagate = $true
$newAdminAuthSpec.Principal = $newUserName
$authManager.SetEntityPermissions($rootFolderMoRef,@($newAdminAuthSpec))

# Remove administrative privileges from root user
$newRootAuthSpec = New-Object VMware.Vim.Permission
$newRootAuthSpec.RoleId = -5 # NoAccess
$newRootAuthSpec.Propagate = $true
$newRootAuthSpec.Principal = "root"
$authManager.SetEntityPermissions($rootFolderMoRef,@($newRootAuthSpec))

Note: If you wish to view the role mapping to the role id, run the following command: $authManager.RoleList to view the full default list.

I know security is top of mind for all organizations and by applying some or all of these security configurations can make it much more difficult for attackers to gain access to your ESXi hosts and the underlining workloads.