WilliamLam.com

How to enable SSH security banner on ESXi

by // 3 Comments

There was a pretty interesting question on the VMTN forums this week about adding a security banner to ESXi prior to user authentication via SSH. With classic ESX, this can easily be enabled by updating the "Banner" entry in /etc/sshd/sshd_config and specifying the path to the file containing your banner. With ESXi, OpenSSH is not being used for the SSH daemon/client, but rather a server/client with a much smaller footprint called Dropbear.

Dropbear not only provides a server/client but it also includes key generator/converter and secure copy (scp) all bundled into a single binary under /sbin/dropbearmulti. To access the different functions, you would just need to create a symlink to (dropbear,dbclient,dropbearkey,dropbearconvert and scp) which is already enabled by default from VMware with the exception of dropbear which is for the server. You can see the dropbear process spawn when you are logged into the Busybox Console (Tech Support Mode) using "ps" command.

Here is what that looks like:

What you may not notice is that there are some additional arguments being passed to the dropbear utility. Dropbear is launched using inetd and it's configurations are defined in the /etc/inetd.conf file.

You may wonder if there are some additional options that dropbear supports? The answer is yes and we can check for these options by manually creating a symlink from /bin/dropbear to /sbin/dropbearmulti and running dropbear -h option.

Note: The symlink creation of "dropbear" is not needed to enable security banner

As you can see there are a ton of additional options including support for a banner file using the -b option.

To enable this, we will first create a banner and for this example, I will be storing it under /etc/banner

Next, we will need to update the dropbear arguments to include the banner file, you will need to update /etc/inetd.conf and specify the path to your banner file.

Next you will need to restart inetd process and use kill -HUP command.

Now if you try to login to your ESXi host via SSH, you should now see the new banner be presented prior to authenticating to the host.

Now before you jump off and start thinking about a creative banner, you should note that changes within the ESXi Busybox Console are not always preserved and persisted across reboots. This is not a well known fact and the reason for this is ESXi is loaded into memory after it boots up.

There are certain configuration files (e.g. /etc/inetd.conf) that are automatically backed up through a cronjob which looks for particular files under /etc that have been marked with the stickybit. A user can not manually mark a file with the stickybit and have it automatically backed up, it requires one additional file which is implemented by the VisorFS. ESXi creates a copy of these stickybit files and renames the original as .#filename. The backup process will then look for any .#* files and back those up. Due to this special permission mechanism, you can not manually create/touch files with this format as explained by a VMware employee on this VMTN thread.

An alternative to this, which is one that I have used in the past is to update the /etc/rc.local file which is automatically backed up. Entries in this file will be executed after the host has booted up and it is the perfect place to re-create our banner file as it will not be persisted across reboots. You will create a simple here document in the script which contains the contents of your banner file, in this example, I am storing it in /etc/banner

Once you have updated and saved /etc/rc.local file, you will need to manually run a backup to ensure we have a good backup in case the host reboots. You can do so by running /sbin/auto-backup.sh which will create the latest backup and store it under /bootbank. You now have successfully enabled a security banner on ESXi and it persist through reboots.

Note: You could also have stored your banner under local VMFS datstore and/or other datastore, but it is probably best that you keep it under normal filesystem paths.

Potential ESX(i) 4.1 Update 1 upgrade caveat

by // Leave a Comment

If you are planing on upgrading to the recent release of ESX(i) 4.1 Update 1 from ESX(i) 4.1, you may want to verify that you will not be impacted by a previous security/password bug found in ESX(i)4.1. The security bug that was identified with ESX(i) 4.1 was the encryption algorithm which changed from default MD5 as previous releases of ESX(i) to legacy DES.

This bug has since been resolved and a VMware KB article (KB1024500) was released with a temporary fix by adding the keyword "md5" to system-auth PAM configuration entry. This fix also required the user to update the root password afterwards, as the previous password was encrypted using DES.

The potential caveat is if you did not apply the fix as mentioned from the above VMware KB article prior to upgrade, after the ESX(i) 4.1 Update 1 upgrade, you will continue to run into the same problem. The fix is to reset your root password after the upgrade of ESX(i) 4.1 Update 1, this will ensure that the new password will be encrypted using the MD5 algorithm. Though the fix is simple, it can be tedious and manual for users who do not regularly rotate their root passwords or have an automated password management system.

To check whether this will impact your upgrade, login to ESXi Tech Support Mode or classic ESX Service Console and check whether the root password was encrypted using MD5 or DES. To do so, you will cat out the contents of /etc/shadow

If the root password was encrypted using MD5 algorithm, you should see root hash start with "$1$"

If the root password was encrypted using DES algorithm, the root hash will not start with "$1$"

If it is the latter, you will need to either apply the fix and update the root password before upgrading or reset the password after your upgrade. To change the root password, you need to login to either the Serivce Console for ESX or Tech Support Mode for ESXi and run the passwd utility to change your password.

It is probably quicker to rebuild than to login to each host an update the root password, especially if you have an automated kickstart environment. This will ensure that that all hosts will be consistent and no manual fixes will be required. IMHO, this is something that VMware should have clearly pointed out in their release notes as not everyone may be aware of the VMware KB article and implemented the fix prior to upgrade.

Automating Active Directory User Management in ESXi Kickstart

by // 2 Comments

In the previous post we looked at Automating Active Directory Domain Join in ESXi Kickstart. We are now going to look at adding domain users without having to manually go through vSphere Client or external scripts after an ESXi host has been provisioned. We are going to be leveraging vim-cmd to accomplish this during the kickstart installation. Before doing so, you will need to know the the available roles on a default ESXi host and the syntax for a given permission.

To see the available roles, you can run the following command on an already provisioned ESXi host:

vim-cmd vimsvc/auth/roles | less

The default roles on an ESXi host are:

  • NoAccess
  • ReadOnly
  • Admin

To see the existing permissions, you can run the following command on an already provisioned ESXi host:

vim-cmd vimsvc/auth/permissions

These entries will match what you see in the vSphere Client and dictate who has login access to the ESXi host.

To add a new permission, we will be using the "vim-cmd vimsvc/auth/entity_permission_add" and it requires five parameters.

  • First - Entity (This can be found by looking at the output from permissions)
  • Second - Username
  • Third - Boolean on whether this is a group or not (should be false)
  • Fourth - The role to be applied to the user
  • Fifth - Boolean on whether to propagate this permission

If you manually add a domain user, you can easily verify the user can login by running the "id" command which will perform a look up on the user. If it is successful, it should return an entry corresponding to something like this:

Note: We need to use the double slash "\" to escape the initial slash when running the query. Also make note of the domain name as it may or may not match your full domain name.

We are now ready to craft a simple script that will add domain users as part of the ESXi kickstart process. The following snippet should be placed in the %firstboot section of your kickstart and after your Active Directory domain join code. Make sure you replace the DOMAIN_NAME variable along with the usernames. In the example I have two separate for loops to handle ReadOnly and Admin users, you do not need both if you are only adding one type of users.

The script basically performs a simple 60sec sleep to ensure the domain join process has completed before continuing. If you do not place a sleep, the subsequent code will fail to execute. The next step is to validate the user by doing a simple lookup using "id" command and upon successful look up of the user, we add the appropriate permissions.

Note: We only have two add these two entities: "vim.Folder:ha-folder-root" and "vim.ComputeResource:ha-compute-res" to properly add a permission.

If everything was successful, after your ESXi installation you now should have your host joined to your Active Directory and a list of domain users who now have permission to login to the ESXi host. You can verify by using the vSphere Client and taking a look at the Permissions tab.

If you would like to create custom roles on your ESXi host, you can use the following command:

vim-cmd vimsvc/auth/role_add

Note: The syntax for the privileges parameter lists only five, but it actually accepts as many as you need with the custom role