ESXi 4.1 - Major Security Issue

On Thursday July 15th, a user raised a question on the VMTN forums regarding an ESXi 4.1 password issue. The problem was described as the following:

Hi all
It seems that authentication only requires the first 8 characters to be correct. My root password is 11 characters long, but so long as the first 8 characters are correct, I can put whatever I like after that and it still authenticates me. Tested this on three ESXi boxes, all running 260247 (release)

I performed a few quick tests to validate the user's claim and in fact this was the case with a new installation of ESXi 4.1. Though in my lab, I had upgraded from ESXi 4.0 Update 2 to ESXi 4.1 and I was not able re-produce the issue. I then tried to reset my password to the same initial password and to my surprise, I hit the exact problem as described by the user. I notified VMware of the issue and was told that they were looking into the problem. As of today, there is still not official word from VMware on whether this is a bug or an expected behavior.

VMware does have a KB article (KB1012033) documenting both ESX and ESXi password requirements, by default a minimum of 8 characters is required but can be changed by modifying the PAM module pam_passwdqc.so which checks for the quality of a password.

Further looking into the issue over the weekend, I have found the following:

• pre ESXi 4.1 (e.g. ESXi 3.5, 4.0, 4.0u1, 4.0u2) does not have this password issue
• password issue affects ALL methods of authentication to an ESXi host: vSphere Client, local and remote Tech Support Mode, /bin/login, SSH localhost and vSphere MOB. (did not get a chance to validate AD logins)
• password issue will NOT affect hosts that were upgraded to ESXi 4.1, unless you change or update your password afterwards, in which case only the first 8 characters will be checked

During my testing, I provisioned a newly built ESXi 4.0 Update 2 (called esxi4-4) and ESXi 4.1 (called esxi4-3) and used a modified SSH login expect script to run my tests. You can download my modified version of the script here. Both hosts had their password created using the DCUI and was set to "VMware123"

Here is an example of SSHing to both host and running "vmware -l" command to show that hosts accepts the password that was created:

As you can see, I am able to login to both hosts using the defined password of "VMware123" and displaying the output of "vmware -l" command.

In the next example, I will purposely provide the wrong password of "VMware12" which should fail:

This should have failed for both hosts, but did not in the case of ESXi 4.1 and still allowed me to login. If you tried to login to the ESXi 4.1 host using "VMware1", access is denied. Basically, if you have a password that is greater than 8 characters, only the first 8 characters are validated and any combination there after is accepted! This in my mind is a big security hole, especially if this is an unexpected behavior that users are not made aware of.

I also performed one additional test by modifying the default password requirement from 8 characters to 9 by editing /etc/pam.d/system-auth:

I then used the passwd utility to create a new root password, I attempted to use the password "VMware12" which should fail as it does not meet the minimum password requirement, which it does:

Once I used password that was 9 characters or greater, the utility allowed me to set the new password:

From what I can understand, the password in fact is being set properly and the rules that govern the length are being regulated. The problem seems to be with the authentication module where it is validating the password and only checking for the first 8 characters. To me this sounds like a authentication bug and one that can severely impact any security policies put in-place for an organization.

I hope that VMware provides either a clarification on the issue and potentially a fix or patch for this problem. In the meantime, if you have security policies in your environment pertaining to password requirements and plan on upgrading to ESXi 4.1, ensure you do not change the password. If you do, change it before the upgrade.

It looks Didier Pironet found the solution and documented the steps in his follow-up blog post. I am still hoping VMware will address this security issue as soon as possible and provide a patch. The above work around does resolve the problem but the change is not persisted through a reboot without manual hacks to force a backup of PAM configuration file.

VMware just released KB article regarding the issue: KB1024500

VMware has just updated the KB article KB1024500 with slightly more details pertaining to the solution of the problem. They still have not confirmed the bug that is identified in the PAM module where DES encryption is used over MD5, but can be assumed via the solution provided. The problem actually exists in both vSphere ESX and ESXi 4.1 and the KB includes a fix to both including a way to persist the changes on ESXi. The security issue apparently is not important enough to get an immediate patch but may eventually get a permanent patch sometime in the future.

UPDATE4: VMware just first the first patch set (p01) for both ESX and ESXi 4.1 that resolves this security issue along with other bugs that have been identified since the release of vSphere 4.1. Here is the specific KB article regarding the fix - http://kb.vmware.com/kb/1027024 and here is the link to the details p01 patch set - http://kb.vmware.com/kb/1027027 As with anything, please test and validate in your test environment prior to rolling out to production.