On Thursday July 15th, a user raised a question on the VMTN forums regarding an ESXi 4.1 password issue. The problem was described as the following:
It seems that authentication only requires the first 8 characters to be correct. My root password is 11 characters long, but so long as the first 8 characters are correct, I can put whatever I like after that and it still authenticates me. Tested this on three ESXi boxes, all running 260247 (release)
I performed a few quick tests to validate the user's claim and in fact this was the case with a new installation of ESXi 4.1. Though in my lab, I had upgraded from ESXi 4.0 Update 2 to ESXi 4.1 and I was not able re-produce the issue. I then tried to reset my password to the same initial password and to my surprise, I hit the exact problem as described by the user. I notified VMware of the issue and was told that they were looking into the problem. As of today, there is still not official word from VMware on whether this is a bug or an expected behavior.
VMware does have a KB article (KB1012033) documenting both ESX and ESXi password requirements, by default a minimum of 8 characters is required but can be changed by modifying the PAM module pam_passwdqc.so which checks for the quality of a password.
Further looking into the issue over the weekend, I have found the following:
- pre ESXi 4.1 (e.g. ESXi 3.5, 4.0, 4.0u1, 4.0u2) does not have this password issue
- password issue affects ALL methods of authentication to an ESXi host: vSphere Client, local and remote Tech Support Mode, /bin/login, SSH localhost and vSphere MOB. (did not get a chance to validate AD logins)
- password issue will NOT affect hosts that were upgraded to ESXi 4.1, unless you change or update your password afterwards, in which case only the first 8 characters will be checked
During my testing, I provisioned a newly built ESXi 4.0 Update 2 (called esxi4-4) and ESXi 4.1 (called esxi4-3) and used a modified SSH login expect script to run my tests. You can download my modified version of the script here. Both hosts had their password created using the DCUI and was set to "VMware123"
Here is an example of SSHing to both host and running "vmware -l" command to show that hosts accepts the password that was created:
As you can see, I am able to login to both hosts using the defined password of "VMware123" and displaying the output of "vmware -l" command.
In the next example, I will purposely provide the wrong password of "VMware12" which should fail:
This should have failed for both hosts, but did not in the case of ESXi 4.1 and still allowed me to login. If you tried to login to the ESXi 4.1 host using "VMware1", access is denied. Basically, if you have a password that is greater than 8 characters, only the first 8 characters are validated and any combination there after is accepted! This in my mind is a big security hole, especially if this is an unexpected behavior that users are not made aware of.
I also performed one additional test by modifying the default password requirement from 8 characters to 9 by editing /etc/pam.d/system-auth:
I then used the passwd utility to create a new root password, I attempted to use the password "VMware12" which should fail as it does not meet the minimum password requirement, which it does:
Once I used password that was 9 characters or greater, the utility allowed me to set the new password:
From what I can understand, the password in fact is being set properly and the rules that govern the length are being regulated. The problem seems to be with the authentication module where it is validating the password and only checking for the first 8 characters. To me this sounds like a authentication bug and one that can severely impact any security policies put in-place for an organization.
I hope that VMware provides either a clarification on the issue and potentially a fix or patch for this problem. In the meantime, if you have security policies in your environment pertaining to password requirements and plan on upgrading to ESXi 4.1, ensure you do not change the password. If you do, change it before the upgrade.
It looks Didier Pironet found the solution and documented the steps in his follow-up blog post. I am still hoping VMware will address this security issue as soon as possible and provide a patch. The above work around does resolve the problem but the change is not persisted through a reboot without manual hacks to force a backup of PAM configuration file.
VMware just released KB article regarding the issue: KB1024500
VMware has just updated the KB article KB1024500 with slightly more details pertaining to the solution of the problem. They still have not confirmed the bug that is identified in the PAM module where DES encryption is used over MD5, but can be assumed via the solution provided. The problem actually exists in both vSphere ESX and ESXi 4.1 and the KB includes a fix to both including a way to persist the changes on ESXi. The security issue apparently is not important enough to get an immediate patch but may eventually get a permanent patch sometime in the future.
UPDATE4: VMware just first the first patch set (p01) for both ESX and ESXi 4.1 that resolves this security issue along with other bugs that have been identified since the release of vSphere 4.1. Here is the specific KB article regarding the fix - http://kb.vmware.com/kb/1027024 and here is the link to the details p01 patch set - http://kb.vmware.com/kb/1027027 As with anything, please test and validate in your test environment prior to rolling out to production.
Mike Horwath says
I was looking at a change I made on a newly upgraded 4.1 system...
And it looks like things have reverted to DES for the /etc/shadow files from a previous setting of MD5 hashes.
Good find, I did not notice this earlier while looking in the shadow file. This would make sense as DES crypt is limited to 8 character password + salt. I suspect the password is just being truncated or being encrypted by DES vs MD5 as you mentioned.
What if you use the AD integration, is that affected?
The problem does not append with AD auth
Scott Vessey says
I can vouch for this issue too - it was my colleague who posted that to the VMware community 😉
He got me to test it, and the problem exists for sure.
Hi William, I did also some researches on this issue and came with back with a workaround... Read more at http://deinoscloud.wordpress.com/2010/07/18/esxi-4-1-major-security-issue-the-sequel-and-the-workaround/
I just got back from the weekend and noticed the post! Great job, I did not have a lot of time to spend trying to find the solution, but it looks like it was a pretty straight forward one.
I also updated my blog post with the link to yours for the solution. One method of ensuring the file is preserved is just edit the auto-backup script to also backup this specific file. I would still recommend waiting for VMware to provide a true fix via a patch/update.
Totie Bash says
William, I read this article a year ago and applied the patch and thought that was the end of it. Fast forward to ESXi5 and it reapper on my network, here's what I mean http://toti3.wordpress.com/2011/09/30/esxi-password-vulnerability-reborn/