WilliamLam.com

How to Format and Create VMFS5 Volume using the CLI in ESXi 5

by // 39 Comments

VMware always recommends formatting and creating a new VMFS volume using the vSphere Client as it automatically aligns your VMFS volume. However, if you do not have access to the vSphere Client or you wanted to format additional VMFS volumes via a kickstart, you can do so using the CLI and the partedUtil under /sbin.

~ # /sbin/partedUtil
Not enough arguments
Usage:
Get Partitions : get
Set Partitions : set ["partNum startSector endSector type attr"]*
Delete Partition : delete Resize Partition : resize
Get Partitions : getptbl
Set Partitions : setptbl

With ESXi 5, an MBR (Master Boot Record) partition table is no longer used and has been replaced with a GPT (GUID Partition Table) partition table. There is also only one block size of 1MB versus the 2,4 and 8 that was available in ESX(i) 4.x

We can view the partitions of a device by using the "getptbl" option and ensure we don't have an existing VMFS volume:

~ # /sbin/partedUtil "getptbl" "/vmfs/devices/disks/mpx.vmhba1:C0:T2:L0"
gpt
652 255 63 10485760

Next we will need to create a partition by using the "setptbl" option:

/sbin/partedUtil "setptbl" "/vmfs/devices/disks/mpx.vmhba1:C0:T2:L0" "gpt" "1 2048 10474379 AA31E02A400F11DB9590000C2911D1B8 0"

The "setptbl" accepts 3 arguments:

  • diskName
  • label
  • partitionNumber startSector endSector type/GUID attribute

The diskName in this example is the full path to the device which is /vmfs/devices/disks/mpx.vmhba1:C0:T2:L0

The label will be gpt

The last argument is actually a string comprised of 5 individual parameters:

  • partitionNumber - Pretty straight forward
  • startSector - This will always be 2048 for 1MB alignment for VMFS5
  • endSector - This will need to be calculated based on size of your device
  • type/GUID - This is the GUID key for a particular partition type, for VMFS it will always be AA31E02A400F11DB9590000C2911D1B8

To view all GUID types, you can use the "showGuids" option:

~ # /sbin/partedUtil showGuids
Partition Type       GUID
vmfs                 AA31E02A400F11DB9590000C2911D1B8
vmkDiagnostic        9D27538040AD11DBBF97000C2911D1B8
VMware Reserved      9198EFFC31C011DB8F78000C2911D1B8
Basic Data           EBD0A0A2B9E5443387C068B6B72699C7
Linux Swap           0657FD6DA4AB43C484E50933C84B4F4F
Linux Lvm            E6D6D379F50744C2A23C238F2A3DF928
Linux Raid           A19D880F05FC4D3BA006743F0F84911E
Efi System           C12A7328F81F11D2BA4B00A0C93EC93B
Microsoft Reserved   E3C9E3160B5C4DB8817DF92DF00215AE
Unused Entry         00000000000000000000000000000000

Once you have the 3 arguments specified, we can now create the partition:

~ # /sbin/partedUtil "setptbl" "/vmfs/devices/disks/mpx.vmhba1:C0:T2:L0" "gpt" "1 2048 10474379 AA31E02A400F11DB9590000C2911D1B8 0"
gpt
0 0 0 0
1 2048 10474379 AA31E02A400F11DB9590000C2911D1B8 0

UPDATE (01/15) - Here is a quick shell snippet that you can use to automatically calculate End Sector as well as creating the VMFS5 volume:

partedUtil mklabel ${DEVICE} msdos
END_SECTOR=$(eval expr $(partedUtil getptbl ${DEVICE} | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1)
/sbin/partedUtil "setptbl" "${DEVICE}" "gpt" "1 2048 ${END_SECTOR} AA31E02A400F11DB9590000C2911D1B8 0"
/sbin/vmkfstools -C vmfs5 -b 1m -S $(hostname -s)-local-datastore ${DEVICE}:1

Note: You can also use the above to create a VMFS-based datastore running on a USB device, however that is not officially supported by VMware and performance with USB-based device will vary depending on the hardware and the speed of the USB connection. 

We can verify by running the "getptbl" option on the device that we formatted:

~ # /sbin/partedUtil "getptbl" "/vmfs/devices/disks/mpx.vmhba1:C0:T2:L0"
gpt
652 255 63 10485760
1 2048 10474379 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

Finally we will now create the VMFS volume using our favorite vmkfstools, the syntax is the same as previous release of ESX(i):

~ # /sbin/vmkfstools -C vmfs5 -b 1m -S himalaya-SSD-storage-3 /vmfs/devices/disks/mpx.vmhba1:C0:T2:L0:1
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs5 file system on "mpx.vmhba1:C0:T2:L0:1" with blockSize 1048576 and volume label "himalaya-SSD-storage-3".
Successfully created new volume: 4dfdb7b0-8c0dcdb5-e574-0050568f0111

Now you can refresh the vSphere Client or run vim-cmd hostsvc/datastore/refresh to view the new datastore that was created.

How to Create Custom Firewall Rules in ESXi 5.0

by // 29 Comments

In ESXi 5.0, the firewall system has been completely revamped to provide the same functionality as the classic ESX Service Console esxcfg-firewall command. To access the firewall configurations, you can use the following esxcli namespace: esxcli network firewall. By default, there are set of predefined services that a user can enable or disable upon startup.

To list the default firewall rules, you can run the following command:  

esxcli network firewall ruleset list

You can also create your own custom firewall rules/services, but unfortunately this has to go outside of the esxcli framework with a custom firewall rule configuration file. An example firewall rule can be viewed under /etc/vmware/firewall/ if you have FDM enabled and you should find a file called fdm.xml which looks a little something like this:
This XML configuration file describes the name of the firewall rule/service and also specifies the various ports, port type, protocol and direction of a given service.

In the following example, I will create a new firewall rule called "virtuallyGhetto" and it will have port 1337 using TCP for both inbound/outbound and port 20120 using UDP for both inbound/oubound. You will need to create a new XML file and specify a name which I have called /etc/vmware/firewall/virtuallyGhetto.xml

<ConfigRoot>
  <service>
    <id>virtuallyGhetto</id>
    <rule id='0000'>
      <direction>inbound</direction>
      <protocol>tcp</protocol>
      <porttype>dst</porttype>
      <port>1337</port>
    </rule>
    <rule id='0001'>
      <direction>outbound</direction>
      <protocol>tcp</protocol>
      <porttype>dst</porttype>
      <port>1337</port>
    </rule>
    <rule id='0002'>
      <direction>inbound</direction>
      <protocol>udp</protocol>
      <porttype>dst</porttype>
      <port>20201</port>
    </rule>
    <rule id='0003'>
      <direction>outbound</direction>
      <protocol>udp</protocol>
      <porttype>dst</porttype>
      <port>20201</port>
    </rule>
    <enabled>true</enabled>
    <required>false</required>
  </service>
</ConfigRoot>

Next we will need to reload the firewall by performing a "refresh" operation and then list the rules again using the following command:  

esxcli network firewall refresh
esxcli network firewall ruleset list

We can also verify that the individual rulesets for our new firewall rule/service by running the following command and grepping for the rule in question:

esxcli network firewall ruleset rule list | grep virtuallyGhetto

The new ESXi firewall also allows you to specify specific IP Address or IP ranges to access a particular service. In the following example I disable the "allow all" and specify a particular range for the virtuallyGhetto service using the following commands:

esxcli network firewall ruleset set --allowed-all false --ruleset-id=virtuallyGhetto
esxcli network firewall ruleset allowedip add --ip-address=172.30.0.0/24 --ruleset-id=virtuallyGhetto

The new firewall rules/services are also viewable under the host configuration section "Security Profile" using the vSphere Client:

As you can see it is pretty straight forward to add your own firewall rules and this can easily be incorporated into your kickstart builds.

UPDATE1: How to persist custom firewall rules in ESXi 5, take a look at these two articles here and here.

UPDATE2: Duncan Epping just posted an article on creating your own vibs which will persist firewall rules, definitely take a look as another option.

UPDATE3: You can now easily create persistent firewall rules and other files using the new VIB Author Fling, please take a look at this article here for some examples.

SSH Keys & Lockdown Mode Caveat in ESXi 5

by // 6 Comments

Hopefully everyone is familiar with ESXi's Lockdown Mode and what it means from a security standpoint. Here is a table of the behavior between normal and Lockdown Mode:

In ESXi 5, the use of SSH keys is officially supported without having to manually create any hacks to preserve .ssh directory as you did with prior releases of ESXi. If you use Lockdown Mode, there is an additional caveat to be aware of in which the use of SSH keys is able to by-pass the Lockdown Mode configuration for an ESXi 5 host.

Here is a quick example demonstrating the process from VMware's VCVA (vCenter Virtual Appliance):

Step 1 - Create SSH keys

vcenter50-1:~ # ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
a1:8c:24:60:0e:fe:9a:cf:4a:35:17:d6:89:ba:08:9d root@vcenter50-1
The key's randomart image is:
+--[ DSA 1024]----+
|o. |
|=. o . |
| o. .+ o. |
| ..+oo.. . |
|. E=..o S |
|. = + |
| = . |
|. o |
| ..o |
+-----------------+

Step 2 - Copy SSH public keys over to destination ESXi 5 host into the authorized file under /etc/ssh/keys-root/authorized_keys

vcenter50-1:~ # scp .ssh/id_dsa.pub root@vesxi50-4:/etc/ssh/keys-root/authorized_keys
Password:
id_dsa.pub 100% 606 0.6KB/s 00:00

Step 3 - Enable Lockdown Mode via vCenter

Step 4 - SSH into locked down ESXi 5 host utilizing SSH keys

By default the support of SSH is enabled, you will need to manually disable it to ensure that you are fully lockdown when you choose to enable Lockdown Mode. To disable SSH key support, you just need to comment the following line in /etc/ssh/sshd_config

# AuthorizedKeysFile /etc/ssh/keys-%u/authorized_keys

The change takes effect right away and you do not need to restart the SSH daemon.

If you are interested in automatically disabling the use of SSH keys or you would like to copy an existing SSH key into your ESXi 5 host via kickstart, take a look at Automating ESXi 5.x Kickstart Tips & Tricks for more details.