Thanks to a conversation I had this morning with a Twitter user @BOK, I learned that CoreOS has just recently published a new CoreOS Alpha Image (v490.0.0) that now includes the Open VMware Tools package. In addition, the new image also now uses the optimized VMXNET3 network adapter. From our chat, it looks like this update is not available in the insecure or VMware Fusion image and I can only guess that these images may eventually get deprecated or removed completely in favor for the newer VMware image which can both run on ESXi as well as hosted products like VMware Fusion/Workstation.
@BOK also shared with me a modified version of my script that I initially created to automate the deployment of CoreOS onto ESXi. The original workflow had to be slightly modified as the new image is only available as bz2 (bunzip2) and ESXi does not contain the bunzip2 utility. This means there are now two steps: first is to extract the VMDK and upload to ESXi datastore which is going to be done manually and you can run the script which will automatically convert the VMDK to the proper format and register the VM in ESXi.
I was thinking about how I could simplify this process, even though it is just a couple of extra commands, I always like to see how I can make something easier to consume and reducing the complexity if possible. I of course decided to create a new script called deploy_coreos_on_esxi2.sh which now runs outside of the ESXi Shell. The script requires a UNIX/Linux system that has the bunzip2 utility and will automatically download both the VMX and VMDK file, perform the extraction and then upload it to ESXi host using an HTTP API provided through the vSphere Datastore. Lastly, it auto-generates the configuration shell script that will run over SSH to the ESXi host (SSH is still required) which will perform the same set of operations as my previous script did.
Note: You will be prompted to enter the ESXi root password when it tries to run the script remotely on the ESXi Shell, there is a timeout for 120seconds in case you step away from the console.
Prior to running the script, you will need to edit the following 7 variables:
- ESXi_HOST
- ESXI_USERNAME
- ESXI_PASSWORD
- ESXI_DATASTORE
- VM_NETWORK
- VM_NAME
Here is a screenshot of running the script from my Mac OS X desktop:
Once the CoreOS VM has fully booted, we can take a look at our vSphere Client and we should see that VMware Tools is in fact running and we can see the IP Address automatically being displayed in the UI:
The really nice thing about having VMware Tools running in the CoreOS image, is that you can use VMware's Guest Operations API to be able to perform operations within the guest which are proxied through VMware Tools and can be quite handy, especially if networking is not available or you want to go through a single management interface such as using the vSphere API.
Thanks! What a fancy .profile, how you did that?
I'm using oh-my-zsh, has a bunch of neat terminal profiles
can't wait to try that thank you!
Hi William! So you skipped the part(s) about login into the VM's. The above setup will get a VM running, but that's all...
For a multi-VM cluster setup some more scripting-setup is needed. I found some ready-made on GitHub: https://github.com/grengojbo/vmware-coreos
No documentation at all, but checking the way script "w-coreos" is working got me started.
Your previous script is used also (had to comment the line with "ethernet0.virtualDev"), so it's still using the images without VMwaretools...
Maybe that code can get you on par with let's say a 4-node cluster, by only adding a little scripting-framework.
Cheers, Henk
Check out my updated script which I now build the cloud-config ISO, upload & attach to VM. However, I wasn't able to get it to auto-configured as I expected, I'm guessing I probably missed something ...
Seems to be working for a great amount, when set a DISCOVERY-URL!
But since there's no fixed MAC-address, the CORE_OS_IP_ADDRESS differs from what my DHCP-server is giving as a lease, so "etcd" and "fleetd" are confused.
Also CORE_OS_HOSTNAME is a typo in your __CLOUD_CONFIG__ (see hostname-entry).
I'm not sure how I missed this last week when I followed your version 1 guide, but your whole process after installing the 490 VMX and VMDK is basically what I figured out on my own, after hours of banging my head against the wall. I actually wrote a script to power down the VM, edit the cloud-config file, create a new ISO, push the ESX, and reboot the VM because I was fiddling with cloud-config so much.
Thanks for putting this script and these posts together! Very informative!
Tried script deploy_coreos_on_esxi2.sh and configured CoreOS on ESXi successfully. Unfortunately not able to shh into VM using credentials below
User: core
Password: $69sWzVthtg/w
and then with
User: vghetto
Password: $69sWzVthtg/w
with no luck.
Could you elaborate on login into VM...
Thanks
Yeap, it's a nice script but I'm struggling to log on too.
You need to create a password hash if you choose to go this route. Always consider the security implications of not using SSH keys.
# On Debian/Ubuntu (via the package "whois")
mkpasswd --method=SHA-512 --rounds=4096
# OpenSSL (note: this will only make md5crypt. While better than plantext it should not be considered fully secure)
openssl passwd -1
# Python (change password and salt values)
python -c "import crypt, getpass, pwd; print crypt.crypt('password', '\$6\$SALT\$')"
# Perl (change password and salt values)
perl -e 'print crypt("password","\$6\$SALT\$") . "\n"'
Hi, William
I've modified your script deploy_coreos_on_esxi2.sh and named new version deploy_coreos_on_esxi3.sh (i will send you a copy on your email *protected email* ... you can share it to everyone)
Changes
1) script can accept arguments as params (full description is script header)
2) script can access to esxi server without prompting for password (via sshpass)
3) script can deploy cluster of coreos VM's (via coreos discovery)
4) i've changed cloud-config file (removed all settings and added ssh rsa public key auth based on client rsa public key )
I'd love to have a copy of deploy_coreos_on_esxi3.sh
I was trying to see if I could use coreos with the free VMware Player and share a directory between Windows (my host OS) and CoreOS - via hgfs I assume. Then I can edit PHP code using PHPStorm (IDE) natively on my windows laptop, but run the PHP code in a Docker container. (I heard VMware is faster than VirtualBox here, but I have failed to get it to work as doing the mount command at https://pubs.vmware.com/workstation-9/index.jsp?topic=%2Fcom.vmware.ws.using.doc%2FGUID-AB5C80FE-9B8A-4899-8186-3DB8201B1758.html reports hgfs as an unknown file system.) I have a VM up in VMware with defined 'shared file system', but if I understand the documentation I have to 'mount' it in Linux to make it visible to CoreOS (and hence the Docker containers).
Does this wound related to your work?
Pretty cool. I think i'm missing the section where you install the actual vmware kernel mods and configure the systemd units, can you elaborate? I took a look at the script here: https://github.com/lamw/vghetto-scripts/blob/master/shell/deploy_coreos_on_esxi2.sh
Some how I'm still missing how that part works, is it already bundled in the coreos vmdk?