One of the most notable changes in the vCenter Server Appliance (VCSA) in vSphere 6.5 is a switch of the underlying OS from SLES to VMware's very own Photon OS. With this change, VMware will now own the entire software stack within the VCSA (OS + Application). This will allow VMware to quickly respond and deliver OS and security updates to customers at a much quicker rate than it was possible before.
During my testing of the VCSA, I had a need to spin up a Docker Container. Given that the VCSA is now Photon OS based, this should be a pretty trivial thing to enable as it is with a standalone installation of Photon OS. After a bit of trial/error, I found what was needed to get this working on the VCSA. Before jumping into the solution, I should say that this is really for lab and educational purposes. In general, I would NOT recommend installing additional software on the VCSA, not only is this NOT supported by VMware but you may also potentially be impacting your vCenter Server by taking resources away from the main application. It is possible to constrain the amount of resources (CPU/Memory) allocated to the Docker Container, please refer to this resource for more information.
For smaller customers, the argument is that I can just run everything on a single system but in reality there are many benefits to having a separate management VM which can be Photon OS or any other OS that your organization supports. You can install additional management tools/scripts and you would not be artificially limited by the VCSA's environment which is really locked down to what is absolutely needed to run the vCenter Server application and its services.
Disclaimer: This is not officially supported by VMware, please use at your own risk.
Given that PowerCLI Core (Linux and Mac OS X) was just recently released, which also includes a Docker Container, I figure this would be a nice example to start with as I know a few of you have asked about this possibility 🙂
Step 1 - Install Docker by running the following command (you will need access to the internet either direct or proxy access from the VCSA)
tdnf -y install docker
Step 2 - Load the following kernel module which will allow us to start the Docker client by running the following command:
insmod /usr/lib/modules/$(uname -r)/kernel/net/bridge/bridge.ko
Note: The above command does not persist across reboots. If you would like to persist this configuration, please refer to the instructions at the very bottom.
Step 3 - Enable and start the Docker Client by running the following command:
systemctl enable docker
systemctl start docker
Step 4 - Pull down the PowerCLI Core Docker Image from Docker Hub by running the following command:
docker pull vmware/powerclicore
Step 5 - Start the PowerCLI Core Docker Container by running the following command:
docker run --rm -it --entrypoint='/usr/bin/powershell' vmware/powerclicore
As you can see from the screenshot above, you now have PowerShell and the PowerCLI module loaded running as a Docker Container on the VCSA 🙂 You can apply this to any Docker Container that you have created or pulling it directly from Docker Hub. If you prefer to build the PowerCLI Core Docker Container from the Dockerfile, you simply just need to download and extract the PowerCLI Core zip file onto the VCSA and then run the following command:
docker build -t vmware/powercli .
How to persist bridge module load across reboots:
Step 1 - Edit /etc/modprobe.d/modprobe.conf and remove the "install bridge /bin/false" entry.
Step 2 - Create a new file called /etc/modules-load.d/bridge.conf which contains the word "bridge" (no quotes). When the system boots up, it will iterate through all the module configuration file and load the respective modules. The bridge module is what is needed to start the Docker Daemon.
Hey,
Great information here!
I thought docker was already installed in PhotonOS, but maybe the vcsa team completely removed the client to keep it as neat as possible.
Are you able to see the other docker instances running after that? 🙂
I wouldn't be surprised if PowerCLI core get integrated in one of the core appliances in the near future. vCSA or vRO?
Wait and see 🙂
One quick question: would the newly to be released v6.5 of vCSA support dual-homed configurations with two external PSCs (in replication) ?
I am getting error "unknown symbol in module" on step 2. Anyone else hit this? It as reboot of the 6.5 VCSA required after docker install? I am new to docker....
root@vlab65 [ ~ ]# insmod /usr/lib/modules/$(uname -r)/kernel/net/bridge/bridge.ko
insmod: ERROR: could not insert module /usr/lib/modules/4.4.8/kernel/net/bridge/bridge.ko: Unknown symbol in module
root@vlab65 [ ~ ]# cd /usr/lib/modules
root@vlab65 [ /usr/lib/modules ]# ls
4.4.8
root@vlab65 [ /usr/lib/modules ]# cd 4.4.8/
root@vlab65 [ /usr/lib/modules/4.4.8 ]# ls
kernel modules.alias.bin modules.builtin.bin modules.dep.bin modules.order modules.symbols
modules.alias modules.builtin modules.dep modules.devname modules.softdep modules.symbols.bin
root@vlab65 [ /usr/lib/modules/4.4.8 ]# insmod /usr/lib/modules/4.4.8/$(uname -r)/kernel/net/bridge/bridge.ko
insmod: ERROR: could not load module /usr/lib/modules/4.4.8/4.4.8/kernel/net/bridge/bridge.ko: No such file or directory
root@vlab65 [ /usr/lib/modules/4.4.8 ]# ls
kernel modules.alias.bin modules.builtin.bin modules.dep.bin modules.order modules.symbols
modules.alias modules.builtin modules.dep modules.devname modules.softdep modules.symbols.bin
root@vlab65 [ /usr/lib/modules/4.4.8 ]# cd kernel
root@vlab65 [ /usr/lib/modules/4.4.8/kernel ]# ls
arch crypto drivers fs lib net security virt
root@vlab65 [ /usr/lib/modules/4.4.8/kernel ]# cd net
root@vlab65 [ /usr/lib/modules/4.4.8/kernel/net ]# ls
802 9p ceph dccp ipv4 key mpls netlink packet sched sunrpc vmw_vsock
8021q bridge core dns_resolver ipv6 llc netfilter openvswitch rds sctp unix xfrm
root@vlab65 [ /usr/lib/modules/4.4.8/kernel/net ]# cd bridge
root@vlab65 [ /usr/lib/modules/4.4.8/kernel/net/bridge ]# ls
bridge.ko br_netfilter.ko netfilter
root@vlab65 [ /usr/lib/modules/4.4.8/kernel/net/bridge ]#
I got this to work...must of had a typo.
HI all i got also an error. Problem is that the symlinks aren't there you have to run Step 3 before Step 2 there will be an error too but now the links have been added. So you will able to do Step 2.
Thank's William for the great article!
Interesting that vCenter appliance is on Photon, but vCenter itself doesn't seem to be containerized. I wonder if that's the next step?
Hello,
if I try to do this:
insmod /usr/lib/modules/4.19.247-7.ph3/kernel/net/bridge/bridge.ko.xz
I got these errors:
bridge: Unknown symbol llc_mac_hdr_init (err -2)
bridge: Unknown symbol stp_proto_register (err -2)
bridge: Unknown symbol stp_proto_unregister (err -2)
Also tried to UNXZ the bridge file but same errors.
I'm on VMware vCenter Server 7.0.3.00700
Thank you
To be honest, if you have a need to run a container ... you should consider looking at something like VMware Event Broker Appliance (VEBA) which will be more scalable for such use cases