When installing Photon Controller, the resource configuration of the Management VM is sized dynamically as mentioned here based on the total available CPU, Memory and Storage on the physical ESXi host that it is being provisioned to. This is generally not a problem when deploying Photon Controller in Production with larger hosts but if you are trying to play with it in a home lab or a resource constrained environment, then this can be a challenge.
Currently, the minimal requirement to play with Photon Controller is a single physical or Nested ESXi VM that is configured with at least 4vCPU, 16GB of memory and 50GB of storage. The biggest constraint for most home labs is usually on memory. As an example, using the configuration above, the default size used for the Photon Controller Management VM is 2vCPU and 4GB of memory which is quite hefty for such a small environment. It potentially could get worse with slightly larger hosts and ultimately this impacts the amount of workload you can run on the ESXi host, especially if you only have one.
In talking to one of the Engineers on the Photon Controller team, I learned about a neat little capability that is currently only available in the Photon CLI which allows you to override the default CPU, Memory and Storage settings for the Photon Controller Management VM. The following three variables can be added to a deployment configuration YAML file which will override the default behavior.
UPDATE (06/02/16) - In v0.9 release of Photon Controller, the MANAGMENT_VM_MEMORY_GB_OVERWRITE variable has been renamed to MANAGEMENT_VM_MEMORY_MB_OVERWRITE. The rest should be the same.
- MANAGEMENT_VM_CPU_COUNT_OVERWRITE - Number of vCPUs for the Management VM
- MANAGEMENT_VM_MEMORY_GB_OVERWRITE - Amount of Memory for the Management VM (It is actually in MB even though variable says GB)
- MANAGEMENT_VM_DISK_GB_OVERWRITE - Amount of storage for the Management VM (there seems to be a bug but property does not actually override the default storage configuration)
Note: One thing that I found while testing out this capability is that you MUST specify all three variables regardless if wish to override one or more resources. If you do not, you will see a strange 500 error code when running the CLI. I assume this is probably a bug and have already reported this to the Engineering team.
Below are the recommended instructions if you plan to override the default configuration for the Photon Controller Management VM.
Step 1 - Open a browser to the IP Address of your Photon Controller Installer VM and go through the wizard as you normally would, but DO NOT click on the Deploy button once you are done. Instead, click on the "Export Configuration" option and save your configuration to your desktop. You can then close the Photon Controller Installer UI window as we will not be using the UI to deploy.
Step 2 - Open the Photon Controller deployment configuration YAML file that you had just saved in the previous step using a text editor of your choice. There will be two modifications that we will need to make. The first is by adding the following three variables under the "metadata" section towards the top and replacing the values with the ones you wish to use. I recommend using 2vCPU/2GB of memory. For storage, there seems to be a bug in which the override does not work, but you STILL MUST specify it in the configuration file else the deployment will fail. Go ahead and leave it as the default 80.
MANAGEMENT_VM_CPU_COUNT_OVERWRITE: 2
MANAGEMENT_VM_MEMORY_GB_OVERWRITE: 2048
MANAGEMENT_VM_DISK_GB_OVERWRITE: 80
Step 3 - The second modification that we need to make to the YAML file is how the datastores are listed under the image_datastores property. In the UI, it stores this property as a collection. However, when using the Photon CLI, it expects it as a string. The fix is quite simple, you just need to change the following
from
deployment:
image_datastores:
- datastore1
to
deployment: image_datastores: datastore1
At this point, we are done modifying our YAML configuration file and we can save our changes and get ready to deploy.
Step 4 - You will need the Photon CLI for the remainder of the steps. If you have not downloaded the Photon CLI, take a look here for the details. Point the Photon CLI to the IP Address of your Photon Controller Installer VM by running the following command:
./photon target set http://192.168.1.250
Step 5 - We will now deploy Photon Controller using the CLI and overriding the default algorithm on how the Photon Controller Management VM is configured by running the following command and specifying the full path to your YAML file:
./photon system deploy esxcloud-installation-export-config-vghetto-sample.yaml
Once the deployment has started, you will be provided with a progress bar. If everything is successful, you should be able to login to your ESXi host using either the ESXi Embedded Host Client or vSphere C# Client and you should see that your Photon Controller Management VM has been deployed with the overrides you had specified earlier.
If you are new to Photon Controller, be sure to check out my blog series on test driving Photon Controller:
- Test driving VMware Photon Controller Part 1: Installation
- Test driving VMware Photon Controller Part 2: Deploying first VM
- Test driving VMware Photon Controller Part 3a: Deploying Kubernetes
- Test driving VMware Photon Controller Part 3b: Deploying Mesos
- Test driving VMware Photon Controller Part 3c: Deploying Docker Swarm
I have been trying to get this going for a while and it turns out, for me at least, 2GB of RAM was not enough, the deployment would keep failing. I just tried with 8GB and it worked OK. Just passing it on...