WilliamLam.com

Test driving VMware Photon Controller Part 3b: Deploying Mesos

04.26.2016 by William Lam // 4 Comments

In the previous article, we demonstrated the first Cluster Orchestration solution supported by Photon Controller by deploying a fully functional Kubernetes Cluster using Photon Controller. In this article, we will now look at deploying a Mesos Cluster using Photon Controller.

The minimal deployment for a Mesos Cluster in Photon Controller consists of 6 Virtual Machines: 3 Masters, 1 Zookeeper, 1 Marathon & 1 Slave. If you only have 16GB of memory on your ESXi host, then you will need to override the default VM Flavor when deploying a Mesos Cluster. If you have more than 16GB of available memory, then you can skip Step 1 and move to Step 2 directly.

Deploying Mesos Cluster

Step 1 - If you have not already created a new cluster-tiny-vm VM Flavor from the previous article that consists of 1vCPU/1GB memory, please run the following command:

./photon -n flavor create --name cluster-tiny-vm --kind "vm" --cost "vm 1 COUNT,vm.flavor.cluster-other-vm 1 COUNT,vm.cpu 1 COUNT,vm.memory 1 GB,vm.cost 1 COUNT"

Step 2- Download the Mesos VMDK from here

Step 3 - We will now upload our Mesos image and make a note of the ID that is generated after the upload completes by running the following command:

./photon -n image create photon-mesos-vm-disk1.vmdk -n photon-meos-vm.vmdk -i EAGER

Step 4 - Next, we will also need the ID of our Photon Controller Instance deployment as it will be required in the next step by running the following command:

./photon deployment list

Step 5 - We will now enable the Mesos Cluster Orchestration on our Photon Controller instance by running the following command and specifying the ID of your deployment as well as the ID of the Mesos image from the previous two steps:

./photon -n deployment enable-cluster-type 569c3963-2519-4893-969c-aed768d12623 -k MESOS -i 51c331ea-d313-499c-9d8f-f97532dd6954

Step 6 - We are now ready to spin up our Mesos Cluster by simply running the following command and substituting the network information from your environment. We are going to only deploying a single Mesos Slave (if you have additional resources you can spin up more or you can always re-size the cluster after it has been deployed). Do not forget to override the default VM Flavor used by specifying -v option and providing the name of our VM Flavor which we had created earlier called cluster-tiny-vm. You can just hit enter when prompted for the two zookeeper IP Addresses.

./photon cluster create -n mesos-cluster -k MESOS --dns 192.168.1.1 --gateway 192.168.1.1 --netmask 255.255.255.0 --zookeeper1 192.168.1.45 -s 1 -v cluster-tiny-vm

Step 7 - The process can take a few minutes and you should see a message like the one shown above which prompts you to run the cluster show command to get more details about the state of the cluster.

./photon cluster show bf962c3a-28a2-435d-bd96-0313ca254667

At this point, you have now successfully deployed a Mesos cluster running on Photon Controller. What you will be looking for in this screen is the IP Address of the Marathon VM which is the management interface to Mesos. We will need this IP Address in the next section if you plan to explore Mesos a bit more.

Exploring Mesos

Using the IP Address obtained from the previous step, you can now open a web browser and enter the following: http://[MARATHON-IP]:8080 which should launch the Marathon UI as shown in the screenshot below. If you wish to deploy a simple application using Marathon, you can follow the workflow here. Since we deployed Mesos using a tiny VM Flavor, we would not be able to exercise the final step of deploying an application running on Mesos. If you have more resources, I definitely recommend you give the workflow a try.

In our last and final article of the series, we will be covering the last Cluster Orchestration supported on Photon Controller which is Docker Swarm.

How to override the default CPU/Memory when deploying Photon Controller Management VM?

04.25.2016 by William Lam // 1 Comment

When installing Photon Controller, the resource configuration of the Management VM is sized dynamically as mentioned here based on the total available CPU, Memory and Storage on the physical ESXi host that it is being provisioned to. This is generally not a problem when deploying Photon Controller in Production with larger hosts but if you are trying to play with it in a home lab or a resource constrained environment, then this can be a challenge.

Currently, the minimal requirement to play with Photon Controller is a single physical or Nested ESXi VM that is configured with at least 4vCPU, 16GB of memory and 50GB of storage. The biggest constraint for most home labs is usually on memory. As an example, using the configuration above, the default size used for the Photon Controller Management VM is 2vCPU and 4GB of memory which is quite hefty for such a small environment. It potentially could get worse with slightly larger hosts and ultimately this impacts the amount of workload you can run on the ESXi host, especially if you only have one.

In talking to one of the Engineers on the Photon Controller team, I learned about a neat little capability that is currently only available in the Photon CLI which allows you to override the default CPU, Memory and Storage settings for the Photon Controller Management VM. The following three variables can be added to a deployment configuration YAML file which will override the default behavior.

UPDATE (06/02/16) - In v0.9 release of Photon Controller, the MANAGMENT_VM_MEMORY_GB_OVERWRITE variable has been renamed to MANAGEMENT_VM_MEMORY_MB_OVERWRITE. The rest should be the same.

MANAGEMENT_VM_CPU_COUNT_OVERWRITE - Number of vCPUs for the Management VM
MANAGEMENT_VM_MEMORY_GB_OVERWRITE - Amount of Memory for the Management VM (It is actually in MB even though variable says GB)
MANAGEMENT_VM_DISK_GB_OVERWRITE - Amount of storage for the Management VM (there seems to be a bug but property does not actually override the default storage configuration)

Note: One thing that I found while testing out this capability is that you MUST specify all three variables regardless if wish to override one or more resources. If you do not, you will see a strange 500 error code when running the CLI. I assume this is probably a bug and have already reported this to the Engineering team.

Below are the recommended instructions if you plan to override the default configuration for the Photon Controller Management VM.

Step 1 - Open a browser to the IP Address of your Photon Controller Installer VM and go through the wizard as you normally would, but DO NOT click on the Deploy button once you are done. Instead, click on the "Export Configuration" option and save your configuration to your desktop. You can then close the Photon Controller Installer UI window as we will not be using the UI to deploy.

Step 2 - Open the Photon Controller deployment configuration YAML file that you had just saved in the previous step using a text editor of your choice. There will be two modifications that we will need to make. The first is by adding the following three variables under the "metadata" section towards the top and replacing the values with the ones you wish to use. I recommend using 2vCPU/2GB of memory. For storage, there seems to be a bug in which the override does not work, but you STILL MUST specify it in the configuration file else the deployment will fail. Go ahead and leave it as the default 80.

MANAGEMENT_VM_CPU_COUNT_OVERWRITE: 2
MANAGEMENT_VM_MEMORY_GB_OVERWRITE: 2048
MANAGEMENT_VM_DISK_GB_OVERWRITE: 80

Step 3 - The second modification that we need to make to the YAML file is how the datastores are listed under the image_datastores property. In the UI, it stores this property as a collection. However, when using the Photon CLI, it expects it as a string. The fix is quite simple, you just need to change the following

from

deployment:
  image_datastores:
    - datastore1

deployment:
  image_datastores: datastore1

At this point, we are done modifying our YAML configuration file and we can save our changes and get ready to deploy.

Step 4 - You will need the Photon CLI for the remainder of the steps. If you have not downloaded the Photon CLI, take a look here for the details. Point the Photon CLI to the IP Address of your Photon Controller Installer VM by running the following command:

./photon target set http://192.168.1.250

Step 5 - We will now deploy Photon Controller using the CLI and overriding the default algorithm on how the Photon Controller Management VM is configured by running the following command and specifying the full path to your YAML file:

./photon system deploy esxcloud-installation-export-config-vghetto-sample.yaml

Once the deployment has started, you will be provided with a progress bar. If everything is successful, you should be able to login to your ESXi host using either the ESXi Embedded Host Client or vSphere C# Client and you should see that your Photon Controller Management VM has been deployed with the overrides you had specified earlier.

If you are new to Photon Controller, be sure to check out my blog series on test driving Photon Controller:

Test driving VMware Photon Controller Part 3a: Deploying Kubernetes

04.21.2016 by William Lam // 6 Comments

If you have been following the series thus far, we have covered installing Photon Controller in Part 1 and then we learned how to create our first virtual machine using Photon Controller in Part 2. Next up, we will demonstrate how easy it is to stand up the three different Cluster Orchestration solutions that are supported on top of Photon Controller, starting with Kubernetes. Once the Cluster Orchestration solution has been setup, you can then deploy your application like you normally would through the Cluster Orchestration and behind the scenes, Photon Controller will automatically provision the necessary infrastructure to run your given application without having to know anything about the underlying resources.

If you recall from our last article, there are several default VM Flavors that are included in the Photon Controller installation. The ones that are named cluster-* are VM Flavors used for deploying the Cluster Orchestration virtual machines that have been configured to support high scale and throughput (up to 4vCPU and 8GB of memory). If you are testing this in a lab environment where you might be constrained on memory resources for your ESXi host (16GB of memory), then you actually have a few options. The first option is to create a new VM Flavor with a smaller configuration (e.g. 1vCPU/2GB memory) and then override the default VM Flavor when deploying the Cluster Orchestration. The second option which I learned from talking to Kris Thieler was that you can actually re-define the default cluster-* VM Flavors to fit your environment needs which he has documented here. To simplify our deployment, we will actually use Option 1 on just creating a new VM Flavor that we will use to override the default VM Flavor. If you have more than 16GB of memory, then you can skip Step 2.

Deploying Kubernetes Cluster

Step 1 - Download the Kubernetes VMDK from here and the Kubectl binary from here.

Step 2 - Run the following command to create our new VM Flavor override which we will call cluster-tiny-vm that is configured with 1vCPU/1GB of memory:

./photon -n flavor create --name cluster-tiny-vm --kind "vm" --cost "vm 1 COUNT,vm.flavor.cluster-other-vm 1 COUNT,vm.cpu 1 COUNT,vm.memory 1 GB,vm.cost 1 COUNT"

Step 3 - We will now upload our Kubernetes image and make note of the ID generated after the upload by running the following command:

./photon -n image create photon-kubernetes-vm-disk1.vmdk -n photon-kubernetes-vm.vmdk -i EAGER

Step 4 - Next, we also need the ID of our Photon Controller Instance deployment as it will be required in the next step by running the following command:

./photon deployment list

Step 5 - We will now enable the Kubernetes Cluster Orchestration on our Photon Controller instance by running the following command and specifying the ID of your deployment as well as the ID of the Kubernetes image from the previous two steps:

./photon -n deployment enable-cluster-type 7fd9a13d-e69e-4165-9b34-d436f4c67ea1 -k KUBERNETES -i 4332af67-2ff0-49f7-ba44-dd4140908e32

Step 6 - We can also see what Cluster Orchestration solutions have been enabled for our Photon Controller by running the following command and specifying our deployment ID:

./photon deployment show 7fd9a13d-e69e-4165-9b34-d436f4c67ea1

As you can see from the screenshot above, there is a Cluster Configuration section which provides a list of Cluster Orchestration solutions that have been enabled as well as their respective image.

Step 7 - We are now ready to spin up our Kubernetes (K8) Cluster by simply running the following command and substituting the network information from your environment. We are also going to only deploying a single K8 Slave (if you have additional resources you can spin up more or you can always re-size the cluster after it has been deployed) and lastly, we will override the default VM Flavor used by specifying -v option and providing the name of our VM Flavor called cluster-tiny-vm. You can just hit enter when prompted for the two etcd IP Addresses, the assumption is that you have DHCP running and those will automatically obtain an address.

./photon cluster create -n k8-cluster -k KUBERNETES --dns 192.168.1.1 --gateway 192.168.1.1 --netmask 255.255.255.0 --master-ip 192.168.1.55 --container-network 10.2.0.0/16 --etcd1 192.168.1.56 -s 1 -v cluster-tiny-vm

Step 8 - The process can take a few minutes and you should see a message like the one shown above which prompts you to run the cluster show command to get more details about the state of the cluster.

./photon cluster show 9b159e92-9495-49a4-af58-53ad4764f616

Exploring Kubernetes

At this point, you have now successfully deployed a fully functional K8 Cluster using Photon Controller with just a single command. We can now take explore our K8 setup a bit by using the kubectl CLI that you had downloaded earlier. For more information on how to interact with K8 Cluster using kubectl command, be sure to check out the official K8 documentation here.

To view the nodes within the K8 Cluster, you can run the following command and specifying the IP Address of the master VM provided in the previous step:

./kubectl -s 192.168.1.55:8080 get nodes

Lets now do something useful with our K8 Cluster and deploy a simple Tomcat application. We first need to download the following two configuration files that will define our application:

We then need to edit the photon-Controller-Tomcat-rc.yml file and delete the last two lines as it contains an incorrect syntax:

labels:
name: "tomcat-server"

To deploy our application, we will run the following two commands which will setup our replication controller as well as the service for our Tomcat application:

./kubectl -s 192.168.1.55:8080 create -f photon-Controller-Tomcat-rc.yml
./kubectl -s 192.168.1.55:8080 create -f photon-Controller-Tomcat-service.yml

We can then check the status of our application deloyment by running the following command:

./kubectl -s 192.168.1.55:8080 get pods

You should see a tomcat-server-* entry and the status should say "Image: tomcat is not ready on the node". You can give it a few seconds and then re-run the command until it is showing "Running" as the status which means our application has been successfully deployed by the K8 Cluster.

We can now open a browser to the IP Address of our K8 Master VM's IP, which in my environment was 192.168.1.55 and specify port 30001 which was defined in the configuration file of Tomcat application and we should see that we now have Tomcat running.

We can also easily scale up the number of replication servers for our Tomcat application by running the following command:

./kubectl -s 192.168.1.55:8080 scale --replicas=2 rc tomcat-server

You can easily scale the application back down by re-running the command and specifying a value of one. Lastly, if we want to delete our application, we can run the following two commands:

./kubectl -s 192.168.1.55:8080 delete service tomcat
./kubectl -s 192.168.1.55:8080 delete rc tomcat-server

Once we are done using using our K8 Cluster, we can tear it down by specifying the ID of the K8 Cluster found in Step 8 by running the following command which will now delete the VMs that Photon Controller had deployed:

./photon -n cluster delete 9b159e92-9495-49a4-af58-53ad4764f616

Hopefully this gave you a quick taste on how easy it is to setup a fully functional K8 Cluster using Photon Controller. In the next article, we will take a look at deploying a Mesos Cluster using Photon Controller, so stay tuned!