Over the weekend, I had finished building a small container application that I had developed on my local desktop, which worked great. However, the real test was to deploy the application on a vSphere Kubernetes Service (VKS) Cluster, which would require the container image to be hosted in a container registry.
Rather than using a public container registry, like many organizations, you typically would setup an internal registry that your infrastructure could have access to. Harbor is my go to container registry and is extremely simple to setup as it uses docker-compose under the hood.
After uploading my container image to Harbor, if you try to deploy it to your VKS Cluster, you will immediately see the following image pull error:
tls: failed to verify certificate: x509: certificate signed by unknown authority
Just like any system that attempts to connect to an endpoint with a self-signed TLS certificate that it can not verify, it will not trust which is the default behavior.
For our VKS Cluster to be able to successfully pull an image from Harbor with a self-signed TLS certificate, we need to add the trusted CA to our vSphere Namespace and then reference that as part of our VKS Cluster deployment.
Step 1 - We need to retrieve the double base64 encoding of container registry CA Certificate. If you are using Harbor, that would be the ca.crt that was generated and you can then run the following command to generate the double-encoding output:
base64 -w 0 ca.crt | base64 -w 0
Step 2 - We need to create a Kubernetes secret that will contain the trusted CA Certificate and deployed into the vSphere Namespace(s) where your VKS Cluster(s) will be deployed and reference.
Create a new called registry-ca-trust-secret.yaml (or any other name you wish to use) with the following content and add your double based64 encoding of the CA certificate as shown below.
apiVersion: v1
data:
harbor-ca: |
TF..YOUR.ENCODED.CA.CERT...Rbz0=
kind: Secret
metadata:
name: demo-user-trusted-ca-secret
namespace: development
type: Opaque
Additionally, you will also need to make a note of the following properties in the Kubernetes Secret:
- namespace property needs to match your vSphere Namespace. In my example, it is called development
- name property can be named anything, but just make a note of it as you will need it when creating your VKS Cluster. In my example, it is called demo-user-trusted-ca-secret
- to access the encoded data, there is a "key" that can be named anything, just make a note of it as you will need it when creating your VKS Cluster. In my example, it is called harbor-ca
Step 3 - Switch to your vSphere Supervisor Kubernetes context and create the secret under our desired vSphere Namespace:
kubectl apply -f registry-ca-trust-secret.yaml
Step 4 - When creating a new VKS Cluster, we need to reference the secret that we had created from the previous step into the osConfiguration variable for trusting a given CA certificate. The easiest way to understand the required YAML is to use the Local Consumption Interface (LCI) operator that provides a nice graphical way to deploy various vSphere Supervisor Services.
You must select the Custom Configuration type flow and you will Trust as optional input parameter as shown in screenshot below. The value for the key will be harbor-ca or whatever you had named it in Step 2. The value for name will be demo-user-trusted-ca-secret or whatever you had named the Kubernetes Secret from Step 2.
Step 5 - Once the VKS Cluster is up and running, we can now successfully deploy the container image from our registry that has a self-signed TLS certificate.
Note: For my Harbor setup, the project allowed anonymous image pulls but if your container registry requires a login, you will need to create an image pull secret.
Thanks for the comment!