Overview
When there is a requirement to deploy a DKP cluster with a lower version than the DKP default, the first step that comes to mind is to check the default and set the Kubernetes version flag using the DKP CLI:
dkp create cluster aws -h
...
--kubernetes-version string Kubernetes version (default "1.24.6")
This alone will not allow a cluster to deploy. If you attempt to deploy with the default AMI for your DKP version, you will see errors similar to the below:
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR KubeletVersion]: the kubelet version is higher than the control plane version. This is not a supported version skew and may lead to a malfunctional cluster. Kubelet version: "1.24.6" Control plane version: "1.23.7"
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
This is because the AMI used to create the machine has a Kubelet version installed on it that is higher than your Control Plane Kubernetes version.
Solution
To remedy this situation, you must create an AMI that contains the Kubelet version corresponding to the Kubernetes version you wish to deploy. The steps below outline the process:
1. Validate your DKP version
dkp version
diagnose: v0.5.1
dkp: v2.4.0
kommander: v2.4.0
konvoy: v2.4.0
mindthegap: v1.3.1
2. Validate Konvoy Image Builder version; you will need to download a compatible version for your DKP version. For DKP 2.4, this can be found under the Konvoy Image builder portion of our documentation
./konvoy-image version
konvoy-image, version v1.19.11 (branch: , revision: )
build date:
go version: go1.18.3
platform: linux/amd64
3. Build a new AMI containing the Kubernetes version we want, which in turn will install the appropriate Kubelet version:
./konvoy-image build images/ami/rhel-84.yaml --kubernetes-version 1.23.7
--KIB output omited--
==> Builds finished. The artifacts of successful builds are:
--> rhel-8.4: AMIs were created:
us-west-2: ami-ID
--> rhel-8.4: AMIs were created:
us-west-2: ami-ID
4. Create cluster with Kubernetes version specified and the new AMI we just built:
dkp create cluster aws -c adoll-0210 --kubernetes-version 1.23.7 --ami ami-ID
5. Wait for the cluster to deploy, after some time grab the kubeconfig and validate the cluster is running with the Kubernetes version specified during deployment:
dkp get kubeconfig -c dkp-testing > conf.conf
kubectl get nodes --kubeconfig conf.conf
NAME STATUS ROLES AGE VERSION
ip-10-0-105-80.us-west-2.compute.internal Ready 8m29s v1.23.7
ip-10-0-114-114.us-west-2.compute.internal Ready 8m29s v1.23.7
ip-10-0-184-84.us-west-2.compute.internal Ready control-plane,master 10m v1.23.7
ip-10-0-235-166.us-west-2.compute.internal Ready control-plane,master 8m27s v1.23.7
ip-10-0-73-162.us-west-2.compute.internal Ready control-plane,master 5m55s v1.23.7
ip-10-0-84-185.us-west-2.compute.internal Ready 8m29s v1.23.7
ip-10-0-85-162.us-west-2.compute.internal Ready 8m29s v1.23.7