When deploying a DKP node to pre-provisioned infrastructure, you may encounter an error that resembles the following:
2022-01-24T18:41:44.729Z ERROR controller.preprovisionedmachine Reconciler error {"reconciler group": "infrastructure.cluster.konvoy.d2iq.io", "reconciler kind": "PreprovisionedMachine", "name": "kaptain-test-control-plane-rcx4x", "namespace": "default", "error": "failed to run bootstrap: error running: \"/run/kubeadm/konvoy-set-kube-proxy-configuration.sh\": Process exited with status 126", "errorVerbose": "error running: \"/run/kubeadm/konvoy-set-kube-proxy-configuration.sh\": Process exited with status 126\nfailed to run bootstrap\ngithub.com/mesosphere/cluster-api-provider-preprovisioned/controllers.(*PreprovisionedMachineReconciler).reconcileNormal\n\t/workspace/controllers/preprovisionedmachine_controller.go:353\ngithub.com/mesosphere/cluster-api-provider-preprovisioned/controllers.(*PreprovisionedMachineReconciler).Reconcile\n\t/workspace/controllers/preprovisionedmachine_controller.go:175\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.7/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.7/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.7/pkg/internal/controller/controller.go:214\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1371"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.7/pkg/internal/controller/controller.go:214
A common cause for this issue is that the node's "/run" directory is mounted with "noexec". You can check this by running:
findmnt -l | grep noexec
In that output, look for the /run directory and check the mount options:
/run tmpfs tmpfs rw,nosuid,nodev,noexec,relatime,size=3282924k,mode=755
If it contains "noexec" like the above example, you will need to re-mount it without this option:
mount -o remount,rw,exec /run