Overview
There are certain situations where an APIserver pod is not running. Because kubectl commands need to be run through the APIserver, usual troubleshooting steps are not available. This article will give you a few things that you can check to look for hints about why the APIserver may not be running.
Solution
Finding the APIserver manifest file
The APIserver is a static pod, which means that we cannot inspect it's configuration using kubectl. Instead it is represented as a YAML file in the /etc/kubernetes/manifests directory. The kubelet on every node will automatically check this directory and attempt to deploy any YAML files it finds there. If one of these files is edited, the kubelet will also detect this and redeploy it with the new changes.
You can check the APIserver file for any possible configuration problems by connecting directly to the relevant node via SSH and opening /etc/kubernetes/manifests/kube-apiserver.yaml with your text editor of choice.
Please be aware that making any changes to the APIserver manifest file is not a supported action in DKP; you do so at your own risk.
Finding APIserver logs
As mentioned previously, kubectl commands will not work if the APIserver is down, so "kubectl logs" is not an option for troubleshooting APIserver problems.
If you would like to inspect the logs for APIserver, you can connect to the relevant node via SSH then navigate to the directory /var/log/pods. There will be a subdirectory there starting with kube-system_kube-apiserver-... corresponding to the ID of the APIserver pod. The most recent logs will be located there.
If there are no logs for any APIserver pods, then that means the pod is not even being created and getting to the point that it can print them. In that case, you can check Kubelet and containerd logs for more information about why the pod is not starting:
journalctl -u kubelet
journalctl -u containerd