Issue with node labels causing kube-state-metrics to be down

Overview/Background

When running a Konvoy 1.5.2 cluster with the Prometheus kubeaddon, you may run into an issue where kube-state-metrics is down. If you encounter this issue and you see the following error in the log, this may be caused by a change in how a '/' in node labels is handled in previous versions of Konvoy:

E0930 15:04:55.803488 1 reflector.go:382] pkg/mod/k8s.io/client-go@v0.0.0-20191109102209-3c0d1af94be5/tools/cache/reflector.go:108: expected type *v1.MutatingWebhookConfiguration, but watch event object had type *v1beta1.MutatingWebhookConfiguration

For instance, if you are using node labels as shown below, this will no longer work because both the '/' and the '.' are seen as identical and a node can't have two labels of the same name:

foo/bar=baz
foo.bar=baz

Solution

In order to resolve the issue, you can simply remove or modify one of the labels so that they are not considered the same.