Konvoy Service type LoadBalancer external IPs pending due to missing tags in AWS

Overview/Background

In certain scenarios (e.g., deploying Konvoy to a custom VPC), you may observe service type LoadBalancer external IPs in a pending state:

kubectl get svc -A | grep 'TYPE\|LoadBalancer'
NAMESPACE      NAME                                                      TYPE           CLUSTER-IP        EXTERNAL-IP   PORT(S)                                     AGE
kubeaddons     traefik-kubeaddons                                        LoadBalancer   10.100.100.111         80:32366/TCP,443:31287/TCP,8080:30937/TCP   2d2h
velero         minio-lb                                                  LoadBalancer   10.100.100.11          9000:31544/TCP                              2d2h

To troubleshoot this issue, you can check the kube-controller-manager logs:

kubectl logs -l component=kube-controller-manager -n kube-system

A common issue that can cause this behavior is missing the following tags for your control-plane and public subnets:

kubernetes.io/cluster = CLUSTER_NAME
kubernetes.io/cluster/CLUSTER_NAME = owned

Should this be the case, in the kube-controller-manager logs you may observe that the service could not be created due to no suitable tagged subnets existing:

I0412 11:11:11.111111       1 event.go:274] Event(v1.ObjectReference{Kind:"Service", Namespace:"kubeaddons", Name:"traefik-kubeaddons", UID:"11111111-1111-1111-1111-111111111111", APIVersion:"v1", ResourceVersion:"1111", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
W0412 11:11:11.111111       1 aws.go:3250] No tagged subnets found; will fall-back to the current subnet only.  This is likely to be an error in a future version of k8s.
E0412 11:11:11.111111       1 service_controller.go:255] error processing service kubeaddons/traefik-kubeaddons (will retry): failed to ensure load balancer: could not find any suitable subnets for creating the ELB

Solution

To resolve the issue, re-execute `konvoy provision` to obtain your cluster name, then ensure that you have tagged the VPC subnets for your control-plane and public resources with the following tags:

kubernetes.io/cluster = CLUSTER_NAME
kubernetes.io/cluster/CLUSTER_NAME = owned