Tags not getting applied to AWS ELBs

Overview/Background

Utilizing Kubernetes cloud provider integration, you can add annotations[1] to a Service type LoadBalancer to manage AWS Elastic Load Balancers. One common use case is adding tags to your ELBs[2]:

metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "environment=prod,owner=devops"

If your control plane nodes do not have the proper policy applied to their IAM role, you may observe the tags not being applied and log lines such as the following in the kube-controller-manager logs:

I0903 22:24:23.087012       1 event.go:281] Event(v1.ObjectReference{Kind:"Service", Namespace:"kubeaddons", Name:"traefik-kubeaddons", UID:"3ade3160-2eaa-4f5a-afa4-a37115667c0b", APIVersion:"v1", ResourceVersion:"12817", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: unable to create additional load balancer tags: error adding tags to load balancer: AccessDenied: User: arn:aws:sts::123456789:assumed-role/test-lb-8b70-node-role/i-0ef75e4d964327ea5 is not authorized to perform: elasticloadbalancing:AddTags on resource: arn:aws:elasticloadbalancing:us-west-2:123456789:loadbalancer/a3ade31602eaa4f5aafa4a37115667c0

[1] https://kubernetes.io/docs/concepts/services-networking/service/
[2] https://kubernetes.io/docs/concepts/services-networking/service/#other-elb-annotations

Solution

To resolve this issue, you can add the following policy to your control plane nodes' IAM role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags"
            ],
            "Resource": "*"
        }
    ]
}

This policy has been added by default as of the following Konvoy versions:

• Konvoy 1.4.7+
• Konvoy 1.5.3+
• Konvoy 1.6.0b1+