How to enable CoreDNS query logging?

Enabling query logging in CoreDNS is crucial for effectively troubleshooting DNS issues within a Kubernetes cluster. This becomes especially important because CoreDNS is the DNS service provider within a Kubernetes cluster deployed with DKP.

Solution

CoreDNS is configured as a deployment with 2 replicas in the kube-system namespace:

kubectl -n kube-system get deployment -l=k8s-app=kube-dns 
NAME     READY UP-TO-DATE AVAILABLE   AGE
coredns  2/2   2          2           5h46m

The DNS service is exposed as a Kubernetes service with a clusterIP address:

kubectl -n kube-system get svc -l=k8s-app=kube-dns 
NAME      TYPE        CLUSTER-IP  EXTERNAL-IP   PORT(S)                 AGE
kube-dns  ClusterIP   10.0.0.10   <none>        53/UDP,53/TCP,9153/TCP  5h47m

CoreDNS can be customized via the corefile stored in the coredns configmap in the kube-dns namespace.  To enable query logging, the logs plugin must be included in the configmap, as the example shown below:

kubectl -n kube-system get cm coredns -oyaml 
apiVersion: v1
data:
  Corefile: |
    .:53 {
        logs
        errors
        health {
          lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
          ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf {
          max_concurrent 1000
        }
        cache 30
        loop
        reload
        loadbalance
     }
…

To do so, the coredns configmap should be edited to match the above example:

kubectl edit cm coredns -n kube-system

then recycle the coredns pods:

kubectl delete pod -l=k8s-app=kube-dns -n kube-system

Since DNS lookups occur at a high rate in most clusters, we recommend removing the logs plugin from the configuration after you have finished debugging.

For additional guidance on the log format please refer to the CoreDNS logs plugin documentation.