Affected Component: Konvoy
Problem
Depending on the size of your cluster, the nodes in the cluster, and the workload running on your nodes, you may find that the default resource limits for addons are not sufficient for your use case. It will be helpful to be able to modify the resource limits and the best place to do that is from your cluster.yaml file. You could manually modify the addons via kubectl edit, but those changes will not persist between cluster updates. Modifying the cluster.yaml file will ensure that your configuration is preserved.
Solution
As an example of how you can override the default values of an addon, we'll be modifying Fluentbit to increase the memory limit for each pod from 500Mi to 1500Mi. Locate the definition for fluentbit in your cluster.yaml:
- name: fluentbit enabled: true
You will notice that there are no resource settings set in cluster.yaml by default. We can examine what the defaults are via kubectl:
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE fluentbit-kubeaddons-fluent-bit 7 7 7 7 7 11m metallb-kubeaddons-speaker 7 7 7 7 7 beta.kubernetes.io/os=linux 11m prometheus-kubeaddons-prometheus-node-exporter 7 7 7 7 7 6m27s
(Output of the following is truncated to show only the resource information)
kubectl describe ds fluentbit-kubeaddons-fluent-bit --namespace=kubeaddons
Containers:
fluent-bit:
Image: fluent/fluent-bit:1.3.2
Port: 2020/TCP
Host Port: 0/TCP
Limits:
memory: 500Mi
Requests:
cpu: 200m
memory: 200Mi
Environment:
From this we can infer the default settings for fluentbit. With this information we can now increase these limits to test how much more resources we need. We can modify the fluentbit configuration in cluster.yaml to specify a new memory limit:
- name: fluentbit
enabled: true
values: |
resources:
limits:
memory: 1500Mi
After making the above change we can deploy a new cluster with higher limits or update the existing cluster via:
konvoy up
If we are only modifying addons for an existing cluster, we can just run:
konvoy deploy addons
After the command runs successfully, we can check that fluentbit is now using the new memory limit we specified via:
(Output of the following is truncated to show only the resource information)
kubectl describe ds fluentbit-kubeaddons-fluent-bit --namespace=kubeaddons
Containers: fluent-bit: Image: fluent/fluent-bit:1.3.2 Port: 2020/TCP Host Port: 0/TCP Limits: memory: 1500Mi Requests: cpu: 200m memory: 200Mi Environment:
Memory isn't the only resource that can be modified, and you don't have to settle for just changing the limits either! Here are some more examples of values you can modify:
- name: elasticsearch
enabled: true
values: |
client:
heapSize: 2048m
resources:
limits:
cpu: 4000m
memory: 4096Mi
requests:
cpu: 2000m
memory: 2048Mi
In this example, we see that both the limit and request for CPU and memory are being modified. Additionally, we see that the heap size is being set.
The values that you can modify will vary between addons, but all possible configurable values can be found in each addon's helm values. For information on working with helm values in Konvoy see the following KB:
https://support.d2iq.com/hc/en-us/articles/4409472602004-Where-To-Find-Konvoy-Addon-Configuration-Values
You can also review our documentation on addons here:
https://docs.d2iq.com/ksphere/konvoy/latest/reference/cluster-configuration/#addon