Istio not applying overrides in DKP 2.1

Problem

When you deploy an application from Kommander, it is accompanied by a helmrelease object deployed on your Kubernetes cluster. In this case, the underlying helmrelease for Istio is missing the overrides value in the spec.valuesFrom section. You can validate this by running kubectl describe helmrelease -n kommander istio and viewing the Values From: section of the output:

Spec:
  Chart:
    Spec:
      Chart: istio
      Source Ref:
        Kind: HelmRepository
        Name: mesosphere.github.io-charts-staging
        Namespace: kommander-flux
    Version: 1.9.1
  Values From:
    Kind: ConfigMap
    Name: istio-1.9.1-d2iq-defaults
...

Because the override configmap is not associated with the helmchart, all of its contents will be ignored, and the values in the 'istio-1.9.1-d2iq-defaults' configmap will be used.

Solution

To resolve this, you can create the istio-overrides configmap manually and modify the kustomization object associated with the helmrelease to include the overrides. Below I have created a sample configmap to provide custom resource requests to the Operator and applied it to my cluster:

apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: istio-overrides
  namespace: kommander
data:
  values.yaml: |
    operator:
      resources:
        limits:
          cpu: 500m
          memory: 256Mi
        requests:
          cpu: 50m
          memory: 128Mi

After the configmap is applied, you will need to go and edit the kustomization object using kubectl edit kustomizations.kustomize.toolkit.fluxcd.io -n kommander istio-helmrelease and add the below yaml to the .spec:

patchesJson6902:
- patch:
  - op: add
    path: /spec/valuesFrom/-
    value:
      kind: ConfigMap
      name: istio-overrides
      optional: true
  target:
    kind: HelmRelease
    name: istio

Below is what your kustomization object should look like after applying the patch:

kubectl describe kustomizations.kustomize.toolkit.fluxcd.io -n kommander istio-helmrelease
spec:
  dependsOn:
  - name: istio-ca
  - name: istio-system-namespace
  force: false
  interval: 10s
  patchesJson6902:
    - patch:
      - op: add
        path: /spec/valuesFrom/-
        value:
          kind: ConfigMap
          name: istio-overrides
          optional: true
  target:
    kind: HelmRelease
    name: istio

Once you have applied the patch, the helm-controller will reconcile the state of the underlying helmrelease object. If, after a few minutes, you do not see the state of the istio helmrelease being reconciled and updated with your new configuration, you can safely delete the helmrelease using kubectl delete helmrelease -n kommander istio. After doing this, give the controller a few minutes to redeploy the helmrelease. Once it has been redeployed, describe the helmrelease using kubectl describe helmrelease -n kommander istio and validate that the overrides configmap is present:

Spec:
  Chart:
    Spec:
      Chart: istio
      Reconcile Strategy: ChartVersion
      Source Ref:
        Kind: HelmRepository
        Name: mesosphere.github.io-charts-staging
        Namespace: kommander-flux
      Version: 1.9.1
  Values From:
    Kind: ConfigMap
    Name: istio-1.9.1-d2iq-defaults
    Kind: ConfigMap
    Name: istio-overrides
    Optional: true

As well as this, you can also check your pods to validate that they have been recreated with your desired overrides; in this case, I can see that my resource requests have been applied:

kubectl describe pod -n istio-system istio-operator-76db5875c9-dbnt7
Containers:
  istio-operator:
    Limits:
      cpu: 500m
      memory: 256Mi
    Requests:
      cpu: 50m
      memory: 128Mi