Skip to main content

Enabling Volume Expansion in the default Storage Provider

Craig Neth
  • Updated

Overview

DKP Versions impacted:  DKP 2.3, DKP 2.4, DKP 2.5

The default storage class provider in AWS, EKS, and GCP deployments are configured by default with the Kubernetes AutoVolumeExpansion feature disabled.    

This feature can be useful for many applications, as it allows volumes to resize as usage increases.  Follow the instructions below to enable it.

Note that this feature is enabled by default for Azure and AKS deployments.

Solution

To enable automatic volume expansion, you must edit the ConfigMap that contains the ClusterResourceSet configuration, and may need to add additional permissions.

On AWS and EKS:

If you are running on AWS, ensure that the IAM instance profile for your control plane nodes (normally named control-plane.cluster-api-provider-aws.sigs.k8s.io) has the ec2:DescribeVolumesModifications permission.   This permission is only needed on the control-plane nodes as the EBS CSI controller only runs on control-plane nodes.   

If you are using EKS, ensure that the EBS CSI driver policy is attached the control plane role:

  aws iam attach-role-policy --role-name nodes.cluster-api-provider-aws.sigs.k8s.io
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy


Then, kubectl edit the aws-ebs-csi-<CLUSTER_NAME> ConfigMap, and add the allowVolumeExpansion: true flag to the aws-ebs-csi.yaml section:

apiVersion: v1
data:
  aws-ebs-csi.yaml: |
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
      name: ebs-sc
    parameters:
      csi.storage.k8s.io/fstype: ext4
      type: gp3
    allowVolumeExpansion: true
    provisioner: ebs.csi.aws.com
    volumeBindingMode: WaitForFirstConsumer
    ---
... [remainder of the resource not shown]


On GCP:

On GCP, the necessary permissions are part of theroles/compute.storageAdminrole.

Proceed to kubectl edit the: gcp-persistent-disk-csi-<CLUSTER_NAME> ConfigMap, and add the allowVolumeExpansion: true flag to the gcp-persistent-disk-csi.yaml section:

apiVersion: v1
data:
  gcp-persistent-disk-csi.yaml: |
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      annotations:
        storageclass.kubernetes.io/is-default-class: "true"
      name: csi-gce-pd
    parameters:
      type: pd-standard
    allowVolumeExpansion: true
    provisioner: pd.csi.storage.gke.io
    volumeBindingMode: WaitForFirstConsumer
    ---

... [remainder of the resource not shown]