How to control which storage backend is used by ElasticSearch to persists its data – D2iQ

Konvoy operators can control the amount of disk, CPU, and memory that elasticsearch pods can consume as described in the following article.

By specifying the parameter “storageClass” in the elasticsearch stanza, the Konvoy operator can control the storage backend that elasticsearch pods will use to persist their data.

When the storageclass is not specified, elasticsearch submits persistent volume claims (PVC) to bind persistent volumes in the storage class “localvolumeprovisioner” by default. The local volume provisioner creates persistent volumes (PV) for each disk in the /mnt/disk directory. To deploy elasticsearch using a storage backend in a different storage class, other than localvolumeprovisioner, a new storage class, and PVs must be created manually.

Example of Storage class (elastic-storageclass.yaml):

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: elastic-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

This is an example of a PV in the storage class “elastic-sc” (pv-elastic-sc.yaml), please note that there must be as many PVs in the storage class "elastic-sc" as the total of master and data nodes:

apiVersion: v1
kind: PersistentVolume
metadata:
 name: elastic-pv-volume-{X}
 labels:
   type: local
spec:
 storageClassName: elastic-sc
 capacity:
   storage: 10Gi
 accessModes:
   - ReadWriteOnce
 persistentVolumeReclaimPolicy: Delete   
 local:
   path: "/opt/elastic_master"
 nodeAffinity:
   required:
     nodeSelectorTerms:
     - matchExpressions:
       - key: kubernetes.io/hostname
         operator: In
         values:
         - konvoy-w{X}

To provision these resources, while the cluster is being created, the operator can create the files storageclass.yaml and pv-elastic-sc.yaml in extras/kubernetes.

This is how the storage class “elastic-sc” should be referred to in the cluster.yaml:

- name: elasticsearch
  enabled: true
  values: |
    master:
      replicas: 3     
      persistence:
        enabled: true
        accessMode: ReadWriteOnce
        name: data
        size: "10Gi"
        storageClass: "elastic-sc"     
    data:
      replicas: 4     
      persistence:
        enabled: true
        accessMode: ReadWriteOnce
        name: data
        size: "20Gi"
        storageClass: "elastic-sc"