Skip to main content

How to resize the Prometheus volume in DKP 2.X

Steven Sylvester
  • Updated
If you have found that the volume used for Kommander's built-in Prometheus monitoring app is not sufficient, you can follow these steps to resize it. Please note that these steps will involve deleting the existing Prometheus volume and all of the data currently in it. At this time we do not have a way to resize the volume while preserving the data.
 

Solution

1. First, follow the steps in this article to edit the Prometheus Monitoring application:
 
Look for the "prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage" value in the Edit menu (default is 100Gi) and change it to your desired value.   After editing the value, wait a few momemts, and check the output of "kubectl get helmrelease -n kommander" until all changes are reconciled.
 
2. Delete the "prometheus" object in the kommander namespace:
kubectl delete prometheus -n kommander kube-prometheus-stack-prometheus
 
3. Delete the PVC:
kubectl delete pvc -n kommander db-prometheus-kube-prometheus-stack-prometheus-0
The PV should also be automatically deleted with this step, but you can check yourself to make sure that this is the case.
4. Trigger a reconciliation of the helmrelease:
kubectl -n kommander patch helmrelease kube-prometheus-stack --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true}]'
kubectl -n kommander patch helmrelease kube-prometheus-stack --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": false}]'
5. Watch the output of "kubectl get helmrelease -n kommander" until the reconciliation is completed. There should be a new PVC created with the new size that you specified in step 1.