In this article we provide the steps to configure life-cycle rules to automatically delete objects from the Loki bucket where grafana-loki ingester persists its data in MinIO tenant grafana-loki-minio in the DKP platform. Configuring rules to delete objects is recommended to manage storage usage and avoid exhaustion of space/inodes.
1. Create a busybox pod and install the MinIO client
kubectl run -i --tty --rm minio-cli --image=busybox --restart=Never -- sh
2. Download the MinIO client
/ # wget https://dl.min.io/client/mc/release/linux-amd64/mc
/ # chmod +x mc
3. Set the MinIO server endpoint in the MinIO client
Configure alias to access the MinIO server endpoint. Credentials of a user with admin access to the MinIO object storage service is required. To access the grafana-loki-minio console, please refer to this article.
/ # ./mc alias set grafana_loki_minio
http://grafana-loki-minio-hl.kommander.svc.cluster.local:9000
Enter Access Key: <USERNAME>
Enter Secret Key: <PASSWORD>
Added `grafana_loki_minio` successfully.
4. Enable object life-cycle configuration on the Loki bucket to setup automatic deletion of objects after a specified number of days or a specified date
./mc ilm import grafana_loki_minio/loki <<EOF
{
"Rules": [
{
"Expiration": {
"Days": 60
},
"ID": "Delete objects from the Loki bucket older than 60 days",
"Status": "Enabled"
}
]
}
EOF
Lifecycle configuration imported successfully to `grafana_loki_minio/loki`.
5. List life-cycle rules to confirm
./mc ilm ls grafana_loki_minio/loki/fake
ID | Type | Version | Tier | Date/Days | Prefix | Enabled | Tags
----------------------|------------|--------------|------------------|--------------|----------------|------------|------------------------
Delete objects from the Loki bucket older than 60 days | Expiry | Current | | 60 day(s) | | ✔ |
----------------------|------------|--------------|------------------|--------------|----------------|------------|------------------------
6. Removing specific life-cycle rules
./mc ilm rm --id "Delete objects from the Loki bucket older than 60 days" grafana_loki_minio/loki
6.1 Forcefully removing all the rules
./mc ilm rm --all --force grafana_loki_minio/loki/fake