There is a known issue in DKP 2.4.0 that prevents the Velero CLI from properly communicating with the DKP cluster which prevents users from downloading backups via the Velero CLI with the following message:
velero backup download test -n kommander
An error occurred: Get "http://rook-ceph-rgw-dkp-object-store.kommander.svc:80/dkp-velero/backups/test/test.tar.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=668U8LPANA2LCJXX2VG9%2F20230327%2Fdkp-object-store%2Fs3%2Faws4_request&X-Amz-Date=20230327T205306Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=8f19a475c1332816b4589f81699e943979475f12c825f5ecb4359481ebd203f0": dial tcp: lookup rook-ceph-rgw-dkp-object-store.kommander.svc on 10.4.6.1:53: no such host
This issue is fixed in DKP 2.5.0 and will be fixed in a later release of DKP 2.4.X, but if you need to download a backup on DKP 2.4.0 you can use the following method:
Solution
1. In one terminal session, create a port forward to the rook-ceph service. This terminal session will be used to proxy traffic from rook-ceph to your local computer. Make sure this terminal session has the KUBECONFIG environment variable defined to point to your cluster.
kubectl port-forward service/rook-ceph-rgw-dkp-object-store -n kommander 8001:80
2. In a separate terminal session that also has the KUBECONFIG variable set correctly, install the aws cli. Note that your cluster does not have to be installed in AWS to use this tool, we are simply using it to access the AWS S3 compatible storage that rook-ceph provides.
2. Extract the dkp-velero bucket credentials:
export AWS_ACCESS_KEY_ID=`kubectl get secret -n kommander dkp-velero -o 'jsonpath={.data.AWS_ACCESS_KEY_ID}' | base64 --decode;echo`
export AWS_SECRET_ACCESS_KEY=`kubectl get secret -n kommander dkp-velero -o 'jsonpath={.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode;echo`
4. List all objects in the bucket:
aws s3 ls --recursive s3://dkp-velero/ --endpoint-url http://localhost:8001
2023-03-27 15:52:20 29 backups/test/test-csi-volumesnapshotclasses.json.gz
2023-03-27 15:52:20 29 backups/test/test-csi-volumesnapshotcontents.json.gz
2023-03-27 15:52:20 29 backups/test/test-csi-volumesnapshots.json.gz
2023-03-27 15:52:19 105919 backups/test/test-logs.gz
2023-03-27 15:52:20 29 backups/test/test-podvolumebackups.json.gz
2023-03-27 15:52:20 26631 backups/test/test-resource-list.json.gz
2023-03-27 15:52:20 29 backups/test/test-volumesnapshots.json.gz
2023-03-27 15:52:20 9057502 backups/test/test.tar.gz
2023-03-27 15:52:19 2187 backups/test/velero-backup.json
6. Download the files for the specific backup you want to use one at a time:
aws s3 --endpoint-url http://localhost:8001 cp s3://dkp-velero/backups/test/test-csi-volumesnapshotclasses.json.gz .
Note that the period at the end of this command is telling the aws cli to download the file to the current directory with the same name as the remote file.