Skip to main content

Getting various Konvoy and DKP cluster details guide

Alex Christopher Ramos
  • Updated

Motivation
The intent of this guide is to help provide a quick reference to finding various cluster details with the use of a CLI, config file, or a generated support bundle. We have divided the instructions per major version and per tool or resource to use.

Note: Many examples utilize Docker and are intended to be executed on MacOS or Linux.

 

Click on the links below to jump into the specific section of the page.

DKP/Konvoy Version 

Infrastructure 

OS and OS Version 

GPU Enabled 

GPU utilized 

Air-Gapped 

FIPS compliant build utilized 

Is this a management cluster 

How many clusters is it managing 

Any Data Services being run 

Kaptain running 

Kaptain Version (If Installed) 

 

DKP/Konvoy Version

DKP 1.x

  • With CLI
./konvoy version
  • With cluster.yaml
cat cluster.yaml | docker run --rm -i karlkfi/yq -r '.spec.version'
  • With bundle
cd <bundle-directory>
cat ./*/konvoy-marker.yaml | docker run --rm -i karlkfi/yq -r '.spec.version'

Alternatively, view the Konvoy marker file and note the top-level version field:

cd <bundle-directory>
cat <bundle-directory>/<node-directory>/konvoy-marker.yaml


DKP 2.x

  • With CLI
./dkp version
  • With bundle
cd <bundle-directory>
cat ./cluster-resources/custom-resources/(awscluster|azurecluster|preprovisionedcluster|vspherecluster)*/*.yaml | docker run --rm -i karlkfi/yq -r '.[].spec.additionalTags | "Cluster: " + ."konvoy.d2iq.io/cluster-name" + " (" + ."konvoy.d2iq.io/version" + ")"'

Infrastructure

DKP 1.x

  • With CLI
kubectl get no -o jsonpath='{range .items[*]}{"Node: "}{.metadata.name}{" -- Provider ID: --> "}{.spec.providerID}{"\n"}'

Blank = preprovisioned/on-prem

  • With Cluster.yaml
cat cluster.yaml | docker run --rm -i karlkfi/yq -r 'select(.kind == "ClusterProvisioner") | .spec.provider'

OR

cat cluster.yaml|grep provider

Blank = preprovisioned

  • With Bundle
cd <bundle-directory>
cat ./*/konvoy-marker.yaml | docker run --rm -i karlkfi/yq -r 'select(.kind == "ClusterConfiguration") | .spec.provider'

Blank = preprovisioned

DKP 2.x

  • With CLI
kubectl get clusters -n default -o=jsonpath='{.items[*].spec.infrastructureRef.kind}{"\n"}'
  • With Bundle
cd <bundle-directory>
cat ./cluster-resources/custom-resources/clusters.cluster.x-k8s.io/*.yaml | docker run --rm -i karlkfi/yq -r '.[] | "Cluster: " + .metadata.name + " -- Provider --> " + .spec.infrastructureRef.kind'

OS and OS Version

DKP 1.x

  • With CLI
kubectl get nodes -owide
  • With bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/nodes.yaml | docker run --rm -i karlkfi/yq -r '.items[] | "Node: " + .metadata.name + " -- OS Version --> " + .status.nodeInfo.osImage'

DKP 2.x

  • With CLI
kubectl get nodes -owide
  • With bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/nodes.yaml | docker run --rm -i karlkfi/yq -r '.items[] | "Node: " + .metadata.name + " -- OS Version --> " + .status.nodeInfo.osImage'

GPU Enabled on the cluster

DKP 1.x

  • With CLI
kubectl get clusteraddon nvidia
  • With cluster.yaml
cat cluster.yaml | docker run --rm -i karlkfi/yq -r 'select(.kind == "ClusterConfiguration") | .spec.addons[].addonsList[] | select(.name == "nvidia")'
  • With Bundle
cd <bundle-directory>
cat <bundle-directory>/cluster-data/api-resources/clusteraddons.kubeaddons.mesosphere.io.yaml | docker run --rm -i karlkfi/yq -r '.items[].metadata | select(.name == "nvidia")'


DKP 2.x

  • With CLI
kubectl get appdeployment nvidia -n kommander

If nvidia is listed, GPU feature is enabled on the cluster

  • With bundle
cd <bundle-directory>
cat ./custom-resources/appdeployments.apps.kommander.d2iq.io/kommander.json | docker run --rm -i stedolan/jq -r '.' | grep nvidia

An entry for kind AppDeployment named nvidia

GPU Utilized

If a cluster is utilizing nvidia GPU, a daemonset named nvidia-kubeaddons-nvidia-device-plugin in namespace kube-system would be present. The plugin takes care of various GPU-related tasks, like exposing the number of GPU's available on the node, which can be confirmed by getting the details of the node.

DKP 1.x

  • With CLI
kubectl describe node <nameOfNode>

The value in allocatable.nvidia.com/gpu represents the number of GPU that is usable

  • With Bundle

To confirm if nvidia-device-plugin is deployed

cd <bundle-directory>
cat ./cluster-data/api-resources/daemonsets.apps.yaml | docker run --rm -i karlkfi/yq -r '.items[].metadata | select(.name == "nvidia-kubeaddons-nvidia-device-plugin")'

To confirm if the node is nvidia GPU capable

cd <bundle-directory>
cat ./cluster-data/api-resources/nodes.yaml | docker run --rm -i karlkfi/yq -r '.items[] | select(.status.allocatable | has("nvidia.com/gpu")) | "Node: " + .metadata.name + " has nvidia.com/gpu key"'

DKP 2.x

  • With CLI
kubectl describe node <nameOfNode>

The value in allocatable.nvidia.com/gpu represents the number of GPU that is usable

  • With Bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/nodes.yaml | docker run --rm -i karlkfi/yq -r '.items[] | select(.status.allocatable | has("nvidia.com/gpu")) | "Node: " + .metadata.name + " has nvidia.com/gpu key"'

Air-Gapped

DKP 1.x

  • With Cluster.yaml
cat cluster.yaml|grep configRepository

Airgap returns a directory path, networked returns a repo URL

  • With Bundle
cd <bundle-directory>
cat ./*/konvoy-marker.yaml | docker run --rm -i karlkfi/yq -r 'select(.kind == "ClusterConfiguration") | .spec.osPackages.enableAdditionalRepositories'

Airgap will have a value false

DKP 2.x

  • With Bundle/Install.yaml

Kommander's --installer-config yaml file will indicate

apiVersion: config.kommander.mesosphere.io/v1alpha1
kind: Installation
airgapped:
  enabled: true


FIPS compliant build utilized

DKP 1.x

  • With CLI
./konvoy check fips
  • With Bundle
cd <bundle-directory>
cat ./*/sysctl.txt | grep crypto.fips_enabled

Will return sysctl.txt file of all nodes, grepping for crypto.fips_enabled
1 = true and 0 = false


DKP 2.x

  • With CLI
./dkp check cluster fips --signature-file=manifest.asc --signature-configmap=signatures --output-configmap=output

Detailed guide here.

  • With bundle
cd <bundle-directory>
cat ./cluster-info/cluster_version.json | docker run --rm -i stedolan/jq -r '.info.gitVersion'

The value should include fips, such as "v1.22.8+fips.0"


Is this a management cluster?

DKP 1.x

  • Kommander management cluster
    • With CLI
kubectl get kommanderclusters -A

When executed on a Kommander host cluster managing other clusters this will display the list of clusters attached/joined

    • With Bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/kommanderclusters.kommander.mesosphere.io.yaml | docker run --rm -i karlkfi/yq -r '.items[] | .metadata.name'

The items list are the clusters attached on this Kommander host. In contrast, if the file does not contain any list, this would mean that there are no clusters attached

DKP 2.x

  • Clusterapi management cluster
    • With CLI
kubectl get clusters -A

When executed on a management cluster, it would return the list of clusters it is managing.

In contrast, if executed on a non-management cluster, it would return an
error: the server doesn't have a resource type "clusters"
If the result of the cluster list contains the same cluster where you executed the command, it means the cluster is self-managed

    • With bundle
cd <bundle-directory>
cat ./cluster-resources/custom-resources/clusters.cluster.x-k8s.io/default.yaml | docker run --rm -i karlkfi/yq -r '.[] | .metadata.name'

This will return the list of cluster names being managed by clusterapi

In contrast, if the cluster is not a management cluster, this CRD would not exist

  • Kommander management cluster
    • With CLI
dkp get clusters -A

OR

dkp get clusters --workspace <nameofworkspace>

When executed this would list the clusters attached in commander. The cluster named “host-cluster” is where this instance of Kommander is hosted

    • With Bundle
cd <bundle-directory>
ls ./cluster-resources/custom-resources/kommanderclusters.kommander.mesosphere.io

Each file returned represents a workspace/cluster that is attached. In contrast, if the kommander cluster is not managing other clusters, this would only contain kommander.yaml which is the kommander host

 

How many clusters is it managing?

DKP 1.x

  • Kommander management cluster
    • With CLI
kubectl get kommanderclusters -A

When executed on a kommander host cluster managing other clusters, this will display the list of clusters attached/joined

    • With Bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/kommanderclusters.kommander.mesosphere.io.yaml | docker run --rm -i karlkfi/yq -r '.items[] | .metadata.name' | wc -l

This will return the count of clusters attached.


DKP 2.x

  • Clusterapi management cluster
    • With CLI
kubectl get clusters -A

If executed on a management cluster, it would return the list of clusters it is managing

    • With Bundle
cd <bundle-directory>
cat ./cluster-resources/custom-resources/clusters.cluster.x-k8s.io/default.yaml | docker run --rm -i karlkfi/yq -r '.[] | .metadata.name' | wc -l

This would contain the list of clusters being managed by clusterapi, including its host cluster

  • Kommander management cluster
    • With CLI
dkp get clusters -A

When executed this would list the clusters attached in kommander

    • With Bundle
cd <bundle-directory>
ls ./cluster-resources/custom-resources/kommanderclusters.kommander.mesosphere.io | wc -l

This will return the count of clusters attached in Kommander.


Any Data Services being run?

Data Services is defined as DKP catalog apps, like Kaptain, Zookeeper, Kafka.
 

DKP 1.x

  • With CLI
kubectl get addons,clusteraddons -A

This would list addons deployed, such as Zookeeper, Kafka, etc.

  • With Bundle/Cluster.yaml

Cluster.yaml would include addons from kubeaddons-enterprise, example

- configRepository: https://github.com/mesosphere/kubeaddons-enterprise
  configVersion: v0.0.3
  addonsList:
  - name: zookeeper
    enabled: true

From a support-bundle, this items would be listed in

cd <bundle-directory>
cat ./cluster-data/api-resources/addons.kubeaddons.mesosphere.io.yaml | docker run --rm -i karlkfi/yq -r '.items[] | .metadata.name'

This will return the name list of kind addons. Check if zk, or Kafka or Kaptain is on the list.

DKP 2.x

  • With CLI
kubectl get apps -A

OR

kubectl get hr -A
  • With Bundle
cd <bundle-directory>
cat ./cluster-resources/custom-resources/appdeployments.apps.kommander.d2iq.io/*.yaml | docker run --rm -i karlkfi/yq -r '.[] | .metadata.name'

This will list all of the AppDeployment from all the workspaces.

 

Kaptain Installed?

DKP 1.x

  • With CLI
kubectl get clusteraddon kaptain
  • With Bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/clusteraddons.kubeaddons.mesosphere.io.yaml | docker run --rm -i karlkfi/yq -r '.items[].metadata | select(.name == "kaptain")'

DKP 2.x

  • With CLI
kubectl get appdeployments -A | grep kaptain
  • With Bundle
cd <bundle-directory>
cat ./custom-resources/appdeployments.apps.kommander.d2iq.io/*.json | docker run --rm -i stedolan/jq -r '.' | grep kaptain

 

Kaptain Version (If Installed)

Kaptain versions are formatted as <kubeflow-version>-<kaptain-version>
So Kaptain 1.3 would be 1.4.0-1.3.0
while Kaptain 1.2 would be 1.3.0-1.2.0

DKP 1.x

  • With CLI
kubectl get operatorversions -A
  • With Bundle
cd <bundle-directory>
cat ./cluster-data/api-resources/instances.kudo.dev.yaml | docker run --rm -i karlkfi/yq -r '.items[] | .spec.operatorVersion.name' | grep kubeflow

OR

cd <bundle-directory>
cat ./cluster-data/api-resources/operatorversions.kudo.dev.yaml | docker run --rm -i karlkfi/yq -r '.items[] | .metadata.name'

Here you would see different component versions, like centraldashboard, admission-webhook.

 

DKP 2.x

  • With CLI
kubectl get operatorversions -A
  • With Bundle
cd <bundle-directory>
cat ./custom-resources/instances.kudo.dev/kubeflow.json | docker run --rm -i stedolan/jq -r '.[] | .spec.operatorVersion.name'

OR

cd <bundle-directory>
cat ./custom-resources/operatorversions.kudo.dev/kubeflow.json | docker run --rm -i stedolan/jq -r '.[] | .metadata.name'

Here you would see different component versions, like centraldashboard, admission-webhook.