How to resolve upgrade/install failure with Konvoy 1.5 on auto-provisioning step

Overview/Background

Konvoy 1.5 introduced a new feature for Auto-scaling clusters based on demand. This feature is turned on by default. For auto-scaling capabilities to be configured on an airgapped cluster some additional configuration is required. Without this additional configuration, a fresh install or upgrade to konvoy 1.5 would fail in an airgapped environment with an error similar to following:

TASK [auto-provisioning : upgrade the auto-provisioning module] *********************************************************************************
FAILED - RETRYING: upgrade the auto-provisioning module (3 retries left).
FAILED - RETRYING: upgrade the auto-provisioning module (2 retries left).
FAILED - RETRYING: upgrade the auto-provisioning module (1 retries left).
fatal: [10.0.10.135 -> localhost]: FAILED! => {
"attempts": 3,
"changed": true,
"cmd": [
"helmv3",
"--kubeconfig",
"/tmp/konvoy/konvoy_v1.5.0/admin.conf",
"upgrade",
"auto-provisioning",
"/opt/konvoy/chart/auto-provisioning",
"--version=v0.1.1",
"--install",
"--atomic",
"-f",
"/tmp/konvoy/konvoy_v1.5.0/default-auto-provisioning-values.yaml",
"-f",
"/tmp/konvoy/konvoy_v1.5.0/auto-provisioning-overrides.yaml",
"-n",
"konvoy"
],
"delta": "0:05:00.922296",
"end": "2020-07-27 08:54:38.270589",
"rc": 1,
"start": "2020-07-27 08:49:37.765680"
}
STDOUT:
Release "auto-provisioning" does not exist. Installing it now.
STDERR:
coalesce.go:196: warning: cannot overwrite table with non table for env (map[])
Error: release auto-provisioning failed, and has been uninstalled due to atomic being set: failed pre-install: timed out waiting for the condition
MSG:
non-zero return code

Solution

There are two options to resolve the issue with auto-scaling errors in an Air-gapped install. You could choose to not use auto-scaling capabilities or perform extra configuration steps to make it work in airgapped environment. If you are not planning to use auto-scaling, deploying without auto-scaling would be the preferred approach.

Skip auto-scaling

You can skip auto-scaling by using the flag --without-auto-provisioning along with konvoy up or konvoy deploy like so :

konvoy up --without-auto-provisioning 

Note that if auto-scaling was previously installed and you would like to remove it, you can do so by executing the following command from your Konvoy deploy directory, ensuring that you substitute  for your Konvoy version (e.g., v1.5.0):

docker run -v $(pwd):/opt/konvoy -e KUBECONFIG=admin.conf -w /opt/konvoy --entrypoint /usr/local/bin/helmv3 mesosphere/konvoy: uninstall auto-provisioning -n konvoy 

Any subsequent runs of `konvoy up` or `konvoy deploy` must include the `--without-auto-provisioning` flag.

Configure auto-scaling for airgapped environment

You can configure auto-scaling for your airgapped environment by adding some extra configuration in your cluster.yaml as documented in auto-scaling documentation.