You may encounter a situation in which certain "konvoy up" operations are getting stuck on the "check nodes" stage, or the "konvoy check nodes" command itself is locking up and not completing.
A common cause for this is if you have nodes that have disk mounts that are not responding to "sg_inq" queries and are unable to return disk information.
To check for this symptom, ssh into the node that the command is getting stuck on and check the "ps" output to see if there are instances of this process that are stuck. The output might resemble the following:
$ ps auxf | grep sg_inq
296:root 29740 0.0 0.0 6740 580 ? D Oct07 0:01 /usr/bin/sg_inq /dev/nvme6n1
320:root 3185 0.0 0.0 6740 584 ? D Oct27 0:00 /usr/bin/sg_inq /dev/nvme6n1
364:root 7698 0.0 0.0 6740 580 ? D Dec02 0:00 /usr/bin/sg_inq /dev/nvme6n1
365:root 24617 0.0 0.0 6740 580 ? D Dec02 0:00 /usr/bin/sg_inq /dev/nvme6n1
366:root 32036 0.0 0.0 6740 584 ? D Dec02 0:00 /usr/bin/sg_inq /dev/nvme6n1
367:root 30563 0.0 0.0 6740 584 ? D Dec02 0:00 /usr/bin/sg_inq /dev/nvme6n1
368:root 11429 0.0 0.0 6740 580 ? D Dec02 0:00 /usr/bin/sg_inq /dev/nvme6n1
369:root 22057 0.0 0.0 6740 580 ? D Dec03 0:00 /usr/bin/sg_inq /dev/nvme6n1
370:root 30800 0.0 0.0 6740 584 ? D Dec03 0:00 /usr/bin/sg_inq /dev/nvme6n1
375:root 14654 0.0 0.0 6740 584 ? D Dec03 0:00 /usr/bin/sg_inq /dev/nvme6n1
376:root 31012 0.0 0.0 6740 676 ? D 17:44 0:00 /usr/bin/sg_inq /dev/nvme6n1
In this example, there is an indication that there are several sg_inq processes that got stuck trying to gather information on the "nvme6n1" mount.
If you notice this symptom, you would need to investigate the relevant disk mount, determine what it is being used for, and if there are any problems with it.
If this mount isn't being used for anything important, the simplest resolution may be to remove the mount. Otherwise, it may be necessary to work with your operating system vendor or cloud provider to resolve the problems with this mount.
If you're experiencing similar symptoms but are unsure about whether you're encountering this exact issue, please feel free to submit a case with our support team and we'll be happy to investigate:
https://support.d2iq.com/hc/en-us/articles/4408231804564