chore: add upgrade script and docs for it

This change adds an upgrade script which will follow the node upgrade process
allowing operators to run a more controlled upgrade experience.

Signed-off-by: Kevin Carter <[email protected]>

Author: cloudnull

docs/k8s-kubespray-upgrade.md

@@ -28,38 +28,14 @@ Once the group variables are set, you can proceed with the upgrade execution.
 
 Running an upgrade with Kubespray is fairly simple and executed via `ansible-playbook`.
 
-Before running the playbook be sure to source your environment variables.
+* We recoommend running the upgrade from a control node that is not part of the cluster being upgraded. This helps to ensure that the upgrade process is not competing for resources with the cluster being upgraded.
 
-``` shell
-source /opt/genestack/scripts/genestack.rc
-```
-
-Change to the `kubespary` directory.
-
-``` shell
-cd /opt/genestack/submodules/kubespray
-```
-
-!!! note
-
-    When running an upgrade be sure to set the `kube_version` variable to the desired version number.
-
-Now run the upgrade.
-
-``` shell
-ansible-playbook upgrade-cluster.yml -e kube_version=${VERSION_NUMBER}
-```
-
-!!! note
-
-    While the basic command could work, be sure to include any and all flags needed for your environment before running the upgrade.
-
-### Running an unsafe upgrade
+* We also recommend running the upgrade within a `screen` or `tmux` session to help ensure that the upgrade process is not interrupted by network issues.
 
-When running an upgrade, it is possible to force the upgrade by running the cluster playbook with the `upgrade_cluster_setup` flag set to **true**. This option is a lot faster, though does introduce the possibility of service disruption during the upgrade operation.
+The script to run the upgrade can be found within the `scripts/kubespray-major-upgrade.sh` file; however, the process is fairly straightforward.
 
 ``` shell
-ansible-playbook cluster.yml -e upgrade_cluster_setup=true -e kube_version=${VERSION_NUMBER}
+--8<-- "scripts/kubespray-major-upgrade.sh"
 ```
 
 ### Upgrade Hangs

scripts/kubespray-major-upgrade.sh

@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+
+set -e
+
+function gitRepoVersion() {
+    # Returns the current git branch name, tag, or commit hash
+    echo "$(git symbolic-ref -q --short HEAD || git describe --tags --exact-match 2>/dev/null || git rev-parse HEAD)"
+}
+
+echo "This script will help you upgrade your Kubernetes cluster managed by Kubespray."
+
+# Ask the user for the target Kubernetes version
+read -p "Enter the target Kubernetes version number (e.g., 1.34.0): " VERSION_NUMBER
+
+# Confirm the version number with the user
+echo "You have entered Kubernetes version number: ${VERSION_NUMBER}"
+
+echo "Your current setup is as follows:"
+# Show checkout version of Genestack and the branch being used
+pushd /opt/genestack &>/dev/null
+    echo "[+] Current Genestack version: $(gitRepoVersion) (SHA:$(git rev-parse HEAD))"
+popd &>/dev/null
+
+pushd /opt/genestack/submodules/kubespray &>/dev/null
+    echo "[+] Current Kubespray version: $(gitRepoVersion) (SHA:$(git rev-parse HEAD))"
+popd &>/dev/null
+
+read -p "Is all of this correct? If yes type \`DOTHETHINGNOW\`: " CONFIRMATION
+
+if [[ "$CONFIRMATION" != "DOTHETHINGNOW" ]]; then
+    echo "Aborting. Please run the script again and enter the correct version number and confirmation."
+    exit 1
+fi
+
+set -v
+
+# Load Genestack environment variables
+. /opt/genestack/scripts/genestack.rc
+
+# Navigate to the Kubespray directory and perform the upgrade
+pushd /opt/genestack/submodules/kubespray &>/dev/null
+    echo "Gathering cluster facts"
+    ansible-playbook playbooks/facts.yml --become
+
+    echo "Upgrading cluster to Kubernetes version ${VERSION_NUMBER}"
+    ansible-playbook upgrade-cluster.yml --become -e kube_version${VERSION_NUMBER} --limit "kube_control_plane:etcd"
+
+    echo "Upgrading worker nodes to Kubernetes version ${VERSION_NUMBER}"
+    ansible-playbook upgrade-cluster.yml --become -e kube_version${VERSION_NUMBER} --limit "!kube_control_plane:!etcd"
+popd &>/dev/null
+
+echo "Kubernetes cluster upgrade to version ${VERSION_NUMBER} completed successfully."
+
+if command -v yq &>/dev/null; then
+    echo "Updating Kubernetes version in inventory files"
+    yq -i ".kube_version = \"${VERSION_NUMBER}\"" /etc/genestack/inventory/group_vars/k8s_cluster/k8s-cluster.yml
+else
+    echo "yq command not found. Please install yq to update inventory files."
+    echo "update the Kubernetes version in /etc/genestack/inventory/group_vars/k8s_cluster/k8s-cluster.yml manually to complete the upgrade."
+fi