feat(operator): strict ordering unit test passing

Author: ayuskauskas

chart/templates/skyhook-crd.yaml

@@ -4,6 +4,8 @@ metadata:
   name: skyhooks.skyhook.nvidia.com
   annotations:
     controller-gen.kubebuilder.io/version: v0.15.0
+    skyhook.nvidia.com/pause: "false"
+    skyhook.nvidia.com/disable: "false"
   labels:
   {{- include "chart.labels" . | nindent 4 }}
 spec:
@@ -426,12 +428,6 @@ spec:
                   type: object
                 description: Packages are the DAG of packages to be applied to nodes.
                 type: object
-              pause:
-                default: false
-                description: |-
-                  Pause halt the operator from proceeding. THIS is for admin use to stop skyhook if there is an issue or
-                  concert without needing to delete to ad in discovery of the issue.
-                type: boolean
               podNonInterruptLabels:
                 description: PodNonInterruptLabels are a set of labels we want to
                   monitor pods for whether they Interruptible
@@ -484,6 +480,11 @@ spec:
                 description: This skyhook is required to have been completed before
                   any workloads can start
                 type: boolean
+              priority:
+                description: Priority determines the order in which skyhooks are applied. Lower values are applied first.
+                type: integer
+                minimum: 0
+                default: 0
               serial:
                 default: false
                 description: Serial tells skyhook if it allowed to run in parallel or

k8s-tests/chainsaw/skyhook/cleanup-pods/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: cleanup-pods
 spec:
-  concurrent: true 
+  concurrent: false 
   description: |
     This test runs a simple skyhook with dependsOn. We wait tell completed, then trigger update to force config cycle on package B. Once config
     is complete, we update again to make the package error, and at the same clear out the node annotation to trigger cleanup.

k8s-tests/chainsaw/skyhook/config-skyhook/chainsaw-test.yaml

@@ -31,7 +31,7 @@ spec:
     accordingly but the package with the config changes may be skipeed and hang, and this asserts that it doesn't hang in that condition. Then once that completes the same two
     packages have the key with a config interrupt changed and it's asserted that the config, interrupt, and post-interrupt runs for both those packages. Then once that completes 
     again it does one more update on a key for the same two packages which doesn't have a config interrupt defined and makes sure that the config steps are ran for that.
-  concurrent: true
+  concurrent: false
   timeouts:
     assert: 360s
   catch: ## if errors, print the most important info

k8s-tests/chainsaw/skyhook/depends-on/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: depends-on
 spec:
-  concurrent: true 
+  concurrent: false 
   description: |
     Test makes sure depends-on works as expected. c depends on a, and b. Make sure a and b complete before c starts.
   timeouts:

k8s-tests/chainsaw/skyhook/failure-skyhook/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: failure-skyhook
 spec:
-  concurrent: true 
+  concurrent: false 
   timeouts:
     assert: 240s
   catch: ## if errors, print the most important info

k8s-tests/chainsaw/skyhook/pod-finalizer/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: pod-finalization
 spec:
-  concurrent: true
+  concurrent: false
   timeouts:
     assert: 180s
   steps:

k8s-tests/chainsaw/skyhook/simple-skyhook/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: simple-skyhook
 spec:
-  concurrent: true 
+  concurrent: false 
   # skip: false ##  this test doesn't seem to be useful, just slows things down, leaving it for, should delete at some point if still skipped
   timeouts:
     assert: 240s

k8s-tests/chainsaw/skyhook/simple-update-skyhook/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: simple-update-skyhook
 spec:
-  concurrent: true
+  concurrent: false
   timeouts:
     assert: 240s
   catch: ## if errors, print the most important info

k8s-tests/chainsaw/skyhook/skyhook-upgrade/chainsaw-test.yaml

@@ -24,7 +24,7 @@ kind: Test
 metadata:
   name: skyhook-upgrade
 spec:
-  concurrent: true 
+  concurrent: false 
   skip: true ## skipping because this test current requires manual updating of the version
   description: |
     This test is skipped because it is because its not automated to change versions while its running.

k8s-tests/chainsaw/skyhook/uninstall-upgrade-skyhook/chainsaw-test.yaml

@@ -31,7 +31,7 @@ spec:
     asserted that it runs in upgrade before finishing as normal. Then finally we assert that the uninstall still works when every package is removed from the SCR by removing the last package
     and then making sure that it's uninstalled successfully.
     Additionally we are assert pod resource requests and limits are set correctly from defaults.
-  concurrent: true
+  concurrent: false
   timeouts:
     assert: 420s
   catch: ## if errors, print the most important info