Skip to content

Conversation

@miyadav
Copy link
Member

@miyadav miyadav commented Oct 13, 2025

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
    • Added end-to-end tests validating Verification and Access Policy (VAP) enforcement during Machine API migration on AWS.
    • Blocks updates to protected machine fields (providerID, taints, metadata) with explicit VAP errors.
    • Prevents changes to AWS providerSpec fields (instance type, AMI, availability zone, subnet, security groups, tags, capacity reservation).
    • Confirms VAP is not enforced when Machine API is authoritative.
    • Verifies protected label and annotation restrictions while allowing non-protected entries.
    • Adds setup, cleanup, and reusable helpers for field-level verification.

@coderabbitai
Copy link

coderabbitai bot commented Oct 13, 2025

Walkthrough

Adds an AWS-only Ginkgo end-to-end test that validates VAP enforcement for Machine API Migration (MAPI), covering spec/label/annotation/providerSpec update restrictions and a bypass when authoritativeAPI is set to MachineAPI.

Changes

Cohort / File(s) Summary
E2E Tests for VAP Enforcement (AWS)
e2e/machine_migration_vap_tests.go
New Ginkgo E2E test file. Implements tests that create a MAPI Machine (authoritativeAPI=CAPI) and corresponding CAPI machine; asserts that updates to spec.providerID, spec.taints, spec.metadata, protected machine.openshift.io/* labels/annotations, and several AWS providerSpec fields are prevented with specific VAP errors; verifies non-protected labels/annotations are allowed and VAP is bypassed when authoritativeAPI=MachineAPI. Adds helper functions: verifyUpdatePrevented, verifyUpdateAllowed, verifyAWSProviderSpecUpdatePrevented, verifyVAPNotAppliedForMachineAPIAuthority, updateAWSMachineProviderSpec, and getAWSProviderSpecFromMachine, plus setup/cleanup and authority reset logic.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant T as Ginkgo Test
  participant K as Kubernetes API
  participant V as VAP Admission

  Note over T: Setup (AWS, MAPI migration enabled)
  T->>K: Create MAPI Machine (authoritativeAPI=CAPI)
  K-->>T: Created
  T->>K: Create corresponding CAPI Machine
  K-->>T: Created

  rect rgb(220,235,255)
    Note right of V: Enforcement when authoritativeAPI=CAPI
    T->>K: Patch protected spec/labels/annotations/providerSpec
    K->>V: AdmissionReview
    V-->>K: Deny with VAP error
    K-->>T: Update rejected (assert)
  end

  rect rgb(235,255,235)
    Note right of V: Switch authority -> MachineAPI
    T->>K: Patch machine.authoritativeAPI -> MachineAPI
    K->>V: AdmissionReview
    V-->>K: Allow
    K-->>T: Update succeeds
  end

  Note over T: Cleanup and restore authoritativeAPI if changed
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Inspect AWS providerSpec parsing/serialization in getAWSProviderSpecFromMachine / updateAWSMachineProviderSpec.
  • Verify VAP error message assertions in verifyUpdatePrevented / verifyAWSProviderSpecUpdatePrevented.
  • Review setup/cleanup and authoritativeAPI reset for race conditions or resource leaks.

Poem

I hop through YAML with a curious nose,
I twitch at taints where the policy grows.
CAPI says "halt" and the updates refrain,
Flip to MachineAPI — the rules wane.
The rabbit applauds as tests bound again. 🐇

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Title check ❓ Inconclusive The pull request title 'OCPCLOUD-2566: tests with VAP msgs' is vague and does not clearly convey the specific purpose of the changes. While it includes a Jira reference and mentions VAP, the term 'tests with VAP msgs' is generic and non-descriptive. It does not clearly communicate that the changeset adds end-to-end tests for Verification and Access Policy enforcement during Machine API migration on AWS. A teammate scanning PR history would struggle to understand what specific functionality or behavior is being tested. Consider revising the title to be more specific about the test coverage, such as 'Add e2e tests for VAP enforcement during Machine API migration on AWS' or 'Add Machine migration VAP enforcement tests for AWS'. This would clearly convey that the PR adds end-to-end tests specifically validating VAP behavior during machine migration, making the purpose immediately clear to reviewers.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 354042a and ebde40b.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (6)
e2e/machine_migration_vap_tests.go (6)

3-36: LGTM! Imports and constants are well-defined.

The imports correctly include encoding/json for RawExtension marshaling, Komega for test utilities, and all necessary OpenShift/Kubernetes types. The test constants and VAP error messages accurately reflect the actual VAP policy requirements.


38-78: LGTM! Test setup properly handles initialization and cleanup.

The suite correctly gates on AWS platform and MachineAPIMigration feature. The wait for status.authoritativeAPI to equal ClusterAPI (lines 62-67) addresses the previous review feedback and prevents race conditions where tests might run before VAP enforcement conditions are met.


80-210: LGTM! Comprehensive test coverage of VAP enforcement rules.

The test contexts thoroughly cover all VAP enforcement scenarios:

  • Spec field restrictions for providerID, taints, and metadata
  • Protected label/annotation restrictions with both enforcement and bypass tests
  • AWS providerSpec field restrictions for all critical fields
  • VAP bypass verification when authoritativeAPI is MachineAPI

The test organization is clear and addresses all items from previous reviews, including the missing protected label and providerSpec tests.


214-229: LGTM! Helper functions correctly use Komega pattern.

The helpers properly leverage Komega's Update function with Eventually/Should patterns, addressing the previous review feedback to use Komega instead of direct client calls. The updateFunc func() signature with closure capture is a valid pattern that works correctly with Komega's object fetching and update flow.


231-266: LGTM! AWS providerSpec helpers correctly use JSON marshaling.

These helpers properly handle AWS providerSpec updates using encoding/json for marshaling/unmarshaling (addressing the previous YAML-related issue). The verifyAWSProviderSpecUpdatePrevented helper is now integrated into the test suite and follows the same Komega pattern as other verification helpers.


268-295: LGTM! VAP bypass verification includes proper status wait.

The function correctly verifies that VAP enforcement doesn't apply when authoritativeAPI is MachineAPI. The wait for status propagation (lines 275-281) addresses the previous review feedback and mirrors the pattern used in the main test suite, preventing potential race conditions during test execution.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.5.0)

Error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions
The command is terminated due to an error: can't load config: unsupported version of the configuration: "" See https://golangci-lint.run/docs/product/migration-guide for migration instructions


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (7)
e2e/machine_migration_vap_tests.go (7)

43-43: Avoid var _ = Describe inside a Describe; just call Describe/Context

Inside a Describe, prefer Describe(...) or Context(...) (no var _ =). This is cleaner and avoids confusing local var decls.

-	var _ = Describe("VAP: machine-api-machine-vap enforcement", Ordered, func() {
+	Describe("VAP: machine-api-machine-vap enforcement", Ordered, func() {

53-54: Make CAPI mirror retrieval resilient

GetMachine(...) may race the mirror creation. Wrap in Eventually (or a helper that waits) to avoid flaky NotFound.

Example:

-			testCAPIMachine = capiframework.GetMachine(cl, testMachineName, capiframework.CAPINamespace)
+			Eventually(func() (*clusterv1.Machine, error) {
+				m := capiframework.GetMachine(cl, testMachineName, capiframework.CAPINamespace)
+				if m == nil {
+					return nil, fmt.Errorf("CAPI machine not found yet")
+				}
+				return m, nil
+			}, capiframework.WaitMedium, capiframework.RetryMedium).Should(Not(BeNil()))
+			testCAPIMachine = capiframework.GetMachine(cl, testMachineName, capiframework.CAPINamespace)

Also confirm capiframework.GetMachine already waits; if so, this is unnecessary. Please verify.


55-63: Guard cleanup against nils

If mirror creation fails, testCAPIMachine can be nil. Ensure cleanupMachineResources tolerates nils or guard before passing.


239-256: Unused helpers; either exercise them or remove

verifyNonProtectedLabelUpdateAllowed and verifyNonProtectedAnnotationUpdateAllowed aren’t called. Keep tests lean: add small Its using them or drop to avoid dead code.

Also applies to: 297-314


279-295: Deletion test must ensure the annotation exists first

If the protected annotation isn’t present, deleting it is a no-op and may not trigger VAP. Seed it first, then attempt deletion in a separate update.

 func verifyProtectedAnnotationDeletionPrevented(machine *mapiv1beta1.Machine, annotationKey, expectedError string) {
   By(fmt.Sprintf("Verifying that deleting protected annotation %s is prevented by VAP", annotationKey))
 
+  // Ensure the annotation exists
+  Eventually(func() error {
+    if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), machine); err != nil {
+      return err
+    }
+    if machine.Annotations == nil {
+      machine.Annotations = map[string]string{}
+    }
+    if _, ok := machine.Annotations[annotationKey]; !ok {
+      machine.Annotations[annotationKey] = "seed"
+    }
+    return cl.Update(ctx, machine)
+  }, capiframework.WaitMedium, capiframework.RetryMedium).Should(Succeed())
+
   Eventually(func() error {
     if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), machine); err != nil {
       return err
     }
@@
   }, capiframework.WaitMedium, capiframework.RetryMedium).Should(
     MatchError(ContainSubstring(expectedError)),
     "Expected protected annotation deletion to be blocked by VAP")
 }

316-340: Add a test that VAP is not applied when authoritativeAPI=MachineAPI

You wrote verifyVAPNotAppliedForMachineAPIAuthority() but never call it. Add an It under its own Context to prevent regressions.

Example:

@@
-			Context("spec field restrictions", func() {
+			Context("spec field restrictions", func() {
@@
 			})
+			Context("when authoritativeAPI is MachineAPI", func() {
+				It("does not apply VAP to spec updates", func() {
+					verifyVAPNotAppliedForMachineAPIAuthority()
+				})
+			})

Also applies to: 66-86


25-30: Be cautious with exact error text assertions

Hard-coding full messages is brittle across policy/message tweaks. You already use ContainSubstring; consider matching a stable fragment (e.g., “modify spec.authoritativeAPI” / “authoritativeAPI is set to Cluster API”) or a regex to reduce churn.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 9704b30 and cb03d42.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (2)
e2e/machine_migration_vap_tests.go (2)

32-41: AWS + feature-gate guards look good

Platform guard and MachineAPIMigration gate are correct and reduce flakiness.


66-86: Spec-field restriction assertions are solid

ProviderID, taints, and spec.metadata checks are clear and use Eventually with substring match on VAP error. Nice.

Optional: After each denial, assert the field remains unchanged to prove no mutation leaked through.

Also applies to: 106-128, 130-171

@sunzhaohua2
Copy link
Contributor

Hi Milind, thanks for helping to check this. I found Theo has some comments in pr #355, can you help to take a look? especially use Komega instead of cl.Update, cl.Get. verifyAnnotationModificationPrevented becomes a more generic call to verifyUpdatePrevented

@miyadav
Copy link
Member Author

miyadav commented Oct 14, 2025

Just to add , sometimes , machine status is going to migrating and then the changes like annotations are not blocked , but reconciled so functionally it is working , but we won't get test for it to pass, as it is not getting the error message we expect.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between cb03d42 and 59c4dc8.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (2)
e2e/machine_migration_vap_tests.go (2)

220-248: LGTM! Good example of Komega usage.

This helper correctly uses komega.Update and komega.Object, demonstrating the pattern that should be followed in other helper functions.


345-363: LGTM! Correct Komega usage.

This helper correctly uses komega.Update and demonstrates proper cleanup with mapiframework.DeleteMachines.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
e2e/machine_migration_vap_tests.go (1)

169-200: Refactor helper to use Komega (per earlier feedback).

Prior review already asked for Komega over raw cl.Get/cl.Update. Please align verifyUpdatePrevented/verifyUpdateAllowed with the project pattern:

-func verifyUpdatePrevented(machine *mapiv1beta1.Machine, updateFunc func(*mapiv1beta1.Machine), expectedError string) {
+func verifyUpdatePrevented(machine *mapiv1beta1.Machine, updateFunc func(), expectedError string) {
@@
-	Eventually(func() error {
-		freshMachine := &mapiv1beta1.Machine{}
-		if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), freshMachine); err != nil {
-			return err
-		}
-		updateFunc(freshMachine)
-		return cl.Update(ctx, freshMachine)
-	}, capiframework.WaitMedium, capiframework.RetryMedium).Should(
+	Eventually(komega.Update(machine, updateFunc), capiframework.WaitMedium, capiframework.RetryMedium).Should(

Apply the same pattern to verifyUpdateAllowed and update call sites to pass closures without the machine argument.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 59c4dc8 and 0e4e698.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)

@miyadav miyadav marked this pull request as draft October 14, 2025 15:32
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 14, 2025
@miyadav miyadav force-pushed the 2566vaptests_latest branch from 918f2a5 to f57cd5c Compare October 15, 2025 12:19
@miyadav
Copy link
Member Author

miyadav commented Oct 15, 2025

/unhold

@miyadav miyadav marked this pull request as ready for review October 15, 2025 12:21
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 15, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (3)
e2e/machine_migration_vap_tests.go (3)

3-18: Stop using YAML for providerSpec; switch to encoding/json.

YAML bytes in runtime.RawExtension cause JSON encoding to fail before VAP, so tests won’t observe the policy error. Replace the YAML import with encoding/json.

 import (
+	"encoding/json"
 	"fmt"
 
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
@@
 	"sigs.k8s.io/controller-runtime/pkg/client"
 	"sigs.k8s.io/controller-runtime/pkg/envtest/komega"
-	yaml "sigs.k8s.io/yaml"
 )

74-101: Missing coverage: spec.providerSpec update should be blocked

Add a test that mutates spec.providerSpec (e.g., instanceType) and asserts the VAP rejection.

 		Context("spec field restrictions", func() {
 			It("should prevent updating spec.providerID", func() {
@@
 			})
 
+			It("should prevent updating spec.providerSpec", func() {
+				verifyAWSProviderSpecUpdatePrevented(
+					testMAPIMachine,
+					"instanceType",
+					testInstanceType,
+					vapSpecLockedMessage,
+				)
+			})
 		})

183-255: providerSpec helper marshals with YAML; this breaks the update path

json is required for RawExtension. Swap yaml.Unmarshal/Marshal to encoding/json to let the Update reach VAP.

-	providerSpec := &mapiv1beta1.AWSMachineProviderConfig{}
-	if err := yaml.Unmarshal(freshMachine.Spec.ProviderSpec.Value.Raw, providerSpec); err != nil {
+	providerSpec := &mapiv1beta1.AWSMachineProviderConfig{}
+	if err := json.Unmarshal(freshMachine.Spec.ProviderSpec.Value.Raw, providerSpec); err != nil {
 		return fmt.Errorf("failed to unmarshal providerSpec: %v", err)
 	}
@@
-	// Marshal back to raw extension
-	modifiedRaw, err := yaml.Marshal(providerSpec)
+	// Marshal back to raw extension (JSON)
+	modifiedRaw, err := json.Marshal(providerSpec)
 	if err != nil {
 		return fmt.Errorf("failed to marshal modified providerSpec: %v", err)
 	}
 
 	freshMachine.Spec.ProviderSpec.Value = &runtime.RawExtension{Raw: modifiedRaw}
🧹 Nitpick comments (3)
e2e/machine_migration_vap_tests.go (3)

167-259: Prefer Komega here for consistency with other helpers

Use komega.Update instead of manual cl.Get/cl.Update for consistency and less boilerplate. Keep JSON marshal/unmarshal as above.

-func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, testValue interface{}, expectedError string) {
-	By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
-
-	Eventually(func() error {
-		// Get fresh copy to avoid conflicts
-		freshMachine := &mapiv1beta1.Machine{}
-		if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), freshMachine); err != nil {
-			return err
-		}
-		// Parse the current providerSpec
-		if freshMachine.Spec.ProviderSpec.Value == nil {
-			return fmt.Errorf("providerSpec is nil")
-		}
-		providerSpec := &mapiv1beta1.AWSMachineProviderConfig{}
-		if err := json.Unmarshal(freshMachine.Spec.ProviderSpec.Value.Raw, providerSpec); err != nil {
-			return fmt.Errorf("failed to unmarshal providerSpec: %v", err)
-		}
-		// ... mutate switch ...
-		modifiedRaw, err := json.Marshal(providerSpec)
-		if err != nil {
-			return fmt.Errorf("failed to marshal modified providerSpec: %v", err)
-		}
-		freshMachine.Spec.ProviderSpec.Value = &runtime.RawExtension{Raw: modifiedRaw}
-		return cl.Update(ctx, freshMachine)
-	}, capiframework.WaitMedium, capiframework.RetryMedium).Should(
-		MatchError(ContainSubstring(expectedError)),
-		"Expected AWS providerSpec.%s update to be blocked by VAP", fieldName)
-}
+func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, testValue interface{}, expectedError string) {
+	By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
+	Eventually(komega.Update(machine, func() {
+		if machine.Spec.ProviderSpec.Value == nil || len(machine.Spec.ProviderSpec.Value.Raw) == 0 {
+			return
+		}
+		providerSpec := &mapiv1beta1.AWSMachineProviderConfig{}
+		if err := json.Unmarshal(machine.Spec.ProviderSpec.Value.Raw, providerSpec); err != nil {
+			return
+		}
+		switch fieldName {
+		case "instanceType":
+			providerSpec.InstanceType = testValue.(string)
+		case "amiID":
+			s := testValue.(string)
+			if providerSpec.AMI.ID == nil {
+				providerSpec.AMI.ID = &s
+			} else {
+				*providerSpec.AMI.ID = s
+			}
+		case "availabilityZone":
+			providerSpec.Placement.AvailabilityZone = testValue.(string)
+		case "subnetID":
+			s := testValue.(string)
+			providerSpec.Subnet = mapiv1beta1.AWSResourceReference{ID: &s}
+		case "securityGroups":
+			s := testValue.(string)
+			providerSpec.SecurityGroups = []mapiv1beta1.AWSResourceReference{{ID: &s}}
+		case "volumeSize":
+			if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
+				v := testValue.(int64)
+				providerSpec.BlockDevices[0].EBS.VolumeSize = &v
+			}
+		case "volumeType":
+			if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
+				s := testValue.(string)
+				providerSpec.BlockDevices[0].EBS.VolumeType = &s
+			}
+		case "encryption":
+			if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
+				b := testValue.(bool)
+				providerSpec.BlockDevices[0].EBS.Encrypted = &b
+			}
+		case "tags":
+			tagMap := testValue.(map[string]string)
+			var tags []mapiv1beta1.TagSpecification
+			for k, v := range tagMap {
+				tags = append(tags, mapiv1beta1.TagSpecification{Name: k, Value: v})
+			}
+			providerSpec.Tags = tags
+		default:
+			return
+		}
+		if raw, err := json.Marshal(providerSpec); err == nil {
+			machine.Spec.ProviderSpec.Value.Raw = raw
+		}
+	}), capiframework.WaitMedium, capiframework.RetryMedium).Should(
+		MatchError(ContainSubstring(expectedError)),
+		fmt.Sprintf("Expected AWS providerSpec.%s update to be blocked by VAP", fieldName),
+	)
+}

26-33: Unused providerSpec test constants

These constants are currently unused. Either use them in providerSpec tests or remove to reduce noise.


102-120: Optional: also assert kubernetes.io/ label protection*

You only test machine.openshift.io/; consider adding a kubernetes.io/ case for completeness.

It("should prevent modifying kubernetes.io/* labels", func() {
	verifyUpdatePrevented(testMAPIMachine, func() {
		if testMAPIMachine.Labels == nil {
			testMAPIMachine.Labels = make(map[string]string)
		}
		testMAPIMachine.Labels["kubernetes.io/test-label"] = testLabelValue
	}, vapProtectedLabelMessage)
})
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 0e4e698 and f57cd5c.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (1)
e2e/machine_migration_vap_tests.go (1)

150-165: Good use of Komega wrappers

verifyUpdatePrevented/Allowed follow the recommended Komega pattern.

@chrischdi
Copy link
Contributor

@miyadav can we have a proper title for the PR? (including OCPCLOUD-<id>: <title>

@miyadav miyadav changed the title OCPCLOUD-2566 -tests with VAP msgs OCPCLOUD-2566: tests with VAP msgs Oct 15, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 15, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 15, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added comprehensive end-to-end tests for Machine API migration VAP enforcement on AWS.
  • Verifies protected machine fields (providerID, taints, providerSpec, metadata) are blocked with specific VAP errors.
  • Confirms updates to authoritativeAPI are allowed and VAP is not enforced when MachineAPI is authoritative.
  • Validates protected label and annotation restrictions while allowing non-protected exceptions.
  • Includes setup, cleanup, and reusable helpers to ensure reliable, field-level enforcement checks.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 21, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added end-to-end tests validating VAP enforcement during Machine API migration on AWS.
  • Ensures updates to protected machine fields (providerID, taints, providerSpec, metadata) are blocked with explicit VAP errors.
  • Confirms VAP is not enforced when MachineAPI is the authoritative API.
  • Verifies protected label and annotation restrictions while allowing non-protected entries.
  • Includes setup, cleanup, and reusable helpers for reliable, field-level verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (5)
e2e/machine_migration_vap_tests.go (5)

74-100: Add a providerSpec update test to fully exercise spec lock.

spec.providerSpec changes are part of VAP enforcement but are not covered here. You already have constants and a helper for this. Add a focused test to prevent regressions.

Apply near this context:

 Context("spec field restrictions", func() {
   It("should prevent updating spec.providerID", func() {
     verifyUpdatePrevented(testMAPIMachine, func() {
       providerIDValue := testProviderID
       testMAPIMachine.Spec.ProviderID = &providerIDValue
     }, vapSpecLockedMessage)
   })

+  It("should prevent updating spec.providerSpec (e.g., instanceType)", func() {
+    // Reuses the existing helper and constants
+    verifyAWSProviderSpecUpdatePrevented(testMAPIMachine, "instanceType", testInstanceType, vapSpecLockedMessage)
+  })
 })

56-72: Optional: wait for status.authoritativeAPI=ClusterAPI to reduce flakes.

Enforcement depends on status.authoritativeAPI. A short readiness wait here can make CI steadier without changing behavior.

 BeforeAll(func() {
   // Create a MAPI machine with ClusterAPI authority to trigger VAP enforcement
   testMAPIMachine = createMAPIMachineWithAuthority(ctx, cl, testMachineName, mapiv1beta1.MachineAuthorityClusterAPI)

   // The VAP requires a matching CAPI machine as parameter
   testCAPIMachine = capiframework.GetMachine(cl, testMachineName, capiframework.CAPINamespace)

+  // Harden: wait until enforcement condition is actually met
+  Eventually(func() (mapiv1beta1.MachineAuthority, error) {
+    fresh := &mapiv1beta1.Machine{}
+    err := cl.Get(ctx, client.ObjectKeyFromObject(testMAPIMachine), fresh)
+    return fresh.Status.AuthoritativeAPI, err
+  }, capiframework.WaitLong, capiframework.RetryMedium).Should(
+    Equal(mapiv1beta1.MachineAuthorityClusterAPI),
+    "expect status.authoritativeAPI=ClusterAPI before enforcement checks",
+  )

167-259: ProviderSpec helper: avoid silent no-ops and align with Komega.

  • Today, several cases can end up not modifying anything (e.g., missing BlockDevices/EBS) yet proceed to Update, risking misleading outcomes if used later.
  • Style: the rest of the file uses komega.Update; consider the same for consistency.

Minimal robustness tweak (keep cl.Get/Update) — ensure a real mutation or fail fast:

 func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, testValue interface{}, expectedError string) {
   By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))

   Eventually(func() error {
@@
-    // Modify the specified field
+    // Modify the specified field
+    modified := false
     switch fieldName {
     case "instanceType":
       providerSpec.InstanceType = testValue.(string)
+      modified = true
     case "amiID":
       testValueStr := testValue.(string)
       if providerSpec.AMI.ID == nil {
         providerSpec.AMI.ID = &testValueStr
       } else {
         *providerSpec.AMI.ID = testValueStr
       }
+      modified = true
     case "availabilityZone":
       providerSpec.Placement.AvailabilityZone = testValue.(string)
+      modified = true
     case "subnetID":
       testValueStr := testValue.(string)
       providerSpec.Subnet = mapiv1beta1.AWSResourceReference{
         ID: &testValueStr,
       }
+      modified = true
     case "securityGroups":
       providerSpec.SecurityGroups = []mapiv1beta1.AWSResourceReference{{
         ID: &[]string{testValue.(string)}[0],
       }}
+      modified = true
     case "volumeSize":
       if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
         providerSpec.BlockDevices[0].EBS.VolumeSize = &[]int64{testValue.(int64)}[0]
+        modified = true
       }
     case "volumeType":
       if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
         providerSpec.BlockDevices[0].EBS.VolumeType = &[]string{testValue.(string)}[0]
+        modified = true
       }
     case "encryption":
       if len(providerSpec.BlockDevices) > 0 && providerSpec.BlockDevices[0].EBS != nil {
         providerSpec.BlockDevices[0].EBS.Encrypted = &[]bool{testValue.(bool)}[0]
+        modified = true
       }
     case "tags":
@@
       providerSpec.Tags = tags
+      modified = true
-    case "capacityReservationId":
-      providerSpec.SpotMarketOptions = &mapiv1beta1.SpotMarketOptions{
-        MaxPrice: &[]string{"0.10"}[0],
-      }
-      // Note: capacityReservationId might be in different location based on AWS API version
     default:
       return fmt.Errorf("unsupported field: %s", fieldName)
     }
+
+    if !modified {
+      return fmt.Errorf("no-op mutation for field %q (missing nested fields?)", fieldName)
+    }

If you prefer full consistency, I can also rewrite this helper to use komega.Update and return a komega closure while still surfacing parse/mutation errors. Want me to push that version?


20-38: Prune or use the extra providerSpec constants.

testInstanceType, testAMIID, testAvailabilityZone, testSubnetID, testSecurityGroupID, testVolumeSize, and testCapacityReservationID are currently unused in tests. Either use them (e.g., in the providerSpec test) or remove to keep the file lean.


261-280: Optional: avoid name collisions for the MachineAPI-authority test.

To be safer under parallel CI or repeated runs, add a unique suffix.

- testMachine := createMAPIMachineWithAuthority(ctx, cl, "vap-test-mapi-authority", mapiv1beta1.MachineAuthorityMachineAPI)
+ uniqueName := fmt.Sprintf("vap-test-mapi-authority-%d", GinkgoParallelProcess())
+ testMachine := createMAPIMachineWithAuthority(ctx, cl, uniqueName, mapiv1beta1.MachineAuthorityMachineAPI)
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between f57cd5c and daaff8d.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (3)
e2e/machine_migration_vap_tests.go (3)

102-120: Protected labels: good coverage.

Blocking machine.openshift.io/* while allowing non-protected labels is clearly tested.


122-140: Protected annotations: good coverage.

Denial for machine.openshift.io/* and allow-list for others looks correct.


150-165: Komega-based update helpers look solid.

Nice switch to komega.Update with bounded waits/retries; assertions read well.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
e2e/machine_migration_vap_tests.go (2)

26-33: Build blocker: unused constants cause compile failure

These constants are never referenced, so go test will fail with “declared and not used”. Either use them in tests or remove them.

Apply this minimal fix (keep only those actually used today and the one we’ll use in the providerSpec test below):

 const (
   // Test values for MAPI machine updates
   testProviderID            = "aws:///us-west-2a/i-test123456"
   testTaintValue            = "test-taint-value"
   testLabelValue            = "test-label-value"
   testInstanceType          = "m5.xlarge"
-  testAMIID                 = "ami-test123456"
-  testAvailabilityZone      = "us-west-2b"
-  testSubnetID              = "subnet-test123456"
-  testSecurityGroupID       = "sg-test123456"
-  testVolumeSize            = int64(120)
-  testCapacityReservationID = "cr-test123456"

74-100: Missing providerSpec coverage — add a focused test

VAP should block providerSpec edits, but there’s no test exercising it. Use the existing helper.

   Context("spec field restrictions", func() {
@@
     It("should prevent updating spec.metadata", func() {
       verifyUpdatePrevented(testMAPIMachine, func() {
         if testMAPIMachine.Spec.ObjectMeta.Labels == nil {
           testMAPIMachine.Spec.ObjectMeta.Labels = make(map[string]string)
         }
         testMAPIMachine.Spec.ObjectMeta.Labels["test-spec-label"] = testLabelValue
       }, vapSpecLockedMessage)
     })
+
+    It("should prevent updating spec.providerSpec (instanceType)", func() {
+      verifyAWSProviderSpecUpdatePrevented(
+        testMAPIMachine,
+        "instanceType",
+        testInstanceType,
+        vapSpecLockedMessage,
+      )
+    })
   })
🧹 Nitpick comments (5)
e2e/machine_migration_vap_tests.go (5)

171-176: Use Komega here for consistency with the rest of the suite

This helper still uses raw client Get/Update. Prefer Komega like your other helpers.

 func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, testValue interface{}, expectedError string) {
   By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
-
-  Eventually(func() error {
-    // Get fresh copy to avoid conflicts
-    freshMachine := &mapiv1beta1.Machine{}
-    if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), freshMachine); err != nil {
-      return err
-    }
+  Eventually(komega.Update(machine, func() {
+    freshMachine := machine
@@
-    freshMachine.Spec.ProviderSpec.Value = &runtime.RawExtension{Raw: modifiedRaw}
-    return cl.Update(ctx, freshMachine)
-  }, capiframework.WaitMedium, capiframework.RetryMedium).Should(
+    freshMachine.Spec.ProviderSpec.Value = &runtime.RawExtension{Raw: modifiedRaw}
+  }), capiframework.WaitMedium, capiframework.RetryMedium).Should(
     MatchError(ContainSubstring(expectedError)),
     "Expected AWS providerSpec.%s update to be blocked by VAP", fieldName)
 }

Also remove the now-unused import "sigs.k8s.io/controller-runtime/pkg/client".

Also applies to: 251-253


211-227: Guard against no-op when BlockDevices/EBS is empty

If BlockDevices is empty or EBS is nil, the mutation won’t change anything, risking false negatives.

-    case "volumeSize":
-      if len(providerSpec.BlockDevices) > 0 {
-        if providerSpec.BlockDevices[0].EBS != nil {
-          providerSpec.BlockDevices[0].EBS.VolumeSize = &[]int64{testValue.(int64)}[0]
-        }
-      }
+    case "volumeSize":
+      if len(providerSpec.BlockDevices) == 0 {
+        providerSpec.BlockDevices = []mapiv1beta1.BlockDeviceMappingSpec{{EBS: &mapiv1beta1.EBSBlockDeviceSpec{}}}
+      } else if providerSpec.BlockDevices[0].EBS == nil {
+        providerSpec.BlockDevices[0].EBS = &mapiv1beta1.EBSBlockDeviceSpec{}
+      }
+      providerSpec.BlockDevices[0].EBS.VolumeSize = &[]int64{testValue.(int64)}[0]
@@
-    case "volumeType":
-      if len(providerSpec.BlockDevices) > 0 {
-        if providerSpec.BlockDevices[0].EBS != nil {
-          providerSpec.BlockDevices[0].EBS.VolumeType = &[]string{testValue.(string)}[0]
-        }
-      }
+    case "volumeType":
+      if len(providerSpec.BlockDevices) == 0 {
+        providerSpec.BlockDevices = []mapiv1beta1.BlockDeviceMappingSpec{{EBS: &mapiv1beta1.EBSBlockDeviceSpec{}}}
+      } else if providerSpec.BlockDevices[0].EBS == nil {
+        providerSpec.BlockDevices[0].EBS = &mapiv1beta1.EBSBlockDeviceSpec{}
+      }
+      providerSpec.BlockDevices[0].EBS.VolumeType = &[]string{testValue.(string)}[0]
@@
-    case "encryption":
-      if len(providerSpec.BlockDevices) > 0 {
-        if providerSpec.BlockDevices[0].EBS != nil {
-          providerSpec.BlockDevices[0].EBS.Encrypted = &[]bool{testValue.(bool)}[0]
-        }
-      }
+    case "encryption":
+      if len(providerSpec.BlockDevices) == 0 {
+        providerSpec.BlockDevices = []mapiv1beta1.BlockDeviceMappingSpec{{EBS: &mapiv1beta1.EBSBlockDeviceSpec{}}}
+      } else if providerSpec.BlockDevices[0].EBS == nil {
+        providerSpec.BlockDevices[0].EBS = &mapiv1beta1.EBSBlockDeviceSpec{}
+      }
+      providerSpec.BlockDevices[0].EBS.Encrypted = &[]bool{testValue.(bool)}[0]

102-120: Add coverage for kubernetes.io/ protected labels*

Your message asserts both machine.openshift.io/* and kubernetes.io/* are protected; only the former is tested.

   Context("protected label restrictions", func() {
@@
     It("should allow modifying non-protected labels", func() {
       verifyUpdateAllowed(testMAPIMachine, func() {
         if testMAPIMachine.Labels == nil {
           testMAPIMachine.Labels = make(map[string]string)
         }
         testMAPIMachine.Labels["test-label"] = "allowed-value"
       })
     })
+
+    It("should prevent modifying kubernetes.io/* labels", func() {
+      verifyUpdatePrevented(testMAPIMachine, func() {
+        if testMAPIMachine.Labels == nil {
+          testMAPIMachine.Labels = make(map[string]string)
+        }
+        testMAPIMachine.Labels["kubernetes.io/test-label"] = testLabelValue
+      }, vapProtectedLabelMessage)
+    })
   })

207-209: Minor: avoid ephemeral pointer trick for strings

Prefer a helper for clarity, e.g., k8s.io/utils/ptr.

-      providerSpec.SecurityGroups = []mapiv1beta1.AWSResourceReference{{
-        ID: &[]string{testValue.(string)}[0],
-      }}
+      sg := testValue.(string)
+      providerSpec.SecurityGroups = []mapiv1beta1.AWSResourceReference{{ID: &sg}}

56-72: Optional: wait for authoritativeAPI readiness to de-flake

If you hit flakes, explicitly wait until status.authoritativeAPI == ClusterAPI before assertions.

   BeforeAll(func() {
@@
     testCAPIMachine = capiframework.GetMachine(cl, testMachineName, capiframework.CAPINamespace)
+
+    Eventually(func() (mapiv1beta1.MachineAuthority, error) {
+      fresh := &mapiv1beta1.Machine{}
+      err := cl.Get(ctx, client.ObjectKeyFromObject(testMAPIMachine), fresh)
+      return fresh.Status.AuthoritativeAPI, err
+    }, capiframework.WaitLong, capiframework.RetryMedium).Should(
+      Equal(mapiv1beta1.MachineAuthorityClusterAPI),
+      "VAP enforces only after status.authoritativeAPI=ClusterAPI",
+    )
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between daaff8d and c4750a1.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (1)
e2e/machine_migration_vap_tests.go (1)

239-241: capacityReservationId mutation looks correct now

This fixes the earlier mismatch (was mutating Spot options). No further action.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
e2e/machine_migration_vap_tests.go (1)

238-241: Simplify the pointer creation for SecurityGroups ID.

The pattern &[]string{testValue.(string)}[0] creates a slice just to take a pointer to its first element, which is unnecessarily complex and harder to read.

Apply this diff to improve clarity:

 	case "securityGroups":
+		id := testValue.(string)
 		providerSpec.SecurityGroups = []mapiv1beta1.AWSResourceReference{{
-			ID: &[]string{testValue.(string)}[0],
+			ID: &id,
 		}}
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between c4750a1 and 3f188cb.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)

@miyadav
Copy link
Member Author

miyadav commented Oct 23, 2025

/test okd-scos-e2e-aws-ovn

1 similar comment
@miyadav
Copy link
Member Author

miyadav commented Oct 23, 2025

/test okd-scos-e2e-aws-ovn

return fmt.Errorf("providerSpec is nil")
}

providerSpec := &mapiv1beta1.AWSMachineProviderConfig{}
Copy link
Contributor

@theobarberbany theobarberbany Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a helperfunc elsewhere that does something very similar to this? I think we added one in one of the other PRs?

(specifically one with an updateFunc() being passed in pattern)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remember now did it in pr 355. Will update

@miyadav miyadav force-pushed the 2566vaptests_latest branch from 3f188cb to b01f005 Compare October 23, 2025 14:51
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 23, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added end-to-end tests validating VAP enforcement during Machine API migration on AWS.
  • Ensures updates to protected machine fields (providerID, taints, metadata, providerSpec fields like instance type, AMI, AZ, subnet, security groups, tags) are blocked with explicit VAP errors.
  • Confirms VAP is not enforced when Machine API is authoritative.
  • Verifies protected label and annotation restrictions while allowing non-protected entries.
  • Includes setup, cleanup, and reusable helpers for field-level verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (8)
e2e/machine_migration_vap_tests.go (8)

51-53: Make test machine names unique to avoid collisions across retries/parallelism.

Static name can collide across runs. Append a short random/unique suffix.

Example:

-		var testMachineName = "machine-vap-test-capi-auth"
+		var testMachineName = fmt.Sprintf("machine-vap-test-capi-auth-%d", time.Now().UnixNano())

Add time import if you choose this approach.


100-107: Also block spec.metadata.annotations to complete spec-metadata coverage.

You cover labels under spec.metadata; add a sibling test for annotations.

 			It("should prevent updating spec.metadata", func() {
 				verifyUpdatePrevented(testMAPIMachine, func() {
 					if testMAPIMachine.Spec.ObjectMeta.Labels == nil {
 						testMAPIMachine.Spec.ObjectMeta.Labels = make(map[string]string)
 					}
 					testMAPIMachine.Spec.ObjectMeta.Labels["test-spec-label"] = testLabelValue
 				}, vapSpecLockedMessage)
 			})
+			It("should prevent updating spec.metadata.annotations", func() {
+				verifyUpdatePrevented(testMAPIMachine, func() {
+					if testMAPIMachine.Spec.ObjectMeta.Annotations == nil {
+						testMAPIMachine.Spec.ObjectMeta.Annotations = make(map[string]string)
+					}
+					testMAPIMachine.Spec.ObjectMeta.Annotations["test-spec-annotation"] = "x"
+				}, vapSpecLockedMessage)
+			})

33-37: Trim the expected-message substrings to reduce brittleness.

OPA/Gatekeeper messages can change slightly. Matching a shorter stable substring (e.g., “You may only modify spec.authoritativeAPI”) will make tests less fragile while preserving intent.


265-284: Use Komega here for consistency with the rest of the helpers.

Refactor providerSpec enforcement helper to komega.Update; avoids manual Get/Update and aligns with other helpers.

-func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, updateFunc func(*mapiv1beta1.AWSMachineProviderConfig), expectedError string) {
-	By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
-
-	Eventually(func() error {
-		// Get fresh copy to avoid conflicts
-		freshMachine := &mapiv1beta1.Machine{}
-		if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), freshMachine); err != nil {
-			return err
-		}
-
-		if err := updateAWSMachineProviderSpec(freshMachine, updateFunc); err != nil {
-			return err
-		}
-
-		return cl.Update(ctx, freshMachine)
-	}, capiframework.WaitMedium, capiframework.RetryMedium).Should(
-		MatchError(ContainSubstring(expectedError)),
-		"Expected AWS providerSpec.%s update to be blocked by VAP", fieldName)
-}
+func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, updateFunc func(*mapiv1beta1.AWSMachineProviderConfig), expectedError string) {
+	By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
+	Eventually(komega.Update(machine, func() {
+		// Mutate the fresh object fetched by komega.Update
+		ps := &mapiv1beta1.AWSMachineProviderConfig{}
+		Expect(json.Unmarshal(machine.Spec.ProviderSpec.Value.Raw, ps)).To(Succeed())
+		updateFunc(ps)
+		b, err := json.Marshal(ps)
+		Expect(err).ToNot(HaveOccurred())
+		machine.Spec.ProviderSpec.Value = &runtime.RawExtension{Raw: b}
+	}), capiframework.WaitMedium, capiframework.RetryMedium).Should(
+		MatchError(ContainSubstring(expectedError)),
+		fmt.Sprintf("Expected AWS providerSpec.%s update to be blocked by VAP", fieldName),
+	)
+}

242-245: Use %w when wrapping errors.

Allows unwrapping with errors.Is/As.

-	return fmt.Errorf("failed to marshal modified providerSpec: %v", err)
+	return fmt.Errorf("failed to marshal modified providerSpec: %w", err)
-	return nil, fmt.Errorf("failed to unmarshal providerSpec: %v", err)
+	return nil, fmt.Errorf("failed to unmarshal providerSpec: %w", err)

Also applies to: 258-260


110-118: Optional: Add a deletion case for protected labels.

To fully exercise “add/modify/delete”, add a test that attempts to delete an existing protected label (ensure it exists first).


130-138: Optional: Add a deletion case for protected annotations.

Mirror the label-deletion suggestion for annotations; pre-create the annotation or switch authority to allow creation, then verify deletion is blocked under ClusterAPI.


208-212: Broaden the MachineAPI authority bypass check.

Also verify non-protected label/annotation updates succeed when authoritativeAPI=MachineAPI to cover metadata path.

 	It("should not apply VAP when authoritativeAPI is MachineAPI", func() {
 		verifyVAPNotAppliedForMachineAPIAuthority()
 	})

Inside verifyVAPNotAppliedForMachineAPIAuthority:

 	// Verify we can update spec fields (VAP should not apply)
 	Eventually(komega.Update(testMachine, func() {
 		// Try to update a spec field - this should be allowed since VAP doesn't apply
 		providerIDValue := testProviderID
 		testMachine.Spec.ProviderID = &providerIDValue
 	}), capiframework.WaitMedium, capiframework.RetryMedium).Should(Succeed(),
 		"Expected spec update to succeed when authoritativeAPI is MachineAPI (VAP should not apply)")
+
+	// Also verify non-protected metadata updates are allowed
+	Eventually(komega.Update(testMachine, func() {
+		if testMachine.Labels == nil {
+			testMachine.Labels = map[string]string{}
+		}
+		testMachine.Labels["vap-allow-label"] = "ok"
+		if testMachine.Annotations == nil {
+			testMachine.Annotations = map[string]string{}
+		}
+		testMachine.Annotations["vap-allow-annotation"] = "ok"
+	}), capiframework.WaitMedium, capiframework.RetryMedium).Should(Succeed())

Also applies to: 298-305

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 3f188cb and b01f005.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (2)
e2e/machine_migration_vap_tests.go (2)

55-70: Nice flake guard: waiting for authoritativeAPI=ClusterAPI before enforcing.

This explicit wait should reduce CI flakes and aligns with the VAP match condition. Looks good.


233-249: Confirm TypeMeta is preserved in providerSpec Raw.

JSON round-trip can drop kind/apiVersion if empty. Please verify they remain set after mutations to avoid downstream decode issues; if needed, re-set from the pre-mutation object when blank.

Also applies to: 251-264

Comment on lines 62 to 69
Eventually(func() (mapiv1beta1.MachineAuthority, error) {
fresh := &mapiv1beta1.Machine{}
err := cl.Get(ctx, client.ObjectKeyFromObject(testMAPIMachine), fresh)
return fresh.Status.AuthoritativeAPI, err
}, capiframework.WaitLong, capiframework.RetryMedium).Should(
Equal(mapiv1beta1.MachineAuthorityClusterAPI),
"VAP requires status.authoritativeAPI=Cluster API before enforcement",
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change to Eventually(komega.Get(...))

Comment on lines 253 to 255
if machine.Spec.ProviderSpec.Value == nil {
return nil, fmt.Errorf("providerSpec is nil")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expect(machine.Spec.ProviderSpec.Value).ToNot(BeNil())

Comment on lines 258 to 260
if err := json.Unmarshal(machine.Spec.ProviderSpec.Value.Raw, providerSpec); err != nil {
return nil, fmt.Errorf("failed to unmarshal providerSpec: %v", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expect(machine.Spec.ProviderSpec.Value.Raw, providerSpec)).To(Succeed())

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there will be multiple changes , let me follow this pattern. Thanks.

removing the part which is covered in other PRs
co-author: cursor & coderabbitAI
@miyadav miyadav force-pushed the 2566vaptests_latest branch from b01f005 to 2eb2ed9 Compare October 24, 2025 07:05
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 24, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added end-to-end tests validating VAP enforcement during Machine API migration on AWS.
  • Blocks updates to protected machine fields (providerID, taints, metadata) with explicit VAP errors.
  • Prevents changes to AWS providerSpec fields (instance type, AMI, AZ, subnet, security groups, tags).
  • Confirms VAP is not enforced when Machine API is authoritative.
  • Verifies protected label and annotation restrictions while allowing non-protected entries.
  • Includes setup, cleanup, and reusable helpers for field-level verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
e2e/machine_migration_vap_tests.go (1)

258-277: Consider using Komega for consistency.

While the current implementation works, verifyAWSProviderSpecUpdatePrevented uses direct cl.Get and cl.Update calls, whereas other helpers (verifyUpdatePrevented, verifyUpdateAllowed) use Komega. For consistency and to follow the pattern established elsewhere in the test suite, consider refactoring to use Komega utilities.

The current approach is functional, so this is optional and can be deferred if the refactor adds complexity without clear benefit.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between b01f005 and 2eb2ed9.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)

@miyadav
Copy link
Member Author

miyadav commented Oct 28, 2025

/test okd-scos-e2e-aws-ovn

@miyadav
Copy link
Member Author

miyadav commented Nov 3, 2025

@theobarberbany PTAL when time permits , looking to get this in :D .

@theobarberbany
Copy link
Contributor

One change to consider, otherwise LGTM!

What's the global timeout on these VAP tests?

/lgtm

@openshift-ci openshift-ci bot added lgtm Indicates that a PR is ready to be merged. and removed lgtm Indicates that a PR is ready to be merged. labels Nov 3, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Nov 3, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added end-to-end tests validating VAP enforcement during Machine API migration on AWS.
  • Blocks updates to protected machine fields (providerID, taints, metadata) with explicit VAP errors.
  • Prevents changes to AWS providerSpec fields (instance type, AMI, AZ, subnet, security groups, tags, capacity reservation).
  • Confirms VAP is not enforced when Machine API is authoritative.
  • Verifies protected label and annotation restrictions while allowing non-protected entries.
  • Includes setup, cleanup, and reusable helpers for field-level verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@miyadav
Copy link
Member Author

miyadav commented Nov 3, 2025

@theobarberbany we are using waitlong at couple of places which is 15min, reducing it to waitMedium ( 3 mins ) , I think that is sufficient for the test.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
e2e/machine_migration_vap_tests.go (1)

258-277: Consider using Komega for consistency.

While verifyUpdatePrevented and verifyUpdateAllowed use Komega (lines 219, 228), this helper uses direct cl.Get and cl.Update calls. Although previous reviews suggested Komega throughout, the current pattern may be intentional to handle the error return from updateAWSMachineProviderSpec.

For consistency and to avoid potential confusion when updateAWSMachineProviderSpec fails with a non-VAP error, consider refactoring to separate concerns: handle unmarshal/marshal errors with Expect (fail fast on setup errors), then use Komega for the update assertion.

Example refactor:

 func verifyAWSProviderSpecUpdatePrevented(machine *mapiv1beta1.Machine, fieldName string, updateFunc func(*mapiv1beta1.AWSMachineProviderConfig), expectedError string) {
 	By(fmt.Sprintf("Verifying that updating AWS providerSpec.%s is prevented by VAP", fieldName))
 
-	Eventually(func() error {
-		// Get fresh copy to avoid conflicts
-		freshMachine := &mapiv1beta1.Machine{}
-		if err := cl.Get(ctx, client.ObjectKeyFromObject(machine), freshMachine); err != nil {
-			return err
-		}
-
-		if err := updateAWSMachineProviderSpec(freshMachine, updateFunc); err != nil {
-			return err
-		}
-
-		return cl.Update(ctx, freshMachine)
-	}, capiframework.WaitMedium, capiframework.RetryMedium).Should(
+	Eventually(komega.Update(machine, func() {
+		Expect(updateAWSMachineProviderSpec(machine, updateFunc)).To(Succeed())
+	}), capiframework.WaitMedium, capiframework.RetryMedium).Should(
 		MatchError(ContainSubstring(expectedError)),
 		"Expected AWS providerSpec.%s update to be blocked by VAP", fieldName)
 }
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 2eb2ed9 and a5aebc1.

📒 Files selected for processing (1)
  • e2e/machine_migration_vap_tests.go (1 hunks)
🔇 Additional comments (6)
e2e/machine_migration_vap_tests.go (6)

3-37: LGTM! Constants and imports are well-organized.

The imports are appropriate for the test suite, and the constants clearly define test values and expected VAP error messages. The use of encoding/json is correct for providerSpec handling.


39-79: LGTM! Test setup is robust.

The test suite properly gates execution on AWS platform and MachineAPIMigration feature flag. The wait for status.authoritativeAPI (lines 62-68) correctly addresses potential race conditions, and cleanup is properly handled with DeferCleanup.


81-212: LGTM! Comprehensive test coverage.

The test contexts provide thorough coverage of VAP enforcement scenarios, including:

  • Spec field restrictions (providerID, taints, metadata)
  • Protected vs non-protected labels and annotations
  • Multiple AWS providerSpec fields
  • VAP bypass verification when authoritativeAPI is MachineAPI

Each test is focused and uses the appropriate helper functions.


215-230: LGTM! Helpers use Komega correctly.

Both verifyUpdatePrevented and verifyUpdateAllowed properly use komega.Update for consistency with the codebase and testing best practices.


232-256: LGTM! ProviderSpec extraction and update helpers are correct.

The helpers properly use encoding/json for marshaling/unmarshaling providerSpec, avoiding the YAML-vs-JSON issues from earlier reviews. The use of Expect in getAWSProviderSpecFromMachine provides clear failure messages if providerSpec is missing.


279-306: LGTM! VAP bypass test is well-structured.

The test correctly:

  • Creates a machine with MachineAPI authority
  • Waits for status.authoritativeAPI to be set (lines 286-292), preventing race conditions
  • Verifies spec updates are allowed when VAP doesn't apply
  • Uses komega.Update for consistency

The use of mapiframework.DeleteMachines for cleanup (line 296) differs from cleanupMachineResources used in the main test (line 72), but both are valid approaches.

@miyadav miyadav force-pushed the 2566vaptests_latest branch from a5aebc1 to 354042a Compare November 3, 2025 13:59
@openshift-ci-robot
Copy link

openshift-ci-robot commented Nov 3, 2025

@miyadav: This pull request references OCPCLOUD-2566 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target either version "4.21." or "openshift-4.21.", but it targets "openshift-4.20" instead.

In response to this:

@sunzhaohua2 @huali9 PTAL when time permits , I have to omit few cases as error messages were not there for them , they actually get reconciled to old values , example labels of machinetype even after changing.

Also , my apologies the PR - #355 has been opened so long , even I forgot changes so , I took the rebase of latest code and recreated this one.

I will close that one.

Summary by CodeRabbit

  • Tests
  • Added end-to-end tests validating Verification and Access Policy (VAP) enforcement during Machine API migration on AWS.
  • Blocks updates to protected machine fields (providerID, taints, metadata) with explicit VAP errors.
  • Prevents changes to AWS providerSpec fields (instance type, AMI, availability zone, subnet, security groups, tags, capacity reservation).
  • Confirms VAP is not enforced when Machine API is authoritative.
  • Verifies protected label and annotation restrictions while allowing non-protected entries.
  • Adds setup, cleanup, and reusable helpers for field-level verification.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

reduce time,changes to follow consistent pattern
@miyadav miyadav force-pushed the 2566vaptests_latest branch from 354042a to ebde40b Compare November 3, 2025 14:11
@theobarberbany
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 3, 2025

@miyadav: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@sunzhaohua2
Copy link
Contributor

/assign @damdo
For approve

@theobarberbany
Copy link
Contributor

/approve
/hold for dam to have a look if he wants

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 11, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 11, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: theobarberbany

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants