Skip to content

fix(approve-installplan): use dedicated SA instead of ArgoCD - #94

Open
pinikomarov wants to merge 3 commits into
openstack-k8s-operators:mainfrom
pinikomarov:fix/approve-installplan-sa
Open

fix(approve-installplan): use dedicated SA instead of ArgoCD #94
pinikomarov wants to merge 3 commits into
openstack-k8s-operators:mainfrom
pinikomarov:fix/approve-installplan-sa

Conversation

@pinikomarov

@pinikomarov pinikomarov commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

The approve-openstack-installplan Job in components/utilities/approve-installplan/ previously used serviceAccountName: openshift-gitops-argocd-application-controller in the openshift-gitops namespace. In Validated Patterns (VP) deployments, ArgoCD runs in a dedicated namespace (rhoso-gitops-standalone), so openshift-gitops-argocd-application-controller never exists in openshift-gitops — causing the Job pod to fail with FailedCreate in a loop indefinitely.

Changes

components/utilities/approve-installplan/rbac.yaml

  • Add dedicated ServiceAccount: installplan-approver in openshift-gitops namespace
  • Update ClusterRoleBinding subject from openshift-gitops-argocd-application-controller to installplan-approver
  • Add clusterserviceversions to the ClusterRole rules (required for the idempotency check)

components/utilities/approve-installplan/job.yaml

  • Update serviceAccountName to installplan-approver
  • Add short-circuit: if the pinned CSV is already in Succeeded phase, exit 0 immediately — handles re-runs where the operator is already installed but OLM has created an upgrade InstallPlan for a newer version

Why a dedicated SA

The original design borrowed the ArgoCD application controller SA because it already had broad cluster access. In VP deployments this SA does not exist in openshift-gitops. A dedicated SA with only the permissions the Job actually needs is more correct regardless of deployment model:

  • Works in both standard GitOps and VP deployments
  • Follows least-privilege principle
  • Makes the component self-contained

Testing

Validated in a VP CI pipeline (doc-extracted-vp) on OCP 4.18 with openstack-operator.v1.0.16:

  • Job pod created successfully with installplan-approver SA ✅
  • InstallPlan approved, operator installed, CRDs registered ✅
  • Re-run idempotency: short-circuit exits 0 when CSV already Succeeded
  • Version-gating: correctly refuses to approve upgrade plan for v1.21.0 when pinned to v1.0.16

Jira: OSPRH-32069

…er SA

The Job was referencing openshift-gitops-argocd-application-controller in
the openshift-gitops namespace. In Validated Patterns deployments ArgoCD
runs in rhoso-gitops-standalone, so that SA never exists in openshift-gitops
and the Job pod could never be created (FailedCreate loop for 9+ hours).

Create a dedicated installplan-approver ServiceAccount in rbac.yaml bound
to the existing installplan-approver-role ClusterRole. The component is now
self-contained and works regardless of which ArgoCD instance manages it.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pinikomarov pinikomarov changed the title fix(approve-installplan): use dedicated SA instead of ArgoCD controll… fix(approve-installplan): use dedicated SA instead of ArgoCD Jul 27, 2026
pinikomarov and others added 2 commits July 27, 2026 10:00
After a successful initial install the subscription has UpgradePending
for a newer version. The version-gating check correctly refused to
approve the upgrade plan, but then fell through to exit 1 because no
plan for the pinned CSV existed.

Add a short-circuit at the start of main execution: if the pinned
startingCSV is already in Succeeded phase, skip InstallPlan approval
entirely and go straight to wait_for_crd. This makes the job idempotent
on re-runs after the operator is already installed.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…check

The short-circuit added in the previous commit calls `oc get csv` to check
if the pinned CSV is already Succeeded. Without get/list on
clusterserviceversions the call silently returns empty (|| true swallows
the RBAC error), bypassing the short-circuit and falling through to the
version-gated plan search.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@pinikomarov
pinikomarov marked this pull request as ready for review July 29, 2026 07:28
@pinikomarov

Copy link
Copy Markdown
Collaborator Author

Dedicated SA comment,
why it was needed: ?

The approve-openstack-installplan Job was configured with serviceAccountName: openshift-gitops-argocd-application-controller in the openshift-gitops namespace. In a standard OpenShift GitOps deployment, that SA is created by the GitOps operator alongside its default ArgoCD instance in openshift-gitops. In VP deployments, ArgoCD runs in vp-gitops and rhoso-gitops-standalone — the openshift-gitops namespace exists but has no ArgoCD SA.

Result: the Job pod could never be created. Every attempt produced:

Error creating: pods "approve-openstack-installplan-" is forbidden:
error looking up service account openshift-gitops/openshift-gitops-argocd-application-controller:
serviceaccount "openshift-gitops-argocd-application-controller" not found
The Job stayed in a FailedCreate loop for 9+ hours without any error surfaced to ArgoCD — the app showed Degraded with no clear cause. The OLM InstallPlan for openstack-operator.v1.0.16 was never approved, so the CRDs never installed, and every downstream app (openstack-operator-cr, openstack-controlplane) was permanently Missing.

Why the original design used the ArgoCD SA:

It was a shortcut — the ArgoCD SA already had a broad ClusterRoleBinding with cluster-wide access, avoiding the need to create dedicated RBAC. It works fine in standard deployments but silently fails in VP.

What the fix does:

Creates a dedicated ServiceAccount: installplan-approver in openshift-gitops with a purpose-built ClusterRole granting only what the Job actually needs — get/list/patch installplans, get/list clusterserviceversions, get/list customresourcedefinitions. The component is now self-contained and works in both standard and VP deployments.

What else was fixed alongside it:

Two bugs discovered during debugging:

Idempotency — on re-runs where the operator is already installed, OLM creates an upgrade plan for a newer version. The Job correctly refused to approve it (version-gating) but then failed with exit 1 instead of exit 0. Added short-circuit: if the pinned CSV is already Succeeded, exit 0 immediately.
RBAC gap — oc get csv was silently failing with 403 (no permission on clusterserviceversions), causing the short-circuit check to always be skipped. Added clusterserviceversions to the ClusterRole.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants