chore: automated UAT test on GCP #355

mchmarny · 2025-11-17T13:19:43Z

Summary

Automate UAT test on GCP with OIDC auth

Type of Change

Component(s) Affected

Testing

Tests pass locally
Manual testing completed
No breaking changes (or documented)

Checklist

Self-review completed
Documentation updated (if needed)
Ready for review

Summary by CodeRabbit

New Features
- GCP integration workflow to provision GKE GPU clusters and run end-to-end UATs.
Tests
- New GCP UATs for GPU Operator and NVSentinel with updated GPU/SXID flows (one SXID path disabled); added GPU event verifications and reboot/quarantine handling.
- Reduced cert-manager and Prometheus footprints to speed test startup; added cluster resource quota for GPU tests.
Documentation
- Added GKE cluster deployment and usage guide for testing.
Chores
- Terraform-based GCP project/cluster and GitHub federation setup; NVIDIA runtime enabled for GPU collectors; GCP-specific Helm value tweaks.

The || { ... } syntax was causing a syntax error with set -euo pipefail. Changed to standard if-then-fi structure for better compatibility. Signed-off-by: Mark Chmarny <[email protected]>

Signed-off-by: Mark Chmarny <[email protected]>

.github/workflows/integration-gcp.yml

scripts/copy-images.sh

tests/uat/gcp/kwok-node-template.yaml

tests/uat/gcp/nvidia-dcgm-daemonset.yaml

tests/uat/gcp/nvidia-driver-daemonset.yaml

tests/uat/gcp/project/LICENSE

tests/uat/install-apps.sh

…-gcp' into feature/oidc-gcp

…nto feature/oidc-gcp

mchmarny added 30 commits November 4, 2025 07:44

chore: implement image copy to gcp using oidc

aaa379b

chore: refactor to env vars

36d1078

chore: setup env vars

4a1b42c

chore: debug variables

3e1b34d

chore: move vars to the job

572238b

chore: inline variables

2584c8d

chore: move to branch

ccd50ca

fix: use if-then syntax instead of || for crane check

785e705

The || { ... } syntax was causing a syntax error with set -euo pipefail. Changed to standard if-then-fi structure for better compatibility. Signed-off-by: Mark Chmarny <[email protected]>

fix: replace }

ae1d7b8

Signed-off-by: Mark Chmarny <[email protected]>

chore: added cluster bringup

1d1433f

fix: remove deprecated logging/monitoring flags from cluster creation

0de0cf2

Signed-off-by: Mark Chmarny <[email protected]>

chore: run delete always

59c9b8a

chore: add default network check

5ae0468

chore: move script to uat

cf82026

chore: unique cluster name per test

0c66246

chore: add auth plugin

4e724a5

chore: add gpu node pool

40822f6

Merge branch 'main' into feature/oidc-gcp

2d091e5

debug: add GitHub context output to troubleshoot OIDC auth

9aacbea

Signed-off-by: Mark Chmarny <[email protected]>

chore: remove debug info

dffb7b5

chore: remove spaces

ce31be2

chore: add app install and tests

895372e

chore: repaint project

bd338a1

chore: update oidc info

006d28b

chore: bind current user to cluster admin role

5be1e13

chore: add container admin role

421bed9

chore: set csp env var

03f7651

chore: add missing values files

e3e776e

chore: added gpu operator values

1e444ac

chore: removing delete, skipping driver install

04b4f07

mchmarny linked an issue Nov 17, 2025 that may be closed by this pull request

[Feature]: end to end tests during UAT functional validation run on real GPU cluster in GCP #264

Closed

1 task

lalitadithya reviewed Nov 17, 2025

View reviewed changes