Skip to content

PROJQUAY-11215: feat: add TLS support for operator-managed PostgreSQL - #1277

Merged
openshift-merge-bot[bot] merged 20 commits into
quay:masterfrom
bcaton85:PROJQUAY-11215-postgres-tls
Jun 18, 2026
Merged

PROJQUAY-11215: feat: add TLS support for operator-managed PostgreSQL#1277
openshift-merge-bot[bot] merged 20 commits into
quay:masterfrom
bcaton85:PROJQUAY-11215-postgres-tls

Conversation

@bcaton85

@bcaton85 bcaton85 commented Jun 10, 2026

Copy link
Copy Markdown
Member

Adds opt-in TLS encryption for operator-managed PostgreSQL instances used by Quay Registry and Clair, enabling customers in regulated industries to meet encryption-in-transit requirements without provisioning external database infrastructure.

What this enables

spec:
components:
- kind: postgres
managed: true
overrides:
tls:
enabled: true
- kind: clairpostgres
managed: true
overrides:
tls:
enabled: true

  • PostgreSQL pods are configured with ssl = on and serve TLS connections
  • Quay and Clair connection strings are updated with sslmode=verify-full
  • Self-signed certificates (ECDSA P-256, 10yr) are generated automatically, or users can provide their own via secretRef (compatible with cert-manager)
  • Existing deployments are unaffected — TLS is only enabled when explicitly configured

Certificate options

  • Self-signed (tls.enabled: true) — Operator generates ECDSA P-256 certs automatically. Best for quick start and dev/test.
  • User-provided (tls.secretRef.name: my-certs) — Reference a Secret containing ca.crt, tls.crt, tls.key. For enterprise PKI integration.
  • cert-manager (tls.secretRef.name: cert-manager-secret) — Point secretRef at a cert-manager-populated Secret. Operator watches for changes and triggers reconciliation on
    rotation.

Commits

  1. apis: TLSOverride struct on the Override type, CEL validation, restricted to postgres/clairpostgres
  2. kustomize (certs): ECDSA P-256 cert generation, persisted in managed keys Secret
  3. kustomize (connection strings): sslmode=verify-full on Quay DB_URI and Clair connstring, CA Secret generation for both self-signed and secretRef modes
  4. middleware: Projected volume with defaultMode: 0600, init container for postgresql.conf patching (existing PVCs), ConfigMap patching (fresh PVCs), cleanup init container for downgrade
  5. controller: secretRef validation (key presence, cert/key match, expiry), RolloutBlocked on failure, Secret watch for cert-manager rotation
  6. e2e tests: 4 Chainsaw test suites

Test plan

Unit tests

  • CRD validation: TLS override accepted on postgres/clairpostgres, rejected on unsupported components
  • Certificate generation: valid chain, correct SANs, PEM encoding, ECDSA P-256
  • Connection strings: DB_URI and Clair connstring with/without TLS, existing params preserved
  • Middleware: init container injection, volume mounts, ConfigMap patching, cleanup on disable
  • Validation: missing secret, missing keys, cert/key mismatch, expired cert

E2E tests (Chainsaw, all passing)

  • postgres_tls — self-signed TLS on both components, cert SANs/chain verification, SHOW ssl = on
  • postgres_tls_mixed — TLS on postgres only, clairpostgres uses sslmode=disable
  • postgres_tls_secretref — user-provided certs work; invalid secretRef sets RolloutBlocked
  • postgres_tls_lifecycle — enable TLS on existing deployment, verify SSL active, disable TLS, verify clean downgrade

Manual verification on OpenShift

  • Deploy with TLS, create user, create org, push image — all over encrypted DB connections
  • Disable TLS, verify clean downgrade, create org — works without TLS
  • Re-enable TLS, verify SSL back on, create org — works with TLS restored
  • SHOW ssl returns on/off correctly in each state
  • pg_stat_ssl confirms encrypted remote connections from Clair

Enhancement: quay/enhancements#41

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds per-component PostgreSQL TLS override support to the Quay operator: defines API types and CRD validation, implements secret validation and context wiring during reconciliation, generates self-signed ECDSA certificates, integrates TLS material into Kustomize inflation and managed secrets, injects TLS volumes/mounts/init-containers via middleware, and validates behavior across four comprehensive e2e test suites (basic, lifecycle, mixed, secretref, service-CA).

Changes

PostgreSQL TLS Override

Layer / File(s) Summary
API contract, CRD, and type validation
apis/quay/v1/quayregistry_types.go, apis/quay/v1/quayregistry_types_test.go, apis/quay/v1/zz_generated.deepcopy.go, bundle/manifests/quayregistries.crd.yaml, config/crd/bases/quay.redhat.com_quayregistries.yaml
TLSOverride type with Enabled bool and optional SecretRef; Override gains optional TLS field; supportsTLSOverride allowlist for postgres/clairpostgres; ValidateOverrides enforces tls.enabled=true required when secretRef set; ComponentSupportsOverride maps "tls" to allowlist; GetTLSOverrideForComponent helper; CRD XValidation CEL rules for secret/enabled coupling; generated deepcopy for TLSOverride; unit tests covering validation paths.
Reconciliation context, secret validation, and controller integration
pkg/context/context.go, controllers/quay/features.go, controllers/quay/features_test.go, controllers/quay/quayregistry_controller.go, controllers/quay/quayregistry_controller_test.go
QuayRegistryContext adds Postgres/Clair Postgres TLS CA/cert/key, service-CA flags, and SSL root-cert strings; checkManagedKeys and legacy path read TLS from managed secrets; resolvePostgresTLSSource determines service CA vs self-signed strategy; checkPostgresTLSSecrets validates referenced secrets (parses X.509, verifies expiry, checks cert/key pair); Reconcile calls validation and sets RolloutBlocked on error; findQuayRegistriesForSecret now watches component override secretRefs; comprehensive unit tests for context loading, secret validation, TLS source resolution, and service CA annotations.
Self-signed certificate generation
pkg/kustomize/certs.go, pkg/kustomize/certs_test.go
generatePostgresTLSCerts creates ECDSA P-256 CA and server certs, self-signed CA, server cert signed by CA, server DNS SANs (service, namespace variants, localhost), returns PEM-encoded CA/cert/key; helpers for SAN list, random serial, PEM encoding; tests verify CA self-signing, server cert chain validity, exact SANs, P-256 key curves, expiration windows, and unique serials.
Kustomize integration and DB_URI inflation
pkg/kustomize/kustomize.go, pkg/kustomize/kustomize_test.go
In KustomizationFor, conditionally generate self-signed TLS when managed and enabled without secretRef (skip if service CA); add TLS literals to managed-keys secret; optionally emit postgres-tls/postgresql-ca and clairpostgres-tls/clairpostgres-ca secrets. In Inflate, rewrite DB_URI host to FQDN when service CA enabled; set sslmode=verify-full + sslrootcert when TLS enabled; remove TLS params when managed but disabled. Tests validate cert properties, SAN verification, managed-keys persistence, secret generation, DB URI formatting, idempotency, and service CA paths.
Clair Postgres TLS configuration
pkg/kustomize/secrets.go, pkg/kustomize/secrets_test.go
Rewrite Clair Postgres host with service DNS suffix when service CA enabled; conditionally apply sslmode=verify-full + sslrootcert based on Clair Postgres TLS override, otherwise default sslmode=disable. Tests cover enabled/disabled TLS and service CA scenarios.
Deployment middleware TLS injection
pkg/middleware/middleware.go, pkg/middleware/middleware_test.go
Extend Process to inject TLS: applyPostgresTLS adds projected cert volumes, mounts, and init-container enabling SSL in postgresql.conf for postgres/clair-postgres deployments; applyClairDBTLS projects Clair Postgres CA for clair-app; update postgresql.conf.sample ConfigMaps with SSL directives; applyPostgresTLSCleanup removes SSL on disable; secret name derivation respects explicit secretRefs. Tests verify volume/mount injection, init-container logic, ConfigMap mutations, service CA skipping, and annotation handling.
E2E test suites and orchestration
test/chainsaw/Makefile, test/chainsaw/postgres_tls/*, test/chainsaw/postgres_tls_lifecycle/*, test/chainsaw/postgres_tls_mixed/*, test/chainsaw/postgres_tls_secretref/*, test/chainsaw/postgres_tls_service_ca/*
Four comprehensive Chainsaw test suites: (1) postgres-tls validates TLS enablement, cert generation, volume/init-container injection, secret content, cert chain validation, Clair DB TLS mounting, and runtime SHOW ssl/pg_stat_ssl verification; (2) postgres-tls-lifecycle exercises enable/disable transitions and SSL state changes; (3) postgres-tls-mixed tests selective component enablement (postgres yes, clairpostgres no) with secret and sslmode assertions; (4) postgres-tls-secretref validates user-provided secrets and invalid secret detection; (5) postgres-tls-service-ca tests OpenShift service CA integration. Makefile categorizes lifecycle and service-CA tests as destructive/platform-specific.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 23.53% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: adding TLS support for operator-managed PostgreSQL, which is the primary objective of the changeset.
Description check ✅ Passed The description is comprehensive and directly related to the changeset, explaining the feature, usage, options, and test coverage for PostgreSQL TLS support.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code

Comment @coderabbitai help to get the list of available commands and usage tips.

@bcaton85
bcaton85 force-pushed the PROJQUAY-11215-postgres-tls branch from 2bbfdb8 to 57dd677 Compare June 10, 2026 18:35

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (1)
pkg/middleware/middleware.go (1)

502-508: 💤 Low value

Consider simplifying the shell escaping for maintainability.

The nested quote escaping pattern '\"'\"' in the printf command is difficult to reason about and maintain. While e2e tests confirm this works correctly, consider refactoring to use a heredoc or simpler escaping pattern.

Alternative approach
 initContainer := corev1.Container{
 	Name:  "postgres-tls-init",
 	Image: dep.Spec.Template.Spec.Containers[0].Image,
 	Command: []string{
-		"sh", "-c",
-		"if [ -f /var/lib/pgsql/data/userdata/postgresql.conf ]; then " +
-			"grep -q '^ssl = on' /var/lib/pgsql/data/userdata/postgresql.conf || " +
-			"printf '\\nssl = on\\nssl_cert_file = '\"'\"'" + tlsCertsMountPath + "/tls.crt'\"'\"'\\nssl_key_file = '\"'\"'" + tlsCertsMountPath + "/tls.key'\"'\"'\\n' >> /var/lib/pgsql/data/userdata/postgresql.conf; " +
-			"fi",
+		"sh", "-c",
+		fmt.Sprintf(`
+if [ -f /var/lib/pgsql/data/userdata/postgresql.conf ]; then
+  grep -q '^ssl = on' /var/lib/pgsql/data/userdata/postgresql.conf || \
+  cat >> /var/lib/pgsql/data/userdata/postgresql.conf <<'EOF'
+
+ssl = on
+ssl_cert_file = '%s/tls.crt'
+ssl_key_file = '%s/tls.key'
+EOF
+fi`, tlsCertsMountPath, tlsCertsMountPath),
 	},
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/middleware/middleware.go` around lines 502 - 508, The shell command
constructing the postgresql.conf append is hard to read due to repeated
`'\"'\"'` escaping; update the Command in middleware.go that builds this sh -c
script (the Command slice containing the long printf and tlsCertsMountPath
usage) to use a simpler heredoc or simpler quoting approach: replace the printf
with a cat >> /var/lib/pgsql/data/userdata/postgresql.conf <<'EOF' ... EOF
pattern (or an equivalent single-quote heredoc) so the ssl lines and
tlsCertsMountPath are injected cleanly without nested quote escapes; ensure you
still guard with the existing if [ -f ... ] check and preserve the same appended
content (ssl = on, ssl_cert_file, ssl_key_file) and variable interpolation
semantics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apis/quay/v1/quayregistry_types.go`:
- Line 161: Update the kubebuilder XValidation rule that guards
overrides.tls.secretRef so that it also ensures secretRef.name is non-empty;
replace or augment the existing rule (the rule currently referencing
has(self.overrides.tls.secretRef) and self.overrides.tls.enabled) with a
condition that allows no secretRef or requires both tls.enabled to be true and
overrides.tls.secretRef.name != "" (e.g. !has(self.overrides) ||
!has(self.overrides.tls) || !has(self.overrides.tls.secretRef) ||
(self.overrides.tls.enabled && self.overrides.tls.secretRef.name != "")),
ensuring the validation fails fast when secretRef.name is empty.

In `@controllers/quay/features_test.go`:
- Around line 578-590: Replace the PEM-formatted private key literals used in
the test fixture with non-sensitive placeholders and update the corresponding
expectations: remove the fake PEM blocks assigned to "POSTGRES_TLS_KEY" and
"CLAIRPOSTGRES_TLS_KEY" and set them to simple innocuous strings (e.g.,
"fake-key" or "placeholder-key"); likewise update expectPostgresTLSKey and
expectClairPgTLSKey to match the new placeholder values so the test asserts
against the harmless string instead of PEM content. Ensure you modify the
variables in controllers/quay/features_test.go where "POSTGRES_TLS_KEY",
"CLAIRPOSTGRES_TLS_KEY", expectPostgresTLSKey, and expectClairPgTLSKey are
defined.

In `@controllers/quay/features.go`:
- Around line 737-765: The TLS secret validation currently only checks presence
of ca.crt and basic parsing of tls.crt/key; update the validation (around
secret.Data checks and the tls.X509KeyPair/pem.Decode/x509.ParseCertificate
code) to also parse ca.crt as PEM and X.509 (use pem.Decode then
x509.ParseCertificate or x509.ParseCertificates), build an x509.CertPool/Roots
from the CA bytes, and then verify the server certificate chain against that CA
and the expected DNS name(s) using cert.Verify with x509.VerifyOptions (set
Roots to the CA pool and DNSName to the appropriate host); for
Postgres/ClairPostgres when validating a DB secret derive the expected
hostname(s) (e.g. "<quay-name>-quay-database" and "<quay-name>-clair-postgres")
and return a clear error if the CA is malformed or the certificate does not
verify for the expected hostname(s).

In `@pkg/kustomize/kustomize.go`:
- Around line 784-805: The code currently removes sslmode/sslrootcert in the
else branch even when GetTLSOverrideForComponent(quay, v1.ComponentPostgres)
returns nil; change the logic so you only strip TLS params when an explicit
override exists and is disabled. Concretely, keep the existing branch that
applies verify-full when override != nil && override.Enabled, but replace the
else with an explicit check (override != nil && !override.Enabled) to remove
sslmode/sslrootcert; if override == nil do nothing (leave ctx.DbUri and
parsedUserConfig["DB_URI"] unchanged). Ensure you reference
GetTLSOverrideForComponent, ComponentPostgres, ctx.DbUri, and parsedUserConfig
when making the change.

In `@pkg/middleware/middleware.go`:
- Around line 526-527: Summary: The sed cleanup only removes directives at
column 0 and misses lines with leading whitespace; update the sed expressions in
the shell command string in middleware.go to match and delete lines with
optional leading whitespace (and optionally spaces around the '=' in the ssl
line). Locate the shell command string that contains "if [ -f
/var/lib/pgsql/data/userdata/postgresql.conf ] && grep -q '^ssl = on' ..." and
replace the deletion patterns (/^ssl = on$/d;/^ssl_cert_file/d;/^ssl_key_file/d)
with patterns that accept leading whitespace (e.g., use POSIX character class or
\s equivalent such as '^[[:space:]]*ssl = ' and '^[[:space:]]*ssl_cert_file' /
'^[[:space:]]*ssl_key_file') and enable extended regex if needed so the sed -i
invocation reliably deletes those lines regardless of indentation.
- Around line 623-625: The current idempotency check uses
strings.Contains(cm.Data[key], "ssl = on") which misses variants like "ssl=on"
or extra spaces; replace it with a whitespace-insensitive match (e.g., use a
regexp like `\bssl\s*=\s*on\b` and call regexp.MatchString or a precompiled
regexp's MatchString against cm.Data[key]) so the check correctly detects
existing SSL directives before returning and avoids appending duplicates;
reference the existing usage of cm.Data[key] and the strings.Contains call when
making the change.

---

Nitpick comments:
In `@pkg/middleware/middleware.go`:
- Around line 502-508: The shell command constructing the postgresql.conf append
is hard to read due to repeated `'\"'\"'` escaping; update the Command in
middleware.go that builds this sh -c script (the Command slice containing the
long printf and tlsCertsMountPath usage) to use a simpler heredoc or simpler
quoting approach: replace the printf with a cat >>
/var/lib/pgsql/data/userdata/postgresql.conf <<'EOF' ... EOF pattern (or an
equivalent single-quote heredoc) so the ssl lines and tlsCertsMountPath are
injected cleanly without nested quote escapes; ensure you still guard with the
existing if [ -f ... ] check and preserve the same appended content (ssl = on,
ssl_cert_file, ssl_key_file) and variable interpolation semantics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Enterprise

Run ID: 64e5b62b-e902-4ba9-b015-5f646c55ed91

📥 Commits

Reviewing files that changed from the base of the PR and between 328d886 and 57dd677.

📒 Files selected for processing (34)
  • apis/quay/v1/quayregistry_types.go
  • apis/quay/v1/quayregistry_types_test.go
  • apis/quay/v1/zz_generated.deepcopy.go
  • bundle/manifests/quayregistries.crd.yaml
  • config/crd/bases/quay.redhat.com_quayregistries.yaml
  • controllers/quay/features.go
  • controllers/quay/features_test.go
  • controllers/quay/quayregistry_controller.go
  • controllers/quay/quayregistry_controller_test.go
  • pkg/context/context.go
  • pkg/kustomize/certs.go
  • pkg/kustomize/certs_test.go
  • pkg/kustomize/kustomize.go
  • pkg/kustomize/kustomize_test.go
  • pkg/kustomize/secrets.go
  • pkg/kustomize/secrets_test.go
  • pkg/middleware/middleware.go
  • pkg/middleware/middleware_test.go
  • test/chainsaw/Makefile
  • test/chainsaw/postgres_tls/00-assert-status.yaml
  • test/chainsaw/postgres_tls/00-create-quay-registry.yaml
  • test/chainsaw/postgres_tls/chainsaw-test.yaml
  • test/chainsaw/postgres_tls_lifecycle/00-assert-status.yaml
  • test/chainsaw/postgres_tls_lifecycle/00-create-quay-registry.yaml
  • test/chainsaw/postgres_tls_lifecycle/01-enable-tls.yaml
  • test/chainsaw/postgres_tls_lifecycle/02-disable-tls.yaml
  • test/chainsaw/postgres_tls_lifecycle/chainsaw-test.yaml
  • test/chainsaw/postgres_tls_mixed/00-assert-status.yaml
  • test/chainsaw/postgres_tls_mixed/00-create-quay-registry.yaml
  • test/chainsaw/postgres_tls_mixed/chainsaw-test.yaml
  • test/chainsaw/postgres_tls_secretref/00-assert-status.yaml
  • test/chainsaw/postgres_tls_secretref/00-create-quay-registry.yaml
  • test/chainsaw/postgres_tls_secretref/01-create-invalid-registry.yaml
  • test/chainsaw/postgres_tls_secretref/chainsaw-test.yaml
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • quay/quay (manual)

Comment thread apis/quay/v1/quayregistry_types.go
Comment thread controllers/quay/features_test.go Outdated
Comment thread controllers/quay/features.go
Comment thread pkg/kustomize/kustomize.go
Comment thread pkg/middleware/middleware.go Outdated
Comment thread pkg/middleware/middleware.go Outdated
@codecov-commenter

codecov-commenter commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.96377% with 83 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.35%. Comparing base (58aeb85) to head (bf34bab).
⚠️ Report is 21 commits behind head on master.

Files with missing lines Patch % Lines
controllers/quay/features.go 82.89% 26 Missing ⚠️
pkg/middleware/middleware.go 83.33% 24 Missing ⚠️
controllers/quay/quayregistry_controller.go 48.38% 16 Missing ⚠️
pkg/kustomize/certs.go 83.58% 11 Missing ⚠️
pkg/kustomize/kustomize.go 96.03% 5 Missing ⚠️
apis/quay/v1/quayregistry_types.go 95.23% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1277      +/-   ##
==========================================
+ Coverage   62.13%   65.35%   +3.22%     
==========================================
  Files          25       26       +1     
  Lines        3784     4327     +543     
==========================================
+ Hits         2351     2828     +477     
- Misses       1433     1499      +66     
Flag Coverage Δ
unit-tests 65.35% <84.96%> (+3.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/context/context.go 0.00% <ø> (ø)
pkg/kustomize/secrets.go 87.57% <100.00%> (+0.38%) ⬆️
apis/quay/v1/quayregistry_types.go 41.48% <95.23%> (+2.69%) ⬆️
pkg/kustomize/kustomize.go 85.48% <96.03%> (+2.72%) ⬆️
pkg/kustomize/certs.go 83.58% <83.58%> (ø)
controllers/quay/quayregistry_controller.go 35.78% <48.38%> (-0.07%) ⬇️
pkg/middleware/middleware.go 72.25% <83.33%> (+10.43%) ⬆️
controllers/quay/features.go 71.82% <82.89%> (+6.98%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LiZhang19817

Copy link
Copy Markdown
Contributor

@bcaton85 pls review this critical bug when using unmanaged postgres database with TLS Cert.

  Critical Bug
  
  1. DB_URI sslmode stripped from unmanaged postgres connections

  pkg/kustomize/kustomize.go — the new else branch at the TLS check:

  if override := v1.GetTLSOverrideForComponent(quay, v1.ComponentPostgres); override != nil && override.Enabled {
      // add sslmode=verify-full ...
  } else {
      // STRIPS sslmode and sslrootcert from ALL DB_URIs
      q.Del("sslmode")
      q.Del("sslrootcert")
  }

  This runs unconditionally — including for unmanaged postgres where the user provides their own DB_URI with sslmode=verify-full to
  connect to an external TLS-enabled PostgreSQL. Since overrides can't be set on unmanaged components, GetTLSOverrideForComponent
  always returns nil, and the else branch strips the user's TLS params.
  
  Impact: Any existing deployment with unmanaged postgres using TLS connections to external databases would have sslmode and
  sslrootcert silently removed from their DB_URI, breaking database connections on the first reconcile after operator upgrade.

  Fix: Guard the else branch with a ComponentIsManaged check:
  } else if v1.ComponentIsManaged(quay.Spec.Components, v1.ComponentPostgres) {
      // only strip TLS params from managed postgres DB_URI

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
pkg/middleware/middleware_test.go (2)

905-952: ⚡ Quick win

Add assertion for FSGroup=26.

The implementation sets FSGroup=26 in the pod security context to allow PostgreSQL (UID 26) to read the private key, as mentioned in the commit message. The test should verify this critical configuration.

✅ Suggested assertion

Add after line 920 or before the volume checks:

 		assert.Equal(t, "quay.io/sclorg/postgresql-13-c9s:latest", dep.Spec.Template.Spec.InitContainers[0].Image)
+
+		// Verify FSGroup is set for PostgreSQL to read private key
+		assert.NotNil(t, dep.Spec.Template.Spec.SecurityContext)
+		assert.NotNil(t, dep.Spec.Template.Spec.SecurityContext.FSGroup)
+		assert.Equal(t, int64(26), *dep.Spec.Template.Spec.SecurityContext.FSGroup)
 
 		// Check projected volume added with correct mode
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/middleware/middleware_test.go` around lines 905 - 952, The test "injects
init container and volume when TLS enabled" is missing an assertion that the pod
security context FSGroup is set to 26; update the test after calling
applyPostgresTLS(quay, dep, v1.ComponentPostgres) (or before the volume checks)
to inspect dep.Spec.Template.Spec.SecurityContext (or PodSecurityContext) and
assert that SecurityContext.FSGroup (or PodSecurityContext.FSGroup) is non-nil
and equals int64(26), referencing the dep variable and the applyPostgresTLS
function to locate the code to modify.

1076-1087: ⚡ Quick win

Add assertion for FSGroup=26 in integration test.

Since Process() internally calls applyPostgresTLS, which sets FSGroup=26, this integration test should verify that the pod security context is correctly configured.

✅ Suggested assertion

Add after line 1078 or before the volume check:

 	assert.Equal(t, "postgres-tls-init", processed.Spec.Template.Spec.InitContainers[0].Name)
+
+	// Verify FSGroup is set for PostgreSQL to read private key
+	assert.NotNil(t, processed.Spec.Template.Spec.SecurityContext)
+	assert.NotNil(t, processed.Spec.Template.Spec.SecurityContext.FSGroup)
+	assert.Equal(t, int64(26), *processed.Spec.Template.Spec.SecurityContext.FSGroup)
 
 	// Projected volume added
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/middleware/middleware_test.go` around lines 1076 - 1087, The integration
test currently checks init containers and volumes but misses asserting that the
pod security context FSGroup is set by applyPostgresTLS; update the test in
middleware_test.go (the test that calls Process()) to fetch
processed.Spec.Template.Spec.SecurityContext and assert that
SecurityContext.FSGroup != nil and equals 26 (or use pointer comparison to 26),
ensuring Process()'s call to applyPostgresTLS results in FSGroup=26 being
applied.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/middleware/middleware_test.go`:
- Around line 905-952: The test "injects init container and volume when TLS
enabled" is missing an assertion that the pod security context FSGroup is set to
26; update the test after calling applyPostgresTLS(quay, dep,
v1.ComponentPostgres) (or before the volume checks) to inspect
dep.Spec.Template.Spec.SecurityContext (or PodSecurityContext) and assert that
SecurityContext.FSGroup (or PodSecurityContext.FSGroup) is non-nil and equals
int64(26), referencing the dep variable and the applyPostgresTLS function to
locate the code to modify.
- Around line 1076-1087: The integration test currently checks init containers
and volumes but misses asserting that the pod security context FSGroup is set by
applyPostgresTLS; update the test in middleware_test.go (the test that calls
Process()) to fetch processed.Spec.Template.Spec.SecurityContext and assert that
SecurityContext.FSGroup != nil and equals 26 (or use pointer comparison to 26),
ensuring Process()'s call to applyPostgresTLS results in FSGroup=26 being
applied.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Enterprise

Run ID: a7062542-99d4-4d3a-81a1-163a2d87cae8

📥 Commits

Reviewing files that changed from the base of the PR and between af7f3b9 and effd300.

📒 Files selected for processing (2)
  • pkg/middleware/middleware.go
  • pkg/middleware/middleware_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • quay/quay (manual)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/middleware/middleware.go

@bcaton85 bcaton85 changed the title Add TLS support for operator-managed PostgreSQL (PROJQUAY-11215) PROJQUAY-11215: feat: add TLS support for operator-managed PostgreSQL Jun 11, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

@bcaton85: This pull request references [Jira Issue PROJQUAY-11215](https://redhat.atlassian.net/browse/PROJQUAY-11215), which is invalid:

  • expected the feature to target the "quay-v3.18.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Adds opt-in TLS encryption for operator-managed PostgreSQL instances used by Quay Registry and Clair, enabling customers in regulated industries to meet encryption-in-transit requirements without provisioning external database infrastructure.

What this enables

spec:
components:

  • kind: postgres
    managed: true
    overrides:
    tls:
    enabled: true

  • kind: clairpostgres
    managed: true
    overrides:
    tls:
    enabled: true

  • PostgreSQL pods are configured with ssl = on and serve TLS connections

  • Quay and Clair connection strings are updated with sslmode=verify-full

  • Self-signed certificates (ECDSA P-256, 10yr) are generated automatically, or users can provide their own via secretRef (compatible with cert-manager)

  • Existing deployments are unaffected — TLS is only enabled when explicitly configured

Certificate options

  • Self-signed (tls.enabled: true) — Operator generates ECDSA P-256 certs automatically. Best for quick start and dev/test.
  • User-provided (tls.secretRef.name: my-certs) — Reference a Secret containing ca.crt, tls.crt, tls.key. For enterprise PKI integration.
  • cert-manager (tls.secretRef.name: cert-manager-secret) — Point secretRef at a cert-manager-populated Secret. Operator watches for changes and triggers reconciliation on
    rotation.

Commits

  1. apis: TLSOverride struct on the Override type, CEL validation, restricted to postgres/clairpostgres
  2. kustomize (certs): ECDSA P-256 cert generation, persisted in managed keys Secret
  3. kustomize (connection strings): sslmode=verify-full on Quay DB_URI and Clair connstring, CA Secret generation for both self-signed and secretRef modes
  4. middleware: Projected volume with defaultMode: 0600, init container for postgresql.conf patching (existing PVCs), ConfigMap patching (fresh PVCs), cleanup init container for downgrade
  5. controller: secretRef validation (key presence, cert/key match, expiry), RolloutBlocked on failure, Secret watch for cert-manager rotation
  6. e2e tests: 4 Chainsaw test suites

Test plan

Unit tests

  • CRD validation: TLS override accepted on postgres/clairpostgres, rejected on unsupported components
  • Certificate generation: valid chain, correct SANs, PEM encoding, ECDSA P-256
  • Connection strings: DB_URI and Clair connstring with/without TLS, existing params preserved
  • Middleware: init container injection, volume mounts, ConfigMap patching, cleanup on disable
  • Validation: missing secret, missing keys, cert/key mismatch, expired cert

E2E tests (Chainsaw, all passing)

  • postgres_tls — self-signed TLS on both components, cert SANs/chain verification, SHOW ssl = on
  • postgres_tls_mixed — TLS on postgres only, clairpostgres uses sslmode=disable
  • postgres_tls_secretref — user-provided certs work; invalid secretRef sets RolloutBlocked
  • postgres_tls_lifecycle — enable TLS on existing deployment, verify SSL active, disable TLS, verify clean downgrade

Manual verification on OpenShift

  • Deploy with TLS, create user, create org, push image — all over encrypted DB connections
  • Disable TLS, verify clean downgrade, create org — works without TLS
  • Re-enable TLS, verify SSL back on, create org — works with TLS restored
  • SHOW ssl returns on/off correctly in each state
  • pg_stat_ssl confirms encrypted remote connections from Clair

Enhancement: quay/enhancements#41

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml (1)

141-145: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Assert SSL_COUNT instead of only logging it.

Lines 141-145 compute and print SSL remote connection count, but the step never fails when count is 0, so the “verify-pg-stat-ssl” check can pass without proving client TLS usage.

Suggested fix
           SSL_COUNT=$(kubectl exec -n $NAMESPACE "${PG_POD}" -- \
             bash -c "psql -t -c \"SELECT count(*) FROM pg_stat_ssl s JOIN pg_stat_activity a ON s.pid = a.pid WHERE s.ssl = true AND a.client_addr IS NOT NULL;\"" \
             2>/dev/null | tr -d ' \n')
-          echo "SSL remote connections: ${SSL_COUNT:-0}"
+          if ! [[ "${SSL_COUNT}" =~ ^[0-9]+$ ]]; then
+            echo "FAIL: invalid SSL connection count: '${SSL_COUNT}'"
+            exit 1
+          fi
+          if [ "${SSL_COUNT}" -lt 1 ]; then
+            echo "FAIL: expected at least one SSL-encrypted remote connection"
+            exit 1
+          fi
+          echo "PASS: SSL remote connections: ${SSL_COUNT}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml` around lines 141 -
145, The script computes SSL_COUNT but only echoes it; update the block around
SSL_COUNT (the kubectl exec "psql ... SELECT count(*) ..." call and the echo
"SSL remote connections: ${SSL_COUNT:-0}") to assert the value is >0: after
assigning SSL_COUNT, add a conditional that treats ${SSL_COUNT:-0} == 0 as a
failure (echo a clear error like "No SSL remote connections detected" and exit
1) so the verify-pg-stat-ssl step fails when no client TLS connections are
present.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml`:
- Around line 124-133: The script checks SHOW ssl by kubectl exec into the
Postgres pod (PG_POD) immediately after waiting for quay-app rollout; insert an
explicit readiness gate for Postgres before the kubectl exec: after computing
PG_POD, wait for that pod to reach the Ready condition (use kubectl wait
--for=condition=ready on the PG_POD with an appropriate timeout) or wait for the
Postgres deployment/StatefulSet to finish rollout, then proceed to run the psql
SHOW ssl check; update the block around PG_POD, kubectl wait, and the kubectl
exec to ensure the pod is ready.

---

Outside diff comments:
In `@test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml`:
- Around line 141-145: The script computes SSL_COUNT but only echoes it; update
the block around SSL_COUNT (the kubectl exec "psql ... SELECT count(*) ..." call
and the echo "SSL remote connections: ${SSL_COUNT:-0}") to assert the value is
>0: after assigning SSL_COUNT, add a conditional that treats ${SSL_COUNT:-0} ==
0 as a failure (echo a clear error like "No SSL remote connections detected" and
exit 1) so the verify-pg-stat-ssl step fails when no client TLS connections are
present.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Enterprise

Run ID: 2ed38e34-2ac8-44e9-8a5b-7db0684a8946

📥 Commits

Reviewing files that changed from the base of the PR and between effd300 and 8bd4030.

📒 Files selected for processing (14)
  • controllers/quay/features.go
  • controllers/quay/features_test.go
  • controllers/quay/quayregistry_controller.go
  • pkg/context/context.go
  • pkg/kustomize/kustomize.go
  • pkg/kustomize/kustomize_test.go
  • pkg/kustomize/secrets.go
  • pkg/kustomize/secrets_test.go
  • pkg/middleware/middleware.go
  • pkg/middleware/middleware_test.go
  • test/chainsaw/Makefile
  • test/chainsaw/postgres_tls_service_ca/00-assert-status.yaml
  • test/chainsaw/postgres_tls_service_ca/00-create-quay-registry.yaml
  • test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • quay/quay (manual)
✅ Files skipped from review due to trivial changes (1)
  • test/chainsaw/postgres_tls_service_ca/00-assert-status.yaml
🚧 Files skipped from review as they are similar to previous changes (7)
  • pkg/context/context.go
  • pkg/kustomize/secrets.go
  • pkg/kustomize/secrets_test.go
  • test/chainsaw/Makefile
  • controllers/quay/quayregistry_controller.go
  • pkg/kustomize/kustomize_test.go
  • pkg/kustomize/kustomize.go

Comment on lines +124 to +133
kubectl rollout status deployment/pgsca-quay-app -n $NAMESPACE --timeout=300s
echo "quay-app rollout complete"

PG_POD=$(kubectl get pods -n $NAMESPACE -l quay-component=postgres \
-o jsonpath='{.items[0].metadata.name}')
echo "Postgres pod: ${PG_POD}"

echo "Checking SHOW ssl..."
SSL_STATUS=$(kubectl exec -n $NAMESPACE "${PG_POD}" -- \
bash -c "psql -t -c 'SHOW ssl;'" 2>/dev/null | tr -d ' \n')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Wait for postgres rollout before running psql checks.

Line 124 waits for pgsca-quay-app, but Lines 127-133 immediately exec into a postgres pod without an explicit postgres readiness gate. This can introduce intermittent failures in slower clusters.

Suggested fix
-          kubectl rollout status deployment/pgsca-quay-app -n $NAMESPACE --timeout=300s
+          kubectl rollout status deployment/pgsca-quay-database -n $NAMESPACE --timeout=300s
+          kubectl rollout status deployment/pgsca-quay-app -n $NAMESPACE --timeout=300s
           echo "quay-app rollout complete"

           PG_POD=$(kubectl get pods -n $NAMESPACE -l quay-component=postgres \
             -o jsonpath='{.items[0].metadata.name}')
+          if [ -z "${PG_POD}" ]; then
+            echo "FAIL: unable to locate postgres pod"
+            exit 1
+          fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
kubectl rollout status deployment/pgsca-quay-app -n $NAMESPACE --timeout=300s
echo "quay-app rollout complete"
PG_POD=$(kubectl get pods -n $NAMESPACE -l quay-component=postgres \
-o jsonpath='{.items[0].metadata.name}')
echo "Postgres pod: ${PG_POD}"
echo "Checking SHOW ssl..."
SSL_STATUS=$(kubectl exec -n $NAMESPACE "${PG_POD}" -- \
bash -c "psql -t -c 'SHOW ssl;'" 2>/dev/null | tr -d ' \n')
kubectl rollout status deployment/pgsca-quay-database -n $NAMESPACE --timeout=300s
kubectl rollout status deployment/pgsca-quay-app -n $NAMESPACE --timeout=300s
echo "quay-app rollout complete"
PG_POD=$(kubectl get pods -n $NAMESPACE -l quay-component=postgres \
-o jsonpath='{.items[0].metadata.name}')
if [ -z "${PG_POD}" ]; then
echo "FAIL: unable to locate postgres pod"
exit 1
fi
echo "Postgres pod: ${PG_POD}"
echo "Checking SHOW ssl..."
SSL_STATUS=$(kubectl exec -n $NAMESPACE "${PG_POD}" -- \
bash -c "psql -t -c 'SHOW ssl;'" 2>/dev/null | tr -d ' \n')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/chainsaw/postgres_tls_service_ca/chainsaw-test.yaml` around lines 124 -
133, The script checks SHOW ssl by kubectl exec into the Postgres pod (PG_POD)
immediately after waiting for quay-app rollout; insert an explicit readiness
gate for Postgres before the kubectl exec: after computing PG_POD, wait for that
pod to reach the Ready condition (use kubectl wait --for=condition=ready on the
PG_POD with an appropriate timeout) or wait for the Postgres
deployment/StatefulSet to finish rollout, then proceed to run the psql SHOW ssl
check; update the block around PG_POD, kubectl wait, and the kubectl exec to
ensure the pod is ready.

Comment thread pkg/kustomize/certs.go Outdated
Comment thread pkg/kustomize/certs.go Outdated
Comment thread pkg/kustomize/kustomize.go Outdated
Comment thread pkg/kustomize/kustomize.go
@bcaton85

Copy link
Copy Markdown
Member Author

/retest

1 similar comment
@bcaton85

Copy link
Copy Markdown
Member Author

/retest

@fullsend-ai-review

fullsend-ai-review Bot commented Jun 17, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:15 PM UTC · Completed 5:31 PM UTC
Commit: c4be77f · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [unreachable-watch-code] controllers/quay/quayregistry_controller.go:1458findQuayRegistriesForSecret was updated to match postgres TLS secrets via overrides.tls.secretRef. The Secret watch uses Watches() with externalTLSSecretPredicate (data-change filter, not a label filter), so the watch triggers correctly for user-provided TLS secrets. However, operator-generated secrets (e.g., <name>-postgres-tls from self-signed cert generation) are not matched by the findQuayRegistriesForSecret handler since they lack a secretRef. These secrets rely on the Owns(&corev1.Secret{}) watch via owner references, which should work correctly, but the code path in findQuayRegistriesForSecret for matching TLS secretRef names is only exercised for user-provided secretRef scenarios.

Low

  • [missing-serial-number] pkg/kustomize/certs.go:27 — The x509.Certificate templates (both CA and server) do not explicitly set SerialNumber. Go 1.20+ (this project uses Go 1.25.7) auto-generates a cryptographically random serial when SerialNumber is nil, so this works correctly. However, setting it explicitly would make the intent clearer and avoid reliance on implicit runtime behavior.

  • [incomplete-validation] controllers/quay/features.go:745 — In checkPostgresTLSSecrets, validation parses the CA cert and server cert independently and verifies cert/key pair match via tls.X509KeyPair, but does not verify that the server certificate is actually signed by the provided CA. A user could provide a valid cert/key pair with an unrelated CA. At runtime, PostgreSQL clients using verify-full would fail to connect (fail-closed), but this would cause a confusing runtime failure rather than a clear validation error at configuration time.
    Remediation: Add cert.Verify(x509.VerifyOptions{Roots: caPool}) to confirm the server cert chains to the provided CA.

  • [pattern-mismatch] pkg/middleware/middleware.go:530 — The TLS-enable init container's grep pattern uses '^ssl = on' (exact match), while the cleanup init container uses '^[[:space:]]*ssl[[:space:]]*=[[:space:]]*on' (whitespace-flexible). If postgresql.conf uses whitespace variants, the enable grep would fail to detect the existing directive and append a duplicate. The patterns should be consistent.

  • [unnecessary-work] pkg/middleware/middleware.go:545 — When TLS is not enabled, applyPostgresTLSCleanup unconditionally injects a cleanup init container into every postgres deployment. For deployments that never had TLS enabled, this does a no-op file check but still adds an init container to every reconcile cycle.

  • [edge-case] pkg/kustomize/kustomize.go:671 — The DB_URI is constructed via fmt.Sprintf without URL-encoding the password. When the new TLS code calls url.Parse(ctx.DbUri) to add SSL query parameters, a password containing @, /, #, ? or other URL-special characters would cause the URL to be misparsed. This is a pre-existing issue amplified by the new TLS code path that parses and reconstructs the URI.

  • [volume-permissions] pkg/middleware/middleware.go:497 — The postgres-tls-certs projected volume uses defaultMode: 0640. This is necessary for sclorg PostgreSQL images where the container runs as an arbitrary uid in the root group (gid 0). The permission model is correct for single-container pods.

  • [secret-handling] pkg/kustomize/kustomize.go:503 — PostgreSQL TLS private keys stored as literal sources in the managed-keys Secret, consistent with existing patterns for DB_URI and passwords.

  • [stale-doc] docs/environment-overrides.md — This document covers environment variable overrides specifically. TLS override documentation should be added somewhere in the docs directory.

  • [incomplete-doc] docs/components.md — The components overview does not document any override types (not just TLS). A comprehensive overrides guide with TLS configuration examples would benefit users.

  • [missing-doc] config/samples/overrides.quayregistry.yaml — The sample shows only storageClassName overrides. Adding commented-out TLS override examples would help discoverability.

  • [api-design] apis/quay/v1/quayregistry_types.go:34TLSOverride.Enabled is a required field (no omitempty), requiring explicit enabled: false when tls: {} is specified. This is intentional for security-sensitive features.

Info

  • [naming-convention] controllers/quay/features.go:543 — Constant names postgresTLSCA, postgresTLSCert follow standard Go conventions for abbreviations, consistent with existing codebase patterns.

  • [missing-rotation] pkg/kustomize/certs.go:14 — Self-signed certificates are generated with a 10-year validity period and no rotation mechanism. Consider documenting this limitation or adding a rotation check in future work.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Jun 17, 2026
bcaton85 and others added 9 commits June 17, 2026 16:00
Add TLS configuration support to the QuayRegistry CRD for managed
PostgreSQL components. The TLSOverride struct provides an opt-in
`enabled` field and an optional `secretRef` for user-provided
certificates (including cert-manager integration).

Restricted to postgres and clairpostgres components via CEL
validation rules. Existing deployments are unaffected as the
field defaults to nil.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add self-signed certificate generation for managed PostgreSQL TLS
using ECDSA P-256 with 10-year validity. Generated certificates
include SANs for all in-cluster DNS names and are persisted in
the managed keys Secret to survive reconcile loops.

Each component (postgres, clairpostgres) gets its own CA so TLS
can be enabled, disabled, and rotated independently.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire TLS into database connection strings when enabled:

- Quay DB_URI: merge sslmode=verify-full and sslrootcert via
  net/url.Parse, preserving existing query params and passwords.
  Strip TLS params on disable for clean downgrade.
- Clair connstring: conditional sslmode=verify-full with
  sslrootcert, replacing the hardcoded sslmode=disable.
- Generate postgresql-ca and clairpostgres-ca Secrets for both
  self-signed and secretRef modes so projected volumes on
  quay-app, mirror, and upgrade jobs can mount the CA cert.
- Move cert generation before secret list construction so certs
  are available for both managed keys persistence and CA Secret
  generation on first reconcile after TLS is enabled.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…UAY-11215)

Add TLS support to PostgreSQL and Clair deployments via middleware:

- applyPostgresTLS: mount TLS certs via projected volume with
  mode 0600, add init container to patch postgresql.conf on
  existing PVCs.
- applyPostgresTLSCleanup: strip SSL directives from
  postgresql.conf when TLS is disabled (downgrade path).
- applyPostgresConfSampleTLS: inject SSL directives into the
  postgres-conf-sample ConfigMap so fresh PVCs get SSL from
  initdb.
- applyClairDBTLS: mount CA cert volume on clair-app for
  sslrootcert verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…1215)

Add secretRef validation during reconciliation before applying
any configuration changes to deployments:

- Verify referenced Secret exists and contains required keys
  (ca.crt, tls.crt, tls.key) in PEM format.
- Validate cert/key pair matches and cert is not expired.
- Set RolloutBlocked condition with specific error messages on
  validation failure, preserving the existing deployment state.
- Extend Secret watch to trigger reconciliation when secretRef
  Secrets change, enabling cert-manager rotation workflows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 4 Chainsaw E2E test suites covering all PostgreSQL TLS
scenarios:

- postgres_tls: self-signed TLS on both postgres and
  clairpostgres — verifies init containers, volumes, secrets,
  certificate SANs/chain, clair-db-tls volume, and SHOW ssl.
- postgres_tls_mixed: TLS on postgres only — verifies
  clairpostgres has no TLS volume and uses sslmode=disable.
- postgres_tls_secretref: user-provided certs via secretRef and
  invalid secretRef — verifies RolloutBlocked with ConfigInvalid.
- postgres_tls_lifecycle: enable/disable TLS migration on a live
  deployment — verifies SSL activates after enable and clean
  downgrade after disable. Excluded from non-destructive runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… (PROJQUAY-11215)

The else branch unconditionally stripped sslmode and sslrootcert from
DB_URI, including unmanaged postgres where users provide their own
connection string with TLS params. Guard the stripping with both an
override != nil check and a ComponentIsManaged check so unmanaged
postgres DB_URIs are left untouched.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add CEL validation rule and Go-level check to reject empty
tls.secretRef.name at admission time rather than failing later
during reconciliation when the secret lookup fails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…1215)

Use non-PEM placeholder strings for TLS key test fixtures to avoid
tripping leak scanners. The test validates passthrough, not PEM format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bcaton85 and others added 10 commits June 17, 2026 16:00
…JQUAY-11215)

Parse ca.crt through pem.Decode and x509.ParseCertificate to catch
malformed CA certificates at validation time instead of failing at
PostgreSQL connection time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…-11215)

Update the sed cleanup init container and ConfigMap idempotency check
to handle optional leading whitespace in postgresql.conf SSL directives
using POSIX character classes and a compiled regex respectively.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ROJQUAY-11215)

The projected volume for TLS certs was mounted with mode 0600 owned by
root, which works on OpenShift (SCC injects fsGroup) but fails on KinD
where the postgres process runs as UID 26 with no group ownership.

Change defaultMode to 0640 and explicitly set fsGroup=26 on the pod
security context so PostgreSQL can read the key file via group
permissions. PostgreSQL accepts root-owned key files with mode 0640
when the database user is in the files group.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AY-11215)

When running on OpenShift with tls.enabled and no secretRef, the operator
now annotates the postgres Service with service.beta.openshift.io/serving-
cert-secret-name instead of generating self-signed certificates. The
existing cluster-service-ca ConfigMap provides the CA trust chain. Falls
back to self-signed certs on vanilla Kubernetes. Per-component (postgres,
clairpostgres) with independent detection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On OpenShift the restricted SCC rejects explicit fsGroup: 26, but the
SCC auto-assigns an fsGroup from the namespace range making cert files
group-readable. On vanilla Kubernetes (KinD) there is no SCC so fsGroup
must be set explicitly. Gate the fsGroup assignment on SupportsRoutes
being false.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On OpenShift the operator uses service CA instead of self-signed certs,
so the CA comes from the cluster-service-ca ConfigMap rather than
operator-generated postgresql-ca secrets. Update the assert-tls-secrets,
assert-cert-sans-and-chain, and assert-clair-db-tls steps to detect
OpenShift and verify the correct CA source on each platform.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…JQUAY-11215)

Remove manual serial number generation for CA and server certificates —
Go 1.23+ auto-generates RFC 5280-compliant serials when SerialNumber is
nil. Fix trailing colon in DB_URI host when user-provided URI omits port
by defaulting to 5432.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace `grep -q` with `grep >/dev/null` in platform detection pipes.
Under `set -euo pipefail`, `grep -q` exits immediately on match, sending
SIGPIPE to the still-writing `kubectl api-resources` process. The SIGPIPE
exit code (141) becomes the pipeline result, causing the test to
incorrectly fall into the non-OpenShift branch on OpenShift clusters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…JQUAY-11215)

On OpenShift the CA is provided via a cluster-service-ca ConfigMap, not
a postgresql-ca secret. The mixed test unconditionally checked for the
secret, failing on OpenShift. Add platform detection to check the
ConfigMap on OpenShift and the secret on KinD. Apply the same fix to the
lifecycle test for consistency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bcaton85
bcaton85 force-pushed the PROJQUAY-11215-postgres-tls branch from c4be77f to 1a2a88c Compare June 17, 2026 20:00
@openshift-merge-bot
openshift-merge-bot Bot merged commit e2d43c5 into quay:master Jun 18, 2026
20 checks passed
@openshift-ci-robot

openshift-ci-robot commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

@bcaton85: An error was encountered updating to the MODIFIED state for bug PROJQUAY-11215 on the Jira server at https://redhat.atlassian.net. No known errors were detected, please see the full error message for details.

Full error message. No transition status with name `MODIFIED` could be found. Please select from the following list: [New Planning In Progress Closed]

Please contact an administrator to resolve this issue, then request a bug refresh with /jira refresh.

Details

In response to this:

Adds opt-in TLS encryption for operator-managed PostgreSQL instances used by Quay Registry and Clair, enabling customers in regulated industries to meet encryption-in-transit requirements without provisioning external database infrastructure.

What this enables

spec:
components:

  • kind: postgres
    managed: true
    overrides:
    tls:
    enabled: true

  • kind: clairpostgres
    managed: true
    overrides:
    tls:
    enabled: true

  • PostgreSQL pods are configured with ssl = on and serve TLS connections

  • Quay and Clair connection strings are updated with sslmode=verify-full

  • Self-signed certificates (ECDSA P-256, 10yr) are generated automatically, or users can provide their own via secretRef (compatible with cert-manager)

  • Existing deployments are unaffected — TLS is only enabled when explicitly configured

Certificate options

  • Self-signed (tls.enabled: true) — Operator generates ECDSA P-256 certs automatically. Best for quick start and dev/test.
  • User-provided (tls.secretRef.name: my-certs) — Reference a Secret containing ca.crt, tls.crt, tls.key. For enterprise PKI integration.
  • cert-manager (tls.secretRef.name: cert-manager-secret) — Point secretRef at a cert-manager-populated Secret. Operator watches for changes and triggers reconciliation on
    rotation.

Commits

  1. apis: TLSOverride struct on the Override type, CEL validation, restricted to postgres/clairpostgres
  2. kustomize (certs): ECDSA P-256 cert generation, persisted in managed keys Secret
  3. kustomize (connection strings): sslmode=verify-full on Quay DB_URI and Clair connstring, CA Secret generation for both self-signed and secretRef modes
  4. middleware: Projected volume with defaultMode: 0600, init container for postgresql.conf patching (existing PVCs), ConfigMap patching (fresh PVCs), cleanup init container for downgrade
  5. controller: secretRef validation (key presence, cert/key match, expiry), RolloutBlocked on failure, Secret watch for cert-manager rotation
  6. e2e tests: 4 Chainsaw test suites

Test plan

Unit tests

  • CRD validation: TLS override accepted on postgres/clairpostgres, rejected on unsupported components
  • Certificate generation: valid chain, correct SANs, PEM encoding, ECDSA P-256
  • Connection strings: DB_URI and Clair connstring with/without TLS, existing params preserved
  • Middleware: init container injection, volume mounts, ConfigMap patching, cleanup on disable
  • Validation: missing secret, missing keys, cert/key mismatch, expired cert

E2E tests (Chainsaw, all passing)

  • postgres_tls — self-signed TLS on both components, cert SANs/chain verification, SHOW ssl = on
  • postgres_tls_mixed — TLS on postgres only, clairpostgres uses sslmode=disable
  • postgres_tls_secretref — user-provided certs work; invalid secretRef sets RolloutBlocked
  • postgres_tls_lifecycle — enable TLS on existing deployment, verify SSL active, disable TLS, verify clean downgrade

Manual verification on OpenShift

  • Deploy with TLS, create user, create org, push image — all over encrypted DB connections
  • Disable TLS, verify clean downgrade, create org — works without TLS
  • Re-enable TLS, verify SSL back on, create org — works with TLS restored
  • SHOW ssl returns on/off correctly in each state
  • pg_stat_ssl confirms encrypted remote connections from Clair

Enhancement: quay/enhancements#41

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@fullsend-ai-retro

fullsend-ai-retro Bot commented Jun 18, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 5:11 PM UTC · Completed 5:20 PM UTC
Commit: bf34bab · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #1277 — PostgreSQL TLS support

Timeline: PR opened June 10 by bcaton85 (+4165/−15, 37 files). CodeRabbit reviewed immediately (June 10–12, 3 rounds). Human reviewer jbpratt reviewed June 15, approved June 16. Fullsend review agent ran June 17 — after human approval — posted 1 medium + 7 low + 5 info findings and applied requires-manual-review. PR merged June 18.

What went well: The human review process worked effectively. CodeRabbit caught early issues (secretRef validation, PEM handling), the human reviewer caught practical bugs (trailing colon in DB_URI, unnecessary serial number generation), and the author addressed feedback promptly across 20 commits.

Review agent observations: The fullsend review ran late (after human approval), so its findings didn't influence the review process. It applied requires-manual-review to an already-approved PR, which is semantically incorrect. The agent's findings were reasonable for a 4000+ line PR but missed the practical DB_URI trailing-colon bug that the human caught.

Existing issue coverage: The key improvement opportunities identified here are already tracked:

No new proposals filed — existing issues adequately cover the identified gaps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

5 participants