Skip to content

Commit d29ac21

Browse files
stvoutsinstvoutsin
committed
Change TAP configuration to allow using TAP_SCHEMA in CloudSQL
- Add update-tap-schema hook and required configuration into repertoire - Change structure of TAP to handle containerized, cloudsql & external databases - Restructure secrets handling for tap applications - Change TAP service apps to make cloudsql default type for uws and tap_schema for ssotap and tap - Keep tap_schema as containerized option on idfprod for now with uws as cloudsql - Use containerized uws and tap_schema for other non-idf envs - Upgrade repertoire to version 0.7.0
1 parent 5853e79 commit d29ac21

58 files changed

Lines changed: 910 additions & 280 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎applications/repertoire/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ description: Service discovery
66
home: "https://repertoire.lsst.io/"
77
sources:
88
- "https://github.com/lsst-sqre/repertoire"
9-
appVersion: 0.6.1
9+
appVersion: 0.7.0
1010

1111
annotations:
1212
phalanx.lsst.io/docs: |

‎applications/repertoire/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,13 @@ Service discovery
1313
| Key | Type | Default | Description |
1414
|-----|------|---------|-------------|
1515
| affinity | object | `{}` | Affinity rules for the repertoire deployment pod |
16+
| cloudsql.enabled | bool | `false` | Enable the Cloud SQL Auth Proxy, used with Cloud SQL databases on Google Cloud. This will be run as a sidecar for the main Gafaelfawr pods, and as a separate service (behind a `NetworkPolicy`) for other, lower-traffic services. |
17+
| cloudsql.image.pullPolicy | string | `"IfNotPresent"` | Pull policy for Cloud SQL Auth Proxy images |
18+
| cloudsql.image.repository | string | `"gcr.io/cloudsql-docker/gce-proxy"` | Cloud SQL Auth Proxy image to use |
19+
| cloudsql.image.tag | string | `"1.37.9"` | Cloud SQL Auth Proxy tag to use |
20+
| cloudsql.instanceConnectionName | string | None, must be set if Cloud SQL Auth Proxy is enabled | Instance connection name for a Cloud SQL PostgreSQL instance |
21+
| cloudsql.resources | object | See `values.yaml` | Resource limits and requests for the Cloud SQL Proxy container |
22+
| cloudsql.serviceAccount | string | None, must be set if Cloud SQL Auth Proxy is enabled | The Google service account that has an IAM binding to the `gafaelfawr` Kubernetes service account and has the `cloudsql.client` role |
1623
| config.applications | list | Set by Argo CD | List of applications deployed in this Phalanx environment (do not set) |
1724
| config.availableDatasets | list | `[]` | Datasets available in the Phalanx environment. This should be overridden by environments to list the datasets they provide. |
1825
| config.baseHostname | string | Set by Argo CD | Base hostname of the Phalanx environment (do not set) |
@@ -36,6 +43,9 @@ Service discovery
3643
| config.sentry.enabled | bool | `false` | Whether to enable the Sentry integration |
3744
| config.slackAlerts | bool | `false` | Whether to send Slack alerts for unexpected failures |
3845
| config.subdomainRules | object | See `values.yaml` | Rules for determining the expected URLs of deployed services that use a subdomain. See the [Repertoire documentation](https://repertoire.lsst.io/) for more information. |
46+
| config.tap.schemaSourceTemplate | string | lsst/sdm_schemas on GitHub | Template for schema artifact URLs (use {schemaVersion} placeholder) |
47+
| config.tap.schemaVersion | string | `"w.2025.49"` | Default schema version for all TAP services (can be overridden per-app) |
48+
| config.tap.servers | object | See `values.yaml` | TAP Server configuration by application name. Configuration is used to populate & update the TAP_SCHEMA database for each enabled TAP application |
3949
| config.useSubdomains | list | `[]` | List of services that use subdomains instead of the main hostname. See the [Repertoire documentation](https://repertoire.lsst.io/) for more information. |
4050
| global.environmentName | string | Set by Argo CD | Name of the Phalanx environment |
4151
| global.host | string | Set by Argo CD | Host name for ingress |
@@ -48,4 +58,5 @@ Service discovery
4858
| podAnnotations | object | `{}` | Annotations for the repertoire deployment pod |
4959
| replicaCount | int | `1` | Number of web deployment pods to start |
5060
| resources | object | See `values.yaml` | Resource limits and requests for the repertoire deployment pod |
61+
| schemaHook.resources | object | See `values.yaml` | Resource limits and requests for the TAP schema update Helm hook job |
5162
| tolerations | list | Tolerate GKE arm64 taint | Tolerations for the repertoire deployment pod |

‎applications/repertoire/secrets.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,27 @@
1+
tap-database-password:
2+
description: >-
3+
Database password for the tap service, used by repertoire to manage
4+
the tap TAP_SCHEMA. Copied from tap application's database-password.
5+
if: config.tap.servers.tap.enabled
6+
copy:
7+
application: tap
8+
key: tap-schema-password
9+
ssotap-database-password:
10+
description: >-
11+
Database password for the ssotap service, used by repertoire to manage
12+
the ssotap TAP_SCHEMA. Copied from ssotap application's database-password.
13+
if: config.tap.servers.ssotap.enabled
14+
copy:
15+
application: ssotap
16+
key: tap-schema-password
17+
livetap-database-password:
18+
description: >-
19+
Database password for the livetap service, used by repertoire to manage
20+
the livetap TAP_SCHEMA. Copied from livetap application's database-password.
21+
if: config.tap.servers.livetap.enabled
22+
copy:
23+
application: livetap
24+
key: tap-schema-password
125
sentry-dsn:
226
description: >-
327
DSN URL to which Sentry trace and error logging will be sent.

‎applications/repertoire/templates/_helpers.tpl‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,45 @@ Selector labels
2121
app.kubernetes.io/name: "repertoire"
2222
app.kubernetes.io/instance: {{ .Release.Name }}
2323
{{- end }}
24+
25+
{{/*
26+
Cloud SQL Auth Proxy sidecar container
27+
*/}}
28+
{{- define "repertoire.cloudsqlSidecar" -}}
29+
- name: "cloud-sql-proxy"
30+
command:
31+
- "/cloud_sql_proxy"
32+
- "-ip_address_types=PRIVATE"
33+
- "-log_debug_stdout=true"
34+
- "-structured_logs=true"
35+
- "-instances={{ required "cloudsql.instanceConnectionName must be specified" .Values.cloudsql.instanceConnectionName }}=tcp:5432"
36+
image: "{{ .Values.cloudsql.image.repository }}:{{ .Values.cloudsql.image.tag }}"
37+
imagePullPolicy: {{ .Values.cloudsql.image.pullPolicy | quote }}
38+
{{- with .Values.cloudsql.resources }}
39+
resources:
40+
{{- toYaml . | nindent 12 }}
41+
{{- end }}
42+
restartPolicy: "Always"
43+
securityContext:
44+
allowPrivilegeEscalation: false
45+
capabilities:
46+
drop:
47+
- "all"
48+
readOnlyRootFilesystem: true
49+
runAsNonRoot: true
50+
runAsUser: 65532
51+
runAsGroup: 65532
52+
{{- end }}
53+
54+
{{/*
55+
Check if any TAP server is enabled
56+
*/}}
57+
{{- define "repertoire.anyTapServerEnabled" -}}
58+
{{- $enabled := false -}}
59+
{{- range $app, $config := .Values.config.tap.servers -}}
60+
{{- if $config.enabled -}}
61+
{{- $enabled = true -}}
62+
{{- end -}}
63+
{{- end -}}
64+
{{- $enabled -}}
65+
{{- end -}}

‎applications/repertoire/templates/configmap-hook.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{{- if eq (include "repertoire.anyTapServerEnabled" .) "true" }}
2+
apiVersion: v1
3+
kind: ConfigMap
4+
metadata:
5+
name: "repertoire-hook-config"
6+
annotations:
7+
helm.sh/hook: "pre-install,pre-upgrade"
8+
helm.sh/hook-delete-policy: "before-hook-creation"
9+
labels:
10+
app.kubernetes.io/name: "repertoire"
11+
data:
12+
config.yaml: |
13+
logLevel: {{ .Values.config.logLevel | quote }}
14+
logProfile: {{ .Values.config.logProfile | quote }}
15+
baseHostname: {{ .Values.global.host | quote }}
16+
tap:
17+
schemaVersion: {{ .Values.config.tap.schemaVersion | quote }}
18+
schemaSourceTemplate: {{ .Values.config.tap.schemaSourceTemplate | quote }}
19+
servers: {{ .Values.config.tap.servers | toJson }}
20+
metrics:
21+
enabled: false
22+
application: "repertoire"
23+
sentry:
24+
enabled: false
25+
{{- end }}

‎applications/repertoire/templates/job-schema-update.yaml‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
{{- $root := . -}}
2+
{{- if eq (include "repertoire.anyTapServerEnabled" .) "true" }}
3+
{{- range $app, $config := .Values.config.tap.servers }}
4+
{{- if $config.enabled }}
5+
---
6+
apiVersion: batch/v1
7+
kind: Job
8+
metadata:
9+
name: "repertoire-{{ $app }}-schema-update"
10+
annotations:
11+
helm.sh/hook: "pre-install,pre-upgrade"
12+
helm.sh/hook-delete-policy: "before-hook-creation,hook-succeeded"
13+
helm.sh/hook-weight: "1"
14+
labels:
15+
app.kubernetes.io/component: "tap-schema-update"
16+
{{- include "repertoire.labels" $root | nindent 4 }}
17+
spec:
18+
template:
19+
metadata:
20+
{{- with $root.Values.podAnnotations }}
21+
annotations:
22+
{{- toYaml . | nindent 8 }}
23+
{{- end }}
24+
labels:
25+
{{- include "repertoire.selectorLabels" $root | nindent 8 }}
26+
app.kubernetes.io/component: "tap-schema-update"
27+
spec:
28+
{{- with $root.Values.affinity }}
29+
affinity:
30+
{{- toYaml . | nindent 8 }}
31+
{{- end }}
32+
{{- if $root.Values.cloudsql.enabled }}
33+
serviceAccountName: "repertoire"
34+
{{- else }}
35+
automountServiceAccountToken: false
36+
{{- end }}
37+
containers:
38+
- name: "tap-schema-update"
39+
command:
40+
- "repertoire"
41+
- "update-tap-schema"
42+
- "--app"
43+
- {{ $app | quote }}
44+
env:
45+
- name: "REPERTOIRE_CONFIG_PATH"
46+
value: "/etc/repertoire/config.yaml"
47+
- name: "REPERTOIRE_DATABASE_PASSWORD"
48+
valueFrom:
49+
secretKeyRef:
50+
name: "repertoire"
51+
key: {{ $config.databasePasswordKey | quote }}
52+
- name: "REPERTOIRE_DATABASE_USER"
53+
value: {{ $config.databaseUser | quote }}
54+
image: "{{ $root.Values.image.repository }}:{{ $root.Values.image.tag | default $root.Chart.AppVersion }}"
55+
imagePullPolicy: {{ $root.Values.image.pullPolicy | quote }}
56+
{{- with $root.Values.schemaHook.resources }}
57+
resources:
58+
{{- toYaml . | nindent 12 }}
59+
{{- end }}
60+
securityContext:
61+
allowPrivilegeEscalation: false
62+
capabilities:
63+
drop:
64+
- "all"
65+
readOnlyRootFilesystem: true
66+
volumeMounts:
67+
- name: "config"
68+
mountPath: "/etc/repertoire"
69+
readOnly: true
70+
- name: "tmp"
71+
mountPath: "/tmp"
72+
{{- if $root.Values.cloudsql.enabled }}
73+
initContainers:
74+
{{- include "repertoire.cloudsqlSidecar" $root | nindent 8 }}
75+
{{- end }}
76+
{{- with $root.Values.nodeSelector }}
77+
nodeSelector:
78+
{{- toYaml . | nindent 8 }}
79+
{{- end }}
80+
restartPolicy: "Never"
81+
securityContext:
82+
runAsNonRoot: true
83+
runAsUser: 1000
84+
runAsGroup: 1000
85+
{{- with $root.Values.tolerations }}
86+
tolerations:
87+
{{- toYaml . | nindent 8 }}
88+
{{- end }}
89+
volumes:
90+
- name: "tmp"
91+
emptyDir: {}
92+
- name: "config"
93+
configMap:
94+
name: "repertoire-hook-config"
95+
{{- end }}
96+
{{- end }}
97+
{{- end }}

‎applications/repertoire/templates/serviceaccount.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{{- if .Values.cloudsql.enabled }}
2+
apiVersion: v1
3+
kind: ServiceAccount
4+
metadata:
5+
name: "repertoire"
6+
labels:
7+
{{- include "repertoire.labels" . | nindent 4 }}
8+
annotations:
9+
helm.sh/hook: "pre-install,pre-upgrade"
10+
helm.sh/hook-delete-policy: "before-hook-creation"
11+
helm.sh/hook-weight: "0"
12+
iam.gke.io/gcp-service-account: {{ required "cloudsql.serviceAccount must be set to a valid Google service account" .Values.cloudsql.serviceAccount | quote }}
13+
{{- end }}

‎applications/repertoire/templates/vault-secrets.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
{{- if (or .Values.config.sentry.enabled .Values.config.slackAlerts .Values.config.influxdbDatabases) -}}
1+
{{- if (or .Values.config.sentry.enabled .Values.config.slackAlerts .Values.config.influxdbDatabases .Values.config.tapServers) -}}
22
apiVersion: ricoberger.de/v1alpha1
33
kind: VaultSecret
44
metadata:

‎applications/repertoire/values-idfdev.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,16 @@ config:
1616
metrics:
1717
enabled: true
1818
slackAlerts: true
19+
tap:
20+
servers:
21+
tap:
22+
enabled: true
23+
ssotap:
24+
enabled: true
1925
useSubdomains:
2026
- "nublado"
27+
28+
cloudsql:
29+
enabled: true
30+
instanceConnectionName: "science-platform-dev-7696:us-central1:science-platform-dev-e9e11de2"
31+
serviceAccount: "repertoire@science-platform-dev-7696.iam.gserviceaccount.com"

‎applications/repertoire/values-idfint.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,16 @@ config:
99
metrics:
1010
enabled: true
1111
slackAlerts: true
12+
tap:
13+
servers:
14+
tap:
15+
enabled: true
16+
ssotap:
17+
enabled: true
1218
useSubdomains:
1319
- "nublado"
20+
21+
cloudsql:
22+
enabled: true
23+
instanceConnectionName: "science-platform-int-dc5d:us-central1:science-platform-int-8f439af2"
24+
serviceAccount: "repertoire@science-platform-int-dc5d.iam.gserviceaccount.com"

0 commit comments

Comments
 (0)