-
Notifications
You must be signed in to change notification settings - Fork 1
feat: add OpenBao agent injector component #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| # Injecting OpenBao Secrets into Pods | ||
|
|
||
| This guide explains how to inject secrets from OpenBao into Kubernetes pods using the OpenBao agent injector. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| The `openbao-injector` component must be installed: | ||
|
|
||
| ```bash | ||
| foundry component install openbao-injector | ||
| ``` | ||
|
|
||
| This deploys the OpenBao agent injector and registers a `MutatingWebhookConfiguration` that intercepts pod creation. Pods annotated with `vault.hashicorp.com/agent-inject: "true"` automatically receive a sidecar that mounts secrets from OpenBao before the main container starts. | ||
|
|
||
| ## How It Works | ||
|
|
||
| 1. You annotate a pod/deployment/cronjob with the secrets it needs | ||
| 2. The injector sidecar fetches those secrets from OpenBao at pod startup | ||
| 3. Secrets are written as files to `/vault/secrets/` | ||
| 4. Your container sources those files to load them as environment variables | ||
|
|
||
| ## OpenBao Setup | ||
|
|
||
| ### Enable Kubernetes Auth | ||
|
|
||
| The injector authenticates pods against OpenBao using the Kubernetes service account JWT. Enable this once: | ||
|
|
||
| ```bash | ||
| vault auth enable kubernetes | ||
|
|
||
| vault write auth/kubernetes/config \ | ||
| kubernetes_host="https://$(kubectl get svc kubernetes -o jsonpath='{.spec.clusterIP}'):443" | ||
| ``` | ||
|
|
||
| ### Create a Policy | ||
|
|
||
| Define what secrets a pod is allowed to read: | ||
|
|
||
| ```bash | ||
| vault policy write my-app - <<EOF | ||
| path "secret/data/apps/my-app" { | ||
| capabilities = ["read"] | ||
| } | ||
| EOF | ||
| ``` | ||
|
|
||
| ### Create a Role | ||
|
|
||
| Bind the policy to a Kubernetes service account: | ||
|
|
||
| ```bash | ||
| vault write auth/kubernetes/role/my-app \ | ||
| bound_service_account_names=default \ | ||
| bound_service_account_namespaces=my-namespace \ | ||
| policies=my-app \ | ||
| ttl=1h | ||
| ``` | ||
|
|
||
| ## Annotating Pods | ||
|
|
||
| Add annotations to your pod spec to request secret injection. The injector creates one file per secret at `/vault/secrets/<name>`. | ||
|
|
||
| ### Basic Example | ||
|
|
||
| ```yaml | ||
| spec: | ||
| template: | ||
| metadata: | ||
| annotations: | ||
| vault.hashicorp.com/agent-inject: "true" | ||
| vault.hashicorp.com/role: "my-app" | ||
|
|
||
| # Inject secret/data/apps/my-app as /vault/secrets/my-app | ||
| vault.hashicorp.com/agent-inject-secret-my-app: "secret/data/apps/my-app" | ||
|
|
||
| # Template the file as shell exports so the container can source it | ||
| vault.hashicorp.com/agent-inject-template-my-app: | | ||
| {{- with secret "secret/data/apps/my-app" -}} | ||
| export DB_PASSWORD="{{ .Data.data.password }}" | ||
| export API_KEY="{{ .Data.data.api_key }}" | ||
| {{- end }} | ||
| ``` | ||
|
|
||
| ### Container Entrypoint | ||
|
|
||
| Because the secrets are shell files (not environment variables), the container must source them before starting: | ||
|
|
||
| ```yaml | ||
| containers: | ||
| - name: my-app | ||
| command: ["/bin/sh", "-c"] | ||
| args: [". /vault/secrets/my-app && exec my-binary"] | ||
| ``` | ||
|
|
||
| > **Note**: Use `.` (dot) not `source` — slim/alpine images use `dash` as `/bin/sh` which does not support `source`. | ||
|
|
||
| ## Helm Chart Pattern | ||
|
|
||
| For a Helm chart with multiple secrets (e.g. reddit-watcher): | ||
|
|
||
| ```yaml | ||
| # In your cronjob/deployment template | ||
| metadata: | ||
| annotations: | ||
| vault.hashicorp.com/agent-inject: "true" | ||
| vault.hashicorp.com/role: "{{ .Values.vault.role }}" | ||
|
|
||
| vault.hashicorp.com/agent-inject-secret-db: "{{ .Values.vault.dbPath }}" | ||
| vault.hashicorp.com/agent-inject-template-db: | | ||
| {{`{{- with secret "`}}{{ .Values.vault.dbPath }}{{`" -}}`}} | ||
| {{`export POSTGRES_URL="{{ .Data.data.postgres_url }}"`}} | ||
| {{`{{- end }}`}} | ||
|
|
||
| vault.hashicorp.com/agent-inject-secret-api: "{{ .Values.vault.apiPath }}" | ||
| vault.hashicorp.com/agent-inject-template-api: | | ||
| {{`{{- with secret "`}}{{ .Values.vault.apiPath }}{{`" -}}`}} | ||
| {{`export API_KEY="{{ .Data.data.key }}"`}} | ||
| {{`{{- end }}`}} | ||
|
|
||
| spec: | ||
| containers: | ||
| - name: my-app | ||
| command: ["/bin/sh", "-c"] | ||
| args: [". /vault/secrets/db && . /vault/secrets/api && exec python -m main"] | ||
| ``` | ||
|
|
||
| ## Storing Secrets in OpenBao | ||
|
|
||
| ```bash | ||
| # Store secrets (run from your local machine with VAULT_ADDR set) | ||
| vault kv put secret/apps/my-app \ | ||
| password="my-db-password" \ | ||
| api_key="my-api-key" | ||
|
|
||
| # Add a key to an existing secret without overwriting others | ||
| vault kv patch secret/apps/my-app new_key="new-value" | ||
|
|
||
| # Using 1Password to inject the value at write time | ||
| op run -- vault kv patch secret/apps/my-app \ | ||
| postgres_url="op://pedro/POSTGRES_URL/credential" | ||
| ``` | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### `/vault/secrets/<name>: No such file` | ||
|
|
||
| The injector sidecar didn't run. Check: | ||
|
|
||
| 1. Is `openbao-injector` installed? | ||
| ```bash | ||
| foundry component status openbao-injector | ||
| kubectl get mutatingwebhookconfigurations | grep openbao | ||
| ``` | ||
|
|
||
| 2. Does the pod have `vault.hashicorp.com/agent-inject: "true"` annotation? | ||
|
|
||
| 3. Is the pod in a namespace the webhook targets? By default it targets all namespaces. | ||
|
|
||
| ### `permission denied` fetching secrets | ||
|
|
||
| The Kubernetes auth role doesn't bind to this pod's service account or namespace: | ||
|
|
||
| ```bash | ||
| # Verify the role bindings | ||
| vault read auth/kubernetes/role/my-app | ||
| ``` | ||
|
|
||
| Ensure `bound_service_account_namespaces` includes the pod's namespace. | ||
|
|
||
| ### `source: not found` | ||
|
|
||
| Using `source` instead of `.` in a `dash`-based image. Replace: | ||
| ```sh | ||
| # Wrong (bash only) | ||
| source /vault/secrets/my-app | ||
|
|
||
| # Correct (POSIX sh / dash compatible) | ||
| . /vault/secrets/my-app | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,7 @@ import ( | |
| "github.com/catalystcommunity/foundry/v1/internal/component/loki" | ||
| "github.com/catalystcommunity/foundry/v1/internal/component/seaweedfs" | ||
| "github.com/catalystcommunity/foundry/v1/internal/component/openbao" | ||
| "github.com/catalystcommunity/foundry/v1/internal/component/openbaoinjector" | ||
| "github.com/catalystcommunity/foundry/v1/internal/component/prometheus" | ||
| componentStorage "github.com/catalystcommunity/foundry/v1/internal/component/storage" | ||
| "github.com/catalystcommunity/foundry/v1/internal/component/velero" | ||
|
|
@@ -109,8 +110,9 @@ var k8sComponents = map[string]bool{ | |
| "prometheus": true, | ||
| "loki": true, | ||
| "grafana": true, | ||
| "external-dns": true, | ||
| "velero": true, | ||
| "external-dns": true, | ||
| "velero": true, | ||
| "openbao-injector": true, | ||
| } | ||
|
|
||
| func runInstall(ctx context.Context, cmd *cli.Command) error { | ||
|
|
@@ -296,6 +298,12 @@ func installK8sComponent(ctx context.Context, cmd *cli.Command, name string, sta | |
| cfg["s3_bucket"] = "velero" | ||
| cfg["s3_region"] = "us-east-1" | ||
| componentWithClients = velero.NewComponent(helmClient, k8sClient) | ||
| case "openbao-injector": | ||
| // Inject the OpenBao address so the webhook knows where to reach it | ||
| if addr, err := stackConfig.GetPrimaryOpenBAOAddress(); err == nil { | ||
| cfg["external_vault_addr"] = fmt.Sprintf("http://%s:8200", addr) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| } | ||
| componentWithClients = openbaoinjector.NewComponent(helmClient) | ||
| default: | ||
| return fmt.Errorf("unknown kubernetes component: %s", name) | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| package openbaoinjector | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "time" | ||
|
|
||
| "github.com/catalystcommunity/foundry/v1/internal/helm" | ||
| ) | ||
|
|
||
| const ( | ||
| // DefaultNamespace is the namespace where the injector will be installed | ||
| DefaultNamespace = "openbao" | ||
|
|
||
| // ReleaseName is the Helm release name | ||
| ReleaseName = "openbao-injector" | ||
|
|
||
| // Helm repo constants | ||
| repoName = "openbao" | ||
| repoURL = "https://openbao.github.io/openbao-helm" | ||
| chart = "openbao/openbao" | ||
|
|
||
| installTimeout = 5 * time.Minute | ||
| ) | ||
|
|
||
| // Install installs the OpenBao agent injector using Helm. | ||
| // Only the injector is deployed (server.enabled=false). The injector registers | ||
| // a MutatingWebhookConfiguration so that pods with vault.hashicorp.com/agent-inject | ||
| // annotations automatically get secrets mounted from OpenBao. | ||
| func Install(ctx context.Context, helmClient HelmClient, cfg *Config) error { | ||
| if helmClient == nil { | ||
| return fmt.Errorf("helm client cannot be nil") | ||
| } | ||
| if cfg == nil { | ||
| return fmt.Errorf("config cannot be nil") | ||
| } | ||
|
|
||
| // Add OpenBao Helm repository | ||
| if err := helmClient.AddRepo(ctx, helm.RepoAddOptions{ | ||
| Name: repoName, | ||
| URL: repoURL, | ||
| ForceUpdate: true, | ||
| }); err != nil { | ||
| return fmt.Errorf("failed to add openbao helm repo: %w", err) | ||
| } | ||
|
|
||
| values := buildHelmValues(cfg) | ||
|
|
||
| // Check if release already exists | ||
| releases, err := helmClient.List(ctx, cfg.Namespace) | ||
| if err == nil { | ||
| for _, rel := range releases { | ||
| if rel.Name == ReleaseName { | ||
| if rel.Status == "deployed" { | ||
| fmt.Println(" Upgrading existing OpenBao injector deployment...") | ||
| return helmClient.Upgrade(ctx, helm.UpgradeOptions{ | ||
| ReleaseName: ReleaseName, | ||
| Namespace: cfg.Namespace, | ||
| Chart: chart, | ||
| Version: cfg.Version, | ||
| Values: values, | ||
| Wait: true, | ||
| Timeout: installTimeout, | ||
| }) | ||
| } | ||
| // Failed/pending release — uninstall and reinstall | ||
| fmt.Printf(" Removing failed release (status: %s)...\n", rel.Status) | ||
| if err := helmClient.Uninstall(ctx, helm.UninstallOptions{ | ||
| ReleaseName: ReleaseName, | ||
| Namespace: cfg.Namespace, | ||
| }); err != nil { | ||
| return fmt.Errorf("failed to remove existing release: %w", err) | ||
| } | ||
| break | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if err := helmClient.Install(ctx, helm.InstallOptions{ | ||
| ReleaseName: ReleaseName, | ||
| Namespace: cfg.Namespace, | ||
| Chart: chart, | ||
| Version: cfg.Version, | ||
| Values: values, | ||
| CreateNamespace: true, | ||
| Wait: true, | ||
| Timeout: installTimeout, | ||
| }); err != nil { | ||
| return fmt.Errorf("failed to install openbao injector: %w", err) | ||
| } | ||
|
|
||
| fmt.Printf(" ✓ OpenBao agent injector installed\n") | ||
| fmt.Printf(" ✓ MutatingWebhookConfiguration registered\n") | ||
| fmt.Printf(" Pods annotated with vault.hashicorp.com/agent-inject=true will now\n") | ||
| fmt.Printf(" automatically receive secrets from OpenBao at %s\n", cfg.ExternalVaultAddr) | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| // buildHelmValues constructs the Helm values for injector-only installation. | ||
| // server.enabled=false — we already have OpenBao running on the host. | ||
| // injector.enabled=true — this is the only thing we're installing. | ||
| func buildHelmValues(cfg *Config) map[string]interface{} { | ||
| return map[string]interface{}{ | ||
| "server": map[string]interface{}{ | ||
| "enabled": false, | ||
| }, | ||
| "injector": map[string]interface{}{ | ||
| "enabled": true, | ||
| "externalVaultAddr": cfg.ExternalVaultAddr, | ||
| }, | ||
| } | ||
| } |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if
GetPrimaryOpenBAOAddress()fails, the error is dropped and the user later gets a generic "external_vault_addr is required" message with no indication why auto-detection failed. Please surface the underlying error.