Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions docs/pod-secrets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# Injecting OpenBao Secrets into Pods

This guide explains how to inject secrets from OpenBao into Kubernetes pods using the OpenBao agent injector.

## Prerequisites

The `openbao-injector` component must be installed:

```bash
foundry component install openbao-injector
```

This deploys the OpenBao agent injector and registers a `MutatingWebhookConfiguration` that intercepts pod creation. Pods annotated with `vault.hashicorp.com/agent-inject: "true"` automatically receive a sidecar that mounts secrets from OpenBao before the main container starts.

## How It Works

1. You annotate a pod/deployment/cronjob with the secrets it needs
2. The injector sidecar fetches those secrets from OpenBao at pod startup
3. Secrets are written as files to `/vault/secrets/`
4. Your container sources those files to load them as environment variables

## OpenBao Setup

### Enable Kubernetes Auth

The injector authenticates pods against OpenBao using the Kubernetes service account JWT. Enable this once:

```bash
vault auth enable kubernetes

vault write auth/kubernetes/config \
kubernetes_host="https://$(kubectl get svc kubernetes -o jsonpath='{.spec.clusterIP}'):443"
```

### Create a Policy

Define what secrets a pod is allowed to read:

```bash
vault policy write my-app - <<EOF
path "secret/data/apps/my-app" {
capabilities = ["read"]
}
EOF
```

### Create a Role

Bind the policy to a Kubernetes service account:

```bash
vault write auth/kubernetes/role/my-app \
bound_service_account_names=default \
bound_service_account_namespaces=my-namespace \
policies=my-app \
ttl=1h
```

## Annotating Pods

Add annotations to your pod spec to request secret injection. The injector creates one file per secret at `/vault/secrets/<name>`.

### Basic Example

```yaml
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "my-app"

# Inject secret/data/apps/my-app as /vault/secrets/my-app
vault.hashicorp.com/agent-inject-secret-my-app: "secret/data/apps/my-app"

# Template the file as shell exports so the container can source it
vault.hashicorp.com/agent-inject-template-my-app: |
{{- with secret "secret/data/apps/my-app" -}}
export DB_PASSWORD="{{ .Data.data.password }}"
export API_KEY="{{ .Data.data.api_key }}"
{{- end }}
```

### Container Entrypoint

Because the secrets are shell files (not environment variables), the container must source them before starting:

```yaml
containers:
- name: my-app
command: ["/bin/sh", "-c"]
args: [". /vault/secrets/my-app && exec my-binary"]
```

> **Note**: Use `.` (dot) not `source` — slim/alpine images use `dash` as `/bin/sh` which does not support `source`.

## Helm Chart Pattern

For a Helm chart with multiple secrets (e.g. reddit-watcher):

```yaml
# In your cronjob/deployment template
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "{{ .Values.vault.role }}"

vault.hashicorp.com/agent-inject-secret-db: "{{ .Values.vault.dbPath }}"
vault.hashicorp.com/agent-inject-template-db: |
{{`{{- with secret "`}}{{ .Values.vault.dbPath }}{{`" -}}`}}
{{`export POSTGRES_URL="{{ .Data.data.postgres_url }}"`}}
{{`{{- end }}`}}

vault.hashicorp.com/agent-inject-secret-api: "{{ .Values.vault.apiPath }}"
vault.hashicorp.com/agent-inject-template-api: |
{{`{{- with secret "`}}{{ .Values.vault.apiPath }}{{`" -}}`}}
{{`export API_KEY="{{ .Data.data.key }}"`}}
{{`{{- end }}`}}

spec:
containers:
- name: my-app
command: ["/bin/sh", "-c"]
args: [". /vault/secrets/db && . /vault/secrets/api && exec python -m main"]
```

## Storing Secrets in OpenBao

```bash
# Store secrets (run from your local machine with VAULT_ADDR set)
vault kv put secret/apps/my-app \
password="my-db-password" \
api_key="my-api-key"

# Add a key to an existing secret without overwriting others
vault kv patch secret/apps/my-app new_key="new-value"

# Using 1Password to inject the value at write time
op run -- vault kv patch secret/apps/my-app \
postgres_url="op://pedro/POSTGRES_URL/credential"
```

## Troubleshooting

### `/vault/secrets/<name>: No such file`

The injector sidecar didn't run. Check:

1. Is `openbao-injector` installed?
```bash
foundry component status openbao-injector
kubectl get mutatingwebhookconfigurations | grep openbao
```

2. Does the pod have `vault.hashicorp.com/agent-inject: "true"` annotation?

3. Is the pod in a namespace the webhook targets? By default it targets all namespaces.

### `permission denied` fetching secrets

The Kubernetes auth role doesn't bind to this pod's service account or namespace:

```bash
# Verify the role bindings
vault read auth/kubernetes/role/my-app
```

Ensure `bound_service_account_namespaces` includes the pod's namespace.

### `source: not found`

Using `source` instead of `.` in a `dash`-based image. Replace:
```sh
# Wrong (bash only)
source /vault/secrets/my-app

# Correct (POSIX sh / dash compatible)
. /vault/secrets/my-app
```
12 changes: 10 additions & 2 deletions v1/cmd/foundry/commands/component/install.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"github.com/catalystcommunity/foundry/v1/internal/component/loki"
"github.com/catalystcommunity/foundry/v1/internal/component/seaweedfs"
"github.com/catalystcommunity/foundry/v1/internal/component/openbao"
"github.com/catalystcommunity/foundry/v1/internal/component/openbaoinjector"
"github.com/catalystcommunity/foundry/v1/internal/component/prometheus"
componentStorage "github.com/catalystcommunity/foundry/v1/internal/component/storage"
"github.com/catalystcommunity/foundry/v1/internal/component/velero"
Expand Down Expand Up @@ -109,8 +110,9 @@ var k8sComponents = map[string]bool{
"prometheus": true,
"loki": true,
"grafana": true,
"external-dns": true,
"velero": true,
"external-dns": true,
"velero": true,
"openbao-injector": true,
}

func runInstall(ctx context.Context, cmd *cli.Command) error {
Expand Down Expand Up @@ -296,6 +298,12 @@ func installK8sComponent(ctx context.Context, cmd *cli.Command, name string, sta
cfg["s3_bucket"] = "velero"
cfg["s3_region"] = "us-east-1"
componentWithClients = velero.NewComponent(helmClient, k8sClient)
case "openbao-injector":
// Inject the OpenBao address so the webhook knows where to reach it
if addr, err := stackConfig.GetPrimaryOpenBAOAddress(); err == nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if GetPrimaryOpenBAOAddress() fails, the error is dropped and the user later gets a generic "external_vault_addr is required" message with no indication why auto-detection failed. Please surface the underlying error.

cfg["external_vault_addr"] = fmt.Sprintf("http://%s:8200", addr)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmt.Sprintf("http://%s:8200", addr) will break silently with TLS or non-default ports. Try deriving protocol/port from config rather than hardcoding. The rest of the stack should already have that in config.

}
componentWithClients = openbaoinjector.NewComponent(helmClient)
default:
return fmt.Errorf("unknown kubernetes component: %s", name)
}
Expand Down
10 changes: 10 additions & 0 deletions v1/cmd/foundry/registry/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/catalystcommunity/foundry/v1/internal/component/loki"
"github.com/catalystcommunity/foundry/v1/internal/component/seaweedfs"
"github.com/catalystcommunity/foundry/v1/internal/component/openbao"
"github.com/catalystcommunity/foundry/v1/internal/component/openbaoinjector"
"github.com/catalystcommunity/foundry/v1/internal/component/prometheus"
"github.com/catalystcommunity/foundry/v1/internal/component/storage"
"github.com/catalystcommunity/foundry/v1/internal/component/velero"
Expand Down Expand Up @@ -43,6 +44,15 @@ func InitComponents() error {
return err
}

// Register OpenBao agent injector - depends on OpenBAO and K3s
// Installs the MutatingWebhookConfiguration so pods can receive secrets
// from OpenBao via vault.hashicorp.com/agent-inject annotations
openbaoInjectorComp := openbaoinjector.NewComponent(nil)
if err := component.Register(openbaoInjectorComp); err != nil {
return err
}


// Register Gateway API - depends on K3s
// Gateway API CRDs are installed as a cluster-level feature, independent of ingress controllers
gatewayAPIComp := gatewayapi.NewComponent(nil)
Expand Down
113 changes: 113 additions & 0 deletions v1/internal/component/openbaoinjector/install.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
package openbaoinjector

import (
"context"
"fmt"
"time"

"github.com/catalystcommunity/foundry/v1/internal/helm"
)

const (
// DefaultNamespace is the namespace where the injector will be installed
DefaultNamespace = "openbao"

// ReleaseName is the Helm release name
ReleaseName = "openbao-injector"

// Helm repo constants
repoName = "openbao"
repoURL = "https://openbao.github.io/openbao-helm"
chart = "openbao/openbao"

installTimeout = 5 * time.Minute
)

// Install installs the OpenBao agent injector using Helm.
// Only the injector is deployed (server.enabled=false). The injector registers
// a MutatingWebhookConfiguration so that pods with vault.hashicorp.com/agent-inject
// annotations automatically get secrets mounted from OpenBao.
func Install(ctx context.Context, helmClient HelmClient, cfg *Config) error {
if helmClient == nil {
return fmt.Errorf("helm client cannot be nil")
}
if cfg == nil {
return fmt.Errorf("config cannot be nil")
}

// Add OpenBao Helm repository
if err := helmClient.AddRepo(ctx, helm.RepoAddOptions{
Name: repoName,
URL: repoURL,
ForceUpdate: true,
}); err != nil {
return fmt.Errorf("failed to add openbao helm repo: %w", err)
}

values := buildHelmValues(cfg)

// Check if release already exists
releases, err := helmClient.List(ctx, cfg.Namespace)
if err == nil {
for _, rel := range releases {
if rel.Name == ReleaseName {
if rel.Status == "deployed" {
fmt.Println(" Upgrading existing OpenBao injector deployment...")
return helmClient.Upgrade(ctx, helm.UpgradeOptions{
ReleaseName: ReleaseName,
Namespace: cfg.Namespace,
Chart: chart,
Version: cfg.Version,
Values: values,
Wait: true,
Timeout: installTimeout,
})
}
// Failed/pending release — uninstall and reinstall
fmt.Printf(" Removing failed release (status: %s)...\n", rel.Status)
if err := helmClient.Uninstall(ctx, helm.UninstallOptions{
ReleaseName: ReleaseName,
Namespace: cfg.Namespace,
}); err != nil {
return fmt.Errorf("failed to remove existing release: %w", err)
}
break
}
}
}

if err := helmClient.Install(ctx, helm.InstallOptions{
ReleaseName: ReleaseName,
Namespace: cfg.Namespace,
Chart: chart,
Version: cfg.Version,
Values: values,
CreateNamespace: true,
Wait: true,
Timeout: installTimeout,
}); err != nil {
return fmt.Errorf("failed to install openbao injector: %w", err)
}

fmt.Printf(" ✓ OpenBao agent injector installed\n")
fmt.Printf(" ✓ MutatingWebhookConfiguration registered\n")
fmt.Printf(" Pods annotated with vault.hashicorp.com/agent-inject=true will now\n")
fmt.Printf(" automatically receive secrets from OpenBao at %s\n", cfg.ExternalVaultAddr)

return nil
}

// buildHelmValues constructs the Helm values for injector-only installation.
// server.enabled=false — we already have OpenBao running on the host.
// injector.enabled=true — this is the only thing we're installing.
func buildHelmValues(cfg *Config) map[string]interface{} {
return map[string]interface{}{
"server": map[string]interface{}{
"enabled": false,
},
"injector": map[string]interface{}{
"enabled": true,
"externalVaultAddr": cfg.ExternalVaultAddr,
},
}
}
11 changes: 11 additions & 0 deletions v1/internal/component/openbaoinjector/types.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading