Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
167 changes: 39 additions & 128 deletions docs/spec/v1/kustomizations.md
Original file line number Diff line number Diff line change
Expand Up @@ -1430,24 +1430,29 @@ Kustomization object itself, it will fall back to these defaults.

See also the [workload identity](/flux/installation/configuration/workload-identity/) docs.

#### AWS KMS
#### Cloud Provider KMS Services

While making use of the [IAM OIDC provider](https://eksctl.io/usage/iamserviceaccounts/)
on your EKS cluster, you can create an IAM Role and Service Account with access
to AWS KMS (using at least `kms:Decrypt` and `kms:DescribeKey`). Once these are
created, you can annotate the kustomize-controller Service Account with the
Role ARN, granting the controller permission to decrypt the Secrets. Please refer
to the [SOPS guide](https://fluxcd.io/flux/guides/mozilla-sops/#aws) for detailed steps.
For cloud provider KMS services, please refer to the specific sections in the integration guides:

```sh
kubectl -n flux-system annotate serviceaccount kustomize-controller \
--field-manager=flux-client-side-apply \
eks.amazonaws.com/role-arn='arn:aws:iam::<ACCOUNT_ID>:role/<KMS-ROLE-NAME>'
```
Service-specific configuration:

- [AWS KMS](https://fluxcd.io/flux/integrations/aws/#for-amazon-key-management-service)
- [Azure Key Vault](https://fluxcd.io/flux/integrations/azure/#for-azure-key-vault)
- [GCP KMS](https://fluxcd.io/flux/integrations/gcp/#for-google-cloud-key-management-service)

Controller-level configuration:

- [AWS](https://fluxcd.io/flux/integrations/aws/#at-the-controller-level)
- [Azure](https://fluxcd.io/flux/integrations/azure/#at-the-controller-level)
- [GCP](https://fluxcd.io/flux/integrations/gcp/#at-the-controller-level)

Furthermore, you can also use the usual [environment variables used for specifying AWS
credentials](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list),
by patching the kustomize-controller Deployment:
These guides provide detailed instructions for setting up authentication,
permissions, and controller configuration for each cloud provider.

#### Hashicorp Vault

To configure a global default for Hashicorp Vault, patch the controller's
Deployment with a `VAULT_TOKEN` environment variable.

```yaml
---
Expand All @@ -1462,128 +1467,32 @@ spec:
containers:
- name: manager
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-creds
key: awsAccessKeyID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-creds
key: awsSecretAccessKey
- name: AWS_SESSION_TOKEN
valueFrom:
secretKeyRef:
name: aws-creds
key: awsSessionToken
```

In addition to this, the
[general SOPS documentation around KMS AWS applies](https://github.com/mozilla/sops#27kms-aws-profiles),
allowing you to specify e.g. a `SOPS_KMS_ARN` environment variable.

**Note:**: If you are mounting a secret containing the AWS credentials as a
file in the `kustomize-controller` Pod, you need to specify an environment
variable `$HOME`, since the AWS credentials file is expected to be present at
`~/.aws`. For example:

```yaml
env:
- name: HOME
value: /home/{$USER}
- name: VAULT_TOKEN
value: <token>
```

#### Azure Key Vault

##### Workload Identity
#### SOPS Age Keys

If you have Workload Identity set up on your AKS cluster, you can establish
a federated identity between the kustomize-controller ServiceAccount and an
identity that has "Decrypt" role on the Azure Key Vault. Once, this is done
you can label and annotate the kustomize-controller ServiceAccount and Pod
with the patch shown below:
To configure global decryption for SOPS Age keys, use the `--sops-age-secret`
controller flag to specify a Kubernetes Secret containing the Age private keys.

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
- patch: |-
apiVersion: v1
kind: ServiceAccount
metadata:
name: kustomize-controller
namespace: flux-system
annotations:
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
labels:
azure.workload.identity/use: "true"
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: kustomize-controller
namespace: flux-system
labels:
azure.workload.identity/use: "true"
spec:
template:
metadata:
labels:
azure.workload.identity/use: "true"
```

##### Kubelet Identity

If the kubelet managed identity has `Decrypt` permissions on Azure Key Vault,
no additional configuration is required for the kustomize-controller to decrypt
data.

#### GCP KMS

While making use of Google Cloud Platform, the [`GOOGLE_APPLICATION_CREDENTIALS`
environment variable](https://cloud.google.com/docs/authentication/production)
is automatically taken into account.
[Granting permissions](https://cloud.google.com/kms/docs/reference/permissions-and-roles)
to the Service Account attached to this will therefore be sufficient to decrypt
data. When running outside GCP, it is possible to manually patch the
kustomize-controller Deployment with a valid set of (mounted) credentials.
First, create a Secret containing the Age private keys with the `.agekey` suffix:

```yaml
---
apiVersion: apps/v1
kind: Deployment
apiVersion: v1
kind: Secret
metadata:
name: kustomize-controller
name: sops-age-keys
namespace: flux-system
spec:
template:
spec:
containers:
- name: manager
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /var/gcp/credentials.json
volumeMounts:
- name: gcp-credentials
mountPath: /var/gcp/
readOnly: true
volumes:
- name: gcp-credentials
secret:
secretName: mysecret
items:
- key: credentials
path: credentials.json
stringData:
identity1.agekey: <identity1 key>
identity2.agekey: <identity1 key>
```

#### Hashicorp Vault
The Secret must be in the same namespace as the kustomize-controller Deployment.

To configure a global default for Hashicorp Vault, patch the controller's
Deployment with a `VAULT_TOKEN` environment variable.
Then, patch the kustomize-controller Deployment to add the `--sops-age-secret` flag:

```yaml
---
Expand All @@ -1597,11 +1506,13 @@ spec:
spec:
containers:
- name: manager
env:
- name: VAULT_TOKEN
value: <token>
args:
- --sops-age-secret=sops-age-keys
```

The field `.spec.decryption.secretRef` in the Kustomization will take precedence
in case both the controller flag and the Kustomization field are set.

### Kustomize secretGenerator

SOPS encrypted data can be stored as a base64 encoded Secret, which enables the
Expand Down
15 changes: 14 additions & 1 deletion internal/controller/kustomization_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ import (
intcache "github.com/fluxcd/kustomize-controller/internal/cache"
"github.com/fluxcd/kustomize-controller/internal/decryptor"
"github.com/fluxcd/kustomize-controller/internal/inventory"
intruntime "github.com/fluxcd/kustomize-controller/internal/runtime"
)

// +kubebuilder:rbac:groups=kustomize.toolkit.fluxcd.io,resources=kustomizations,verbs=get;list;watch;create;update;patch;delete
Expand Down Expand Up @@ -104,6 +105,7 @@ type KustomizationReconciler struct {
NoRemoteBases bool
FailFast bool
DefaultServiceAccount string
SOPSAgeSecret string
KubeConfigOpts runtimeClient.KubeConfigOptions
ConcurrentSSA int
DisallowedFieldManagers []string
Expand Down Expand Up @@ -642,7 +644,18 @@ func (r *KustomizationReconciler) generate(obj unstructured.Unstructured,
func (r *KustomizationReconciler) build(ctx context.Context,
obj *kustomizev1.Kustomization, u unstructured.Unstructured,
workDir, dirPath string) ([]byte, error) {
dec, cleanup, err := decryptor.NewTempDecryptor(workDir, r.Client, obj, r.TokenCache)

// Build decryptor.
decryptorOpts := []decryptor.Option{
decryptor.WithRoot(workDir),
}
if r.TokenCache != nil {
decryptorOpts = append(decryptorOpts, decryptor.WithTokenCache(*r.TokenCache))
}
if name, ns := r.SOPSAgeSecret, intruntime.Namespace(); name != "" && ns != "" {
decryptorOpts = append(decryptorOpts, decryptor.WithSOPSAgeSecret(name, ns))
}
dec, cleanup, err := decryptor.New(r.Client, obj, decryptorOpts...)
if err != nil {
return nil, err
}
Expand Down
Loading