Enable passing read only credentials for flux bootstrap secret creation

[MihailoPlavsic34]

Problem

Currently, flux bootstrap requires read-write (RW) credentials to commit manifests to the repository, and these same credentials are stored in the cluster's flux-system secret for ongoing sync operations. This creates a security concern where the cluster retains more permissions than necessary for runtime operations.

Current Workarounds

While it's possible to rotate the RW credential with 3rd party tools as I suggested in this comment, this requires post-bootstrap manipulation and doesn't follow the principle of least privilege from the start.

Proposed Solution

Add support for providing separate credentials during bootstrap:

• Bootstrap credentials: RW access for initial manifest commits
• Runtime credentials: Read-only access for ongoing sync operations

The bootstrap process would use RW credentials to commit manifests but store the read-only credentials in the cluster's flux-system secret.

Implementation

I'd love to take on implementing this feature. My understanding is this would require updates to:

1. Bootstrap package - Core logic for handling dual credentials
2. Bootstrap CLI commands - New flags for runtime credentials
3. Terraform provider - Corresponding terraform resource attributes

Proposed CLI Interface

# Example with separate credentials
flux bootstrap github \
  --owner=myorg \
  --repository=fleet-infra \
  --bootstrap-token=$GITHUB_TOKEN_RW \
  --runtime-token=$GITHUB_TOKEN_RO \
  --path=clusters/production

# Or with SSH
flux bootstrap git \
  --url=ssh://[email protected]/myorg/fleet-infra \
  --private-key-file=bootstrap-key.pem \
  --runtime-private-key-file=readonly-key.pem \
  --path=clusters/production
  
 # Open to suggestions, I suck at naming 😄 

[stefanprodan]

You can do this today with the CLI using 2 commands:

• flux create secret git flux-system (using a readonly PAT or SSH key)
• flux bootstrap

The bootstrap command will not override the flux-system Kubernetes Secret if it exists in the cluster, so the one passed to bootstrap will only be used to push the manifests.

The same procedure works with the Flux Terraform provider, you can create the Kubernetes Secret with the read-only PAT before running the bootstrap module.

This creates a security concern where the cluster retains more permissions than necessary for runtime operations.

Flux image automation needs write access to the repo and so for the majority of users having a read-only key will not suffice.

[MihailoPlavsic34]

Thanks for the reply @stefanprodan.

You can do this today with the CLI using 2 commands:

• flux create secret git flux-system (using a readonly PAT or SSH key)

• flux bootstrap

The end goal is to update this provider which makes use of the bootstrap package.
There would need to be a big change to the logic to implement secret creation in the suggested way, but I'm ok with doing it that way as well 😄

Flux image automation needs write access to the repo and so for the majority of users having a read-only key will not suffice.

The idea is for this to be an optional flag, that would not impact existing functionality.
This would specifically help users who don't use image automation and prefer read-only runtime credentials, without (hopefully) impacting others.

[stefanprodan]

The TF provider supports this already, see https://github.com/fluxcd/terraform-provider-flux/blob/e3fa692eaedc9e3c9f75b6b14077e2a283fd99b6/examples/github-self-managed-ssh-keypair/main.tf#L101

[MihailoPlavsic34]

Thanks!

Would you not have to create the secret in a different way during you automation workflow?

e.g. using the kubernetes provider via manifest.

The part I really like with the bootstrap provider is it does not manage lifecycle of the secret resource so you can:

1. bootstrap the cluser
2. Install addons (including e.g. External Secrets)
3. Set up flux secret rotation via ES resource

Without terraform and ES controller managing the same resource.

[stefanprodan]

So set lifecycle ignore_changes in terraform for the secret.

[MihailoPlavsic34]

🤦‍♂️ quite a bit less effort then implementing RO in the provider.

Thanks!

[stefanprodan]

Implementing your proposal in the bootstrap Go package, CLI and TF provider is a lot of work, the whole code needs to be refactored, new tests, etc. Personally I don't have time to spend some weeks on this when there are so many other priorities in Flux.

I recommend switching to Flux Operator and its terraform script, no more push access needed to repo, the whole TF code is reduced by 99%.

[mloskot]

@MihailoPlavsic34 I use the TF provider and I bring my own GitHub token this way this way, all using Terraform

1. Create github_repository_deploy_key for Flux and store in Azure Key Vault
2. Deploy External Secrets using one of Terraform providers kubernetes or kubectl or helm
3. Deploy ExtenalSecret importing the github_repository_deploy_key from AKV into cluster
4. Deploy Flux using flux_bootstrap_git set with disable_secret_creation=true and secret_name according to 3.

without use of the lifecycle directive.

[MihailoPlavsic34]

Hopefully I'd have implemented this feature myself (if maintainers found it useful) I know you are busy with more priority work (love the Gitless GitOps approach).

I was looking to do some open source contribution and what better way then solving a problem I have 😄 (turns out I had tunnel vision).

I recommend switching to Flux Operator and its terraform script, no more push access needed to repo, the whole TF code is reduced by 99%.

Looking into it already, lots of stuff I'd need to change to make it work with the current setup...
But with the new features in 2.6... might be worth overhauling everything 👀

Thanks for the help, and all the work you do for Flux 🎉