Is it possible to restructure Flux Repo

[jseiser]

We have 1 Flux deployment, that controls 1 Cluster.

> tree -L 2
.
├── clusters
│   └── prod
├── deploys
│   └── kustomization.yaml
├── docker-hub-creds.yaml
├── infrastructure
│   ├── atlantis
│   ├── aws-cloudwatch-metrics
│   ├── aws-ebs-csi-driver
│   ├── aws-efs-csi-driver
│   ├── aws-load-balancer-controller
│   ├── cert-manager
│   ├── external-dns
│   ├── fluentbit
│   ├── goldilocks
│   ├── image-scan
│   ├── ingress-nginx-external
│   ├── ingress-nginx-internal
│   ├── karpenter
│   ├── kube-prometheus-stack
│   ├── kustomization.yaml
│   ├── linkerd
│   ├── linkerd-viz
│   ├── loki
│   ├── metrics-server
│   ├── notifications
│   ├── promtail
│   ├── sonarqube
│   ├── sources
│   └── wikijs
├── Pipfile
├── Pipfile.lock
├── README.md
└── scripts
    └── validate.sh

clusters/prod
├── deploys.yaml
├── flux-system
│   ├── gotk-components.yaml
│   ├── gotk-sync.yaml
│   └── kustomization.yaml
└── infrastructure.yaml

> cat clusters/prod/infrastructure.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 10m0s
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure
  prune: true
  decryption:
    provider: sops
    secretRef:
      name: sops-age

Is it possible, without having to uninstall/re-create various HelmReleases, to be able to split up the infrastructure Kustomization ? I would like to lessen the blast radius for times I do something wrong and create a situation where an issue with 1 helm chart, can hold up the processing of the other charts.

I want to end up with something like the below.

/infrastructure/monitoring
/infrastructure/linkerd
/infrastructure/nginx
/infrastructure/karpenter
/infrastructure/aws
/infrastructure/cert-manager
/infrastructure/services

Thanks,

[stefanprodan]

Please see FAQ on how to safely move resources between Flux Kustomizations https://fluxcd.io/flux/faq/#how-can-i-safely-move-resources-from-one-dir-to-another

You can also run flux uninstall which doesn't touches any managed workloads, then after the structure refactoring is done you can rerun bootstrap.

[jseiser]

@stefanprodan

This worked for things like my image sources, or helm repositories. When moving an application to a new kustomization, I error out.

> tree applications/monitoring -L 2
applications/monitoring
├── kustomization.yaml
├── loki
│   ├── docker-hub-creds.yaml
│   ├── kustomization.yaml
│   ├── namespace.yaml
│   └── release.yaml

> cat applications/monitoring/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: flux-system
resources:
    - loki

I get an error for a docker secret in the loki folder

kustomize build failed: namespace transformation produces ID conflict: [{"apiVersion":"v1","data":{".dockerconfigjson":"ENC[AES256_GCM,data:C0/FWx9QqqsFpY8q0cnmZvA64EmUilb

> ls -l applications/monitoring/loki
.rw-r--r--@ 2.7k justin  7 May 12:06 docker-hub-creds.yaml
.rw-r--r--@  182 justin  7 May 12:06 kustomization.yaml
.rw-r--r--@  226 justin  7 May 12:06 namespace.yaml
.rw-r--r--@ 3.8k justin  7 May 12:06 release.yaml

> cat applications/monitoring/loki/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: loki
resources:
  - namespace.yaml
  - docker-hub-creds.yaml
  - release.yaml

There is no namespace field defined in the yaml.

> cat applications/monitoring/loki/docker-hub-creds.yaml
apiVersion: v1
data:
    .dockerconfigjson: stuff here
kind: Secret
metadata:
    creationTimestamp: null
    name: dhc

Secret in k8s

> kubectl describe secret dhc -n loki
Name:         dhc
Namespace:    loki
Labels:       kustomize.toolkit.fluxcd.io/name=infrastructure
              kustomize.toolkit.fluxcd.io/namespace=flux-system
Annotations:  <none>

Type:  kubernetes.io/dockerconfigjson

Data
====
.dockerconfigjson:  239 bytes

[stefanprodan]

Why would you set namespace: flux-system in applications/monitoring/kustomization.yaml, that's where the conflict comes from

[jseiser]

No idea, bad copy paste :/

Thanks, i was not understanding what it was telling me.