Granular control of slack notifications

[tageskri]

I am in the process of transitioning 13 clusters managed by flux from a cluster-per-branch setup to a cluster directories in a main branch setup as advised.

From before we have separate slack channels with notifications from flux from each cluster and repo. However, when moving the setup into a monobranch setup the messages from one cluster is sent to all channels on Slack. Our manifests looks like this in each respective cluster:

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: slack-alerts
  namespace: flux-system
spec:
  providerRef:
    name: slack
  eventSeverity: info
  eventSources:
    - kind: GitRepository
      name: "*"
    - kind: Kustomization
      name: "*"
    - kind: HelmRelease
      name: "*"

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Provider
metadata:
  name: slack
  namespace: flux-system
spec:
  type: slack
  channel: <slack channel name, cluster-based>
  proxy: <proxy server>
  secretRef:
    name: slack-url

apiVersion: v1
stringData:
    address:  <slack hook address>
kind: Secret
metadata:
    name: slack-url
    namespace: flux-system

I couldn't find anything in the official docs about this, but how can I keep our existing slack channels having each cluster send notifications to each respective channel?

[matheuscscp]

Having a single source for all your clusters is still a problem even if you move away from the bad multi-branch pattern. Events about reconciliations unrelated to the cluster landing in the cluster Slack channel is the smallest of your problems. The bigger problem is causing the reconciliations in the first place. To fix this our recommendation is to move from GitRepository to OCIRepository. This way you can manage all your clusters in a single Git branch but still break the repo down in small per-cluster artifacts, this is the key design change in ControlPlane's D2 Reference Architecture:

https://fluxcd.control-plane.io/guides/d2-architecture-reference/

[tageskri]

Thanks for your reply, that is a lot to take in. The initial plan however is to migrate to the example layout, the flux2-kustomize-helm-example. It would be quite a major change in our infrastructure that is not going to happen overnight. So looking at the example repo just to stick to that, how would notifications be handled there in separate slack channel?

[matheuscscp]

how would notifications be handled there in separate slack channel?

I don't see a way if your GitRepository is looking at the same Git ref for all clusters. Every time you push, the GitRepository object inside each cluster will see the push and send an event. I already gave you the solution 🤷

[tageskri]

Is there a way to change the name of the GitRepository object without everything going haywire then? Like changing metadata.name or something? Currenly, the gotk-sync.yaml looks like this:

# This manifest was generated by flux. DO NOT EDIT.
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: main
  secretRef:
    name: flux-system
  url: <url to repo>
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 10m0s
  path: ./<name of cluster, previously the branch name>
  decryption:
    provider: sops
    secretRef:
      name: <key>
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system

The path of this is .//flux-system and in this directory the slack-notifications reside, it would make sense to me at least if it was a way to have a succinct GitRepository object so that having multiple slack channels would work.

[matheuscscp]

You can change the GitRepository object names but it will make no difference. Think about it. The objects are all looking at the same branch of the same Git repository, they will all see the new commit, it's inevitable, there are no "skip this commit" features when watching a branch in source-controller.