Label conflict between Capsule and Capsule Proxy when using GlobalTenantResource

[larsgerber]

Bug description

When using GlobalTenantResource with both proxy feature gates enabled, Capsule and Capsule Proxy fight about the capsule.clastix.io/managed-by label on the replicated resources inside the tenant namespace.

How to reproduce

I was able to reproduce the issue in a local Kind cluster:
https://github.com/larsgerber/kind/tree/main/capsule/replication

Expected behavior

Capsule should ignore this label when doing the reconciliation loop. Otherwise after x seconds (resyncPeriod) all resources replicated by Capsule will be mutated two times.

Logs

=== NetworkPolicy allow-all-ingress UPDATE detected ===
User: system:serviceaccount:capsule-system:capsule
Change detected in full object (excluding volatile fields):
--- old
+++ new
@@ -9,7 +9,6 @@
     "creationTimestamp": "2025-09-12T10:53:06Z",
     "generation": 1,
     "labels": {
-      "capsule.clastix.io/managed-by": "oil",
       "capsule.clastix.io/resources": "0",
       "capsule.clastix.io/tenant": "oil",
       "foo": "bay",
10.244.0.1 - - [12/Sep/2025 12:43:15] "POST /validate?timeout=10s HTTP/1.1" 200 -

=== NetworkPolicy allow-all-ingress UPDATE detected ===
User: system:serviceaccount:capsule-system:capsule-proxy
Change detected in full object (excluding volatile fields):
--- old
+++ new
@@ -9,6 +9,7 @@
     "creationTimestamp": "2025-09-12T10:53:06Z",
     "generation": 1,
     "labels": {
+      "capsule.clastix.io/managed-by": "oil",
       "capsule.clastix.io/resources": "0",
       "capsule.clastix.io/tenant": "oil",
       "foo": "bay",
10.244.0.1 - - [12/Sep/2025 12:43:15] "POST /validate?timeout=10s HTTP/1.1" 200 -

Additional context

• Capsule-Proxy version: capsule-proxy:v0.9.13
• Helm Chart version: capsule-proxy-0.9.13
• Kubernetes version: v1.34.0

[adberger]

@oliverbaehler Do you have an opinion on how a problem like this should be solved?
Seems to be like #530

[adberger]

Problem actually got worse with the new capsule version (helm 0.11.2).

[oliverbaehler]

Yeh i will look into it

[oliverbaehler]

@prometherion we should probably take watchdog and put it into the capsule repository, as it's only required once. WDYT? This does not solve this issue per se, but would be a combined change

[oliverbaehler]

Well refactor everything to use SSA with a custom field manager for both capsule and watchdog. We will also add the option to define ignores (FluxCD-Like)

[adberger]

@oliverbaehler Thank you :)
Let me know if we can help for testing etc.

[oliverbaehler]

Preview config:

apiVersion: capsule.clastix.io/v1beta2
kind: CapsuleConfiguration
metadata:
  name: default
spec:
  replications:
    ignore:
      - paths:
          - "/metadata/labels/capsule.clastix.io~1managed-by"

And for TenantResources:

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: global-scope
spec:
  scope: Tenant
  resyncPeriod: 5s
  resources:
    - ignore:
        - paths:
          - "/metadata/labels/capsule.clastix.io~1managed-by"
          target:
            kind: ConfigMap
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: game-demo
              namespace: default
            data:
              # property-like keys; each key maps to a simple value
              player_initial_lives: "3"
              ui_properties_file_name: "user-interface.properties"
            
              # file-like keys
              game.properties: |
                {{- toYaml . | nindent 4 }}

@adberger from a user perspective, do these placing make sense (ignore)?
Just tested this and technically this works and solves the problem in a generic way, i am happy with it. But i am yet unsure of the placement of the configs. They idea of having it in the capsuleconfiguration is essentially = This applies for any Global/TenantResource. While you can still ignore per resource field on a TenantResource. I am unsure if this should really be fro each resource or rather for the entire TenantResource

[adberger]

@oliverbaehler I've discussed this internally and we've come to the conclusion, that for our use-case we would need it only "globally" in the CapsuleConfiguration for now.
The naming makes sense, imo.

Would be nice to have /metadata/labels/capsule.clastix.io~1managed-by as default via helm chart: https://github.com/projectcapsule/capsule/blob/866c69ffc322f0cff604601f314ef9b79673eae8/charts/capsule/templates/configuration.yaml

Maybe the second part (TenantResource& GlobalTenantResource) could be ignored for now, until a use-case emerges.

[adberger]

Not really part of this issue, but other operators might also conflict with capsule-proxy like cert-manager.
But this could be solved in the future with https://kubernetes.io/docs/reference/using-api/server-side-apply/#field-management?

[oliverbaehler]

Yeah we are not doing the ignore thing but now migrating everything to SSA with field-manager and introduce the options to use force. This also solves the problem and allows for much cooler templating eg.

apiVersion: capsule.clastix.io/v1beta2
kind: GlobalTenantResource
metadata:
  name: global-scope
spec:
  # New
  scope: Tenant
  force: false
  resyncPeriod: 5s
  serviceaccount: 
    name: capsule
    namespace: capsule-system
  resources:
    - #additionalMetadata:
      #  labels:
      #    "replicated-by": "capsule" 
      context:
        resources:
          - index: "sec"
            apiVersion: v1
            kind: Secret
            name: capsule-tls
      generators:
        - template: |
            ---
            apiVersion: v1
            kind: ConfigMap
            metadata:
              name: test-demo
              namespace: default
            data:
            {{ .Tenant.ObjectMeta.Name }}.conf: |
                1

then we jsut have the managed fields.

apiVersion: v1
data:
  green.conf: |
    1
  solar.conf: |
    1
  wind.conf: |
    1
kind: ConfigMap
metadata:
  creationTimestamp: "2025-11-11T08:54:39Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        f:green.conf: {}
    manager: capsule/cluster/global-scope/green/default/ConfigMap/test-demo/0/gen-0-0
    operation: Apply
    time: "2025-11-11T08:54:39Z"
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        f:solar.conf: {}
    manager: capsule/cluster/global-scope/solar/default/ConfigMap/test-demo/0/gen-0-0
    operation: Apply
    time: "2025-11-11T08:54:39Z"
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        f:wind.conf: {}
    manager: capsule/cluster/global-scope/wind/default/ConfigMap/test-demo/0/gen-0-0
    operation: Apply
    time: "2025-11-11T08:54:39Z"
  name: test-demo
  namespace: default
  resourceVersion: "589589"
  uid: e40545a3-d47c-4829-aebd-ae70e1ad83b0