|
| 1 | +# Proposal: Standardize KRO Labels |
| 2 | + |
| 3 | +## Problem statement |
| 4 | + |
| 5 | +The current labels in KRO are used to track ownership and metadata of resources managed by the orchestrator. While these labels provide essential information, they lack a clear and standardized way to represent the relationships between different KRO components, such as ResourceGraphDefinitions (RGDs), instances, and the resources they generate. This ambiguity can make it difficult to query for related objects and understand the provenance of resources within a cluster. |
| 6 | + |
| 7 | +Specifically, we need a clearer way to identify: |
| 8 | +1. Which RGD is responsible for reconciling a given instance. |
| 9 | +2. Which RGD a created resource originates from. |
| 10 | +3. Which instance a created resource belongs to. |
| 11 | + |
| 12 | +This proposal aims to introduce new, more descriptive labels to clarify these relationships, aligning with Kubernetes best practices for resource management and observability. |
| 13 | + |
| 14 | +## Proposal |
| 15 | + |
| 16 | +We propose introducing a set of standardized labels to better represent the relationships between RGDs, instances, and the resources they manage. This will improve the ability to discover, manage, and debug resources created and managed by KRO. |
| 17 | + |
| 18 | +#### Overview |
| 19 | + |
| 20 | +The proposal is to add the following labels: |
| 21 | +- A label on an instance to indicate which RGD is reconciling it. |
| 22 | +- A label on resources created by an instance reconciliation to indicate which RGD they come from. |
| 23 | +- Labels on resources created by an instance reconciliation to indicate which instance created them. |
| 24 | + |
| 25 | +This aligns with the common Kubernetes practice of using labels for discovery and organization of related resources, as seen in tools like Helm and Kustomize. |
| 26 | + |
| 27 | +#### Design details |
| 28 | + |
| 29 | +We will rename or introduce the following labels, which will be defined in `@pkg/metadata/labels.go`: |
| 30 | + |
| 31 | +1. **Labels for mapping created resources back to the instance:** |
| 32 | + - To provide a direct and queryable link from a created resource back to the instance that caused its creation. This set of labels uniquely identifies the instance. |
| 33 | + - **Proposed Labels:** |
| 34 | + - `kro.run/instance-gvk: mygroup.example.com/v1/MyKind`: The Group Version Kind (GVK) of the instance |
| 35 | + - `kro.run/instance: default/my-instance`: The namespace and name of the instance. |
| 36 | + - `app.kubernetes.io/instance: MyKind/default/my-instance`: Kind namespace and name of the instance |
| 37 | + - **Existing Labels** |
| 38 | + - `kro.run/instance-id: 6e6a1d95-4855-4062-b51b-8f288cb96365` |
| 39 | + - `kro.run/instance-name: test-instance` |
| 40 | + - `kro.run/instance-namespace: default` |
| 41 | +2. **Label for mapping created resources back to the RGD:** |
| 42 | + - To identify which RGD was used as a template to create a resource during the reconciliation of an instance. This label will be applied to all resources created by the instance controller. |
| 43 | + - **Proposed Labels:** |
| 44 | + - `kro.run/part-of: my-rgd-name` - The name of the ResourceGraphDefinition |
| 45 | + - `app.kubernetes.io/part-of: my-rgd-name` |
| 46 | + - **Existing Labels:** |
| 47 | + - `kro.run/resource-graph-definition-id: 761e8fb7-14d5-4da1-b4f9-0a16fc334d6c` |
| 48 | + - `kro.run/resource-graph-definition-name: my-rgd-name` |
| 49 | +3. **Label for nested RGD instance:** |
| 50 | + - To differenciate the RGD that defines (and hence reconciles) the instance and the RGD that created it, we need to introduce a separate label. |
| 51 | + - Currently we dont differentiate, it results in a conflict (https://github.com/kro-run/kro/pull/631) |
| 52 | + - To indicate that an instance is being actively reconciled against a specific RGD. This label will be applied to the instance itself. |
| 53 | + - **Proposed Labels:** |
| 54 | + - `kro.run/managed-by-rgd: my-rgd-name` - name of the ResourceGraphDefinition that reconciles the instance. |
| 55 | + - **Existing Labels:** |
| 56 | + - `kro.run/resource-graph-definition-id: 761e8fb7-14d5-4da1-b4f9-0a16fc334d6c` |
| 57 | + - `kro.run/resource-graph-definition-name: my-rgd-name` |
| 58 | + - Conflicts with label used in creation path for nested RGD (https://github.com/kro-run/kro/pull/631) |
| 59 | +4. **Kubernetes Common Labels:** |
| 60 | + - **Purpose:** Conforming to standard/common k8s labels allows easier integration across tools in k8s ecosystem. |
| 61 | + - **Proposed Labels:** |
| 62 | + - `app.kubernetes.io/managed-by: KRO` : Fixed value |
| 63 | + |
| 64 | +## Other solutions considered |
| 65 | + |
| 66 | +1. We could continue using the existing labels, but their purpose is not as explicit for relationship tracking, which can lead to confusion for users and client tools. |
| 67 | +2. We could also use annotations, but labels are better suited for this purpose as they are queryable via the Kubernetes API, which is a key requirement for observability and tooling. |
| 68 | +3. Use common k8s labels: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/ |
| 69 | + * `app.kubernetes.io/name: mysql` |
| 70 | + * `app.kubernetes.io/instance: mysql-abcxyz` |
| 71 | + * `app.kubernetes.io/version: 5.7.21` |
| 72 | + * `app.kubernetes.io/component: database` |
| 73 | + * `app.kubernetes.io/part-of: wordpress` |
| 74 | + * `app.kubernetes.io/managed-by: Helm` |
| 75 | + |
| 76 | + |
| 77 | +## Scoping |
| 78 | + |
| 79 | +#### What is in scope for this proposal? |
| 80 | + |
| 81 | +- Defining and adding the new labels to the KRO API. |
| 82 | +- Updating the instance controller to apply these labels to instances and created resources during reconciliation. |
| 83 | +- Updating the official KRO documentation to reflect the new labels and their usage. |
| 84 | +- Adding docs reflecting the new labels. Old labels would not be documented. |
| 85 | +- Deprecate old labels in next minor release or the one after that. |
| 86 | + |
| 87 | +#### What is not in scope? |
| 88 | + |
| 89 | +- Tracking UID od RGD, Instance in the labels. We need to handle recreation of parent resources where UID changes. Can be a future enhancement. |
| 90 | +- Supporting deeply nested RGDs. When RGD1 created RGD2 and that creates RGD3, do we add labels in objects of RGD3 linking it back to RGD2 and RGD1 ? |
| 91 | +- Changing how KRO uses annotations. |
| 92 | +- Changing these existing labels: |
| 93 | + - kro.run/kro-version: v0.4.0 |
| 94 | + - kro.run/owned: "true" |
| 95 | + |
| 96 | +## Testing strategy |
| 97 | + |
| 98 | +#### Requirements |
| 99 | + |
| 100 | +- A running Kubernetes cluster to deploy KRO and test the label application. |
| 101 | + |
| 102 | +#### Test plan |
| 103 | + |
| 104 | +- **Unit tests:** Add unit tests for any new labeler functions in `pkg/metadata/labels.go` to ensure they construct the correct labels. |
| 105 | +- **Integration tests:** |
| 106 | + - Create an RGD and an instance of that RGD. |
| 107 | + - Verify that the instance object has the `kro.run/reconciled-by` label pointing to the RGD. |
| 108 | + - Verify that all resources created by the instance reconciliation have the `kro.run/created-by` label, and the `kro.run/instance-group`, `kro.run/instance-kind`, `kro.run/instance-namespace`, and `kro.run/instance-name` labels with the correct values. |
| 109 | + - Update an instance to be reconciled by a different RGD and verify that the `kro.run/reconciled-by` label is updated accordingly. |
0 commit comments