Skip to content

Extend the readiness evaluation of dependencies with CEL expressions #1491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 29, 2025
Merged

Extend the readiness evaluation of dependencies with CEL expressions #1491

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5edcf5b
api: Add the `readyExpr` field to `dependsOn`
stefanprodan Jul 22, 2025
c2754dd
controller: Add `AdditiveCELDependencyCheck` feature gate
stefanprodan Jul 22, 2025
e0e6e22
controller: Implement CEL evaluation for dependency checks
stefanprodan Jul 22, 2025
d17e5d2
docs: Add dependency ready expression to API docs
stefanprodan Jul 22, 2025
fd63b52
controller: Move manager to a dedicated file
stefanprodan Jul 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions api/v1/kustomization_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,11 @@ type KustomizationSpec struct {
// +optional
CommonMetadata *CommonMetadata `json:"commonMetadata,omitempty"`

// DependsOn may contain a meta.NamespacedObjectReference slice
// DependsOn may contain a DependencyReference slice
// with references to Kustomization resources that must be ready before this
// Kustomization can be reconciled.
// +optional
DependsOn []meta.NamespacedObjectReference `json:"dependsOn,omitempty"`
DependsOn []DependencyReference `json:"dependsOn,omitempty"`

// Decrypt Kubernetes secrets before applying them on the cluster.
// +optional
Expand Down Expand Up @@ -333,9 +333,19 @@ func (in Kustomization) GetDeletionPolicy() string {
return in.Spec.DeletionPolicy
}

// GetDependsOn returns the list of dependencies across-namespaces.
// GetDependsOn returns the dependencies as a list of meta.NamespacedObjectReference.
//
// This function makes the Kustomization type conformant with the meta.ObjectWithDependencies interface
// and allows the controller-runtime to index Kustomizations by their dependencies.
func (in Kustomization) GetDependsOn() []meta.NamespacedObjectReference {
return in.Spec.DependsOn
deps := make([]meta.NamespacedObjectReference, len(in.Spec.DependsOn))
for i := range in.Spec.DependsOn {
deps[i] = meta.NamespacedObjectReference{
Name: in.Spec.DependsOn[i].Name,
Namespace: in.Spec.DependsOn[i].Namespace,
}
}
return deps
}

// GetConditions returns the status conditions of the object.
Expand Down
26 changes: 25 additions & 1 deletion api/v1/reference_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ limitations under the License.

package v1

import "fmt"
import (
"fmt"
)

// CrossNamespaceSourceReference contains enough information to let you locate the
// typed Kubernetes resource object at cluster level.
Expand All @@ -40,9 +42,31 @@ type CrossNamespaceSourceReference struct {
Namespace string `json:"namespace,omitempty"`
}

// String returns a string representation of the CrossNamespaceSourceReference
// in the format "Kind/Name" or "Kind/Namespace/Name" if Namespace is set.
func (s *CrossNamespaceSourceReference) String() string {
if s.Namespace != "" {
return fmt.Sprintf("%s/%s/%s", s.Kind, s.Namespace, s.Name)
}
return fmt.Sprintf("%s/%s", s.Kind, s.Name)
}

// DependencyReference defines a Kustomization dependency on another Kustomization resource.
type DependencyReference struct {
// Name of the referent.
// +required
Name string `json:"name"`

// Namespace of the referent, defaults to the namespace of the Kustomization
// resource object that contains the reference.
// +optional
Namespace string `json:"namespace,omitempty"`

// ReadyExpr is a CEL expression that can be used to assess the readiness
// of a dependency. When specified, the built-in readiness check
// is replaced by the logic defined in the CEL expression.
// To make the CEL expression additive to the built-in readiness check,
Copy link
Member

@matheuscscp matheuscscp Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The word additive here all by itself can sound a bit vague. Since here is a space where we don't have to spare words (like in the case of choosing a name for a feature gate), we can probably be a bit more verbose/redundant here and say that "the hard-coded logic and the CEL expression are AND'ed together".

Copy link
Member Author

@stefanprodan stefanprodan Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the hard-coded logic and the CEL expression are AND'ed together

This sounds wrong to me, it's not hard-coded but bulit-in, the AND'ed is no better than additive.

Copy link
Member

@matheuscscp matheuscscp Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, hard-coded vs built-in sounds the same to me, but I'd say AND'ed is pretty clear and additive not so much

// the feature gate `AdditiveCELDependencyCheck` must be set to `true`.
// +optional
ReadyExpr string `json:"readyExpr,omitempty"`
}
17 changes: 16 additions & 1 deletion api/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 14 additions & 6 deletions config/crd/bases/kustomize.toolkit.fluxcd.io_kustomizations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,20 +123,28 @@ spec:
type: string
dependsOn:
description: |-
DependsOn may contain a meta.NamespacedObjectReference slice
DependsOn may contain a DependencyReference slice
with references to Kustomization resources that must be ready before this
Kustomization can be reconciled.
items:
description: |-
NamespacedObjectReference contains enough information to locate the referenced Kubernetes resource object in any
namespace.
description: DependencyReference defines a Kustomization dependency
on another Kustomization resource.
properties:
name:
description: Name of the referent.
type: string
namespace:
description: Namespace of the referent, when not specified it
acts as LocalObjectReference.
description: |-
Namespace of the referent, defaults to the namespace of the Kustomization
resource object that contains the reference.
type: string
readyExpr:
description: |-
ReadyExpr is a CEL expression that can be used to assess the readiness
of a dependency. When specified, the built-in readiness check
is replaced by the logic defined in the CEL expression.
To make the CEL expression additive to the built-in readiness check,
the feature gate `AdditiveCELDependencyCheck` must be set to `true`.
type: string
required:
- name
Expand Down
73 changes: 67 additions & 6 deletions docs/api/v1/kustomize.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,14 +89,14 @@ overridden if its key matches a common one.</p>
<td>
<code>dependsOn</code><br>
<em>
<a href="https://godoc.org/github.com/fluxcd/pkg/apis/meta#NamespacedObjectReference">
[]github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference
<a href="#kustomize.toolkit.fluxcd.io/v1.DependencyReference">
[]DependencyReference
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>DependsOn may contain a meta.NamespacedObjectReference slice
<p>DependsOn may contain a DependencyReference slice
with references to Kustomization resources that must be ready before this
Kustomization can be reconciled.</p>
</td>
Expand Down Expand Up @@ -609,6 +609,67 @@ field.</p>
</table>
</div>
</div>
<h3 id="kustomize.toolkit.fluxcd.io/v1.DependencyReference">DependencyReference
</h3>
<p>
(<em>Appears on:</em>
<a href="#kustomize.toolkit.fluxcd.io/v1.KustomizationSpec">KustomizationSpec</a>)
</p>
<p>DependencyReference defines a Kustomization dependency on another Kustomization resource.</p>
<div class="md-typeset__scrollwrap">
<div class="md-typeset__table">
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>name</code><br>
<em>
string
</em>
</td>
<td>
<p>Name of the referent.</p>
</td>
</tr>
<tr>
<td>
<code>namespace</code><br>
<em>
string
</em>
</td>
<td>
<em>(Optional)</em>
<p>Namespace of the referent, defaults to the namespace of the Kustomization
resource object that contains the reference.</p>
</td>
</tr>
<tr>
<td>
<code>readyExpr</code><br>
<em>
string
</em>
</td>
<td>
<em>(Optional)</em>
<p>ReadyExpr is a CEL expression that can be used to assess the readiness
of a dependency. When specified, the built-in readiness check
is replaced by the logic defined in the CEL expression.
To make the CEL expression additive to the built-in readiness check,
the feature gate <code>AdditiveCELDependencyCheck</code> must be set to <code>true</code>.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<h3 id="kustomize.toolkit.fluxcd.io/v1.KustomizationSpec">KustomizationSpec
</h3>
<p>
Expand Down Expand Up @@ -647,14 +708,14 @@ overridden if its key matches a common one.</p>
<td>
<code>dependsOn</code><br>
<em>
<a href="https://godoc.org/github.com/fluxcd/pkg/apis/meta#NamespacedObjectReference">
[]github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference
<a href="#kustomize.toolkit.fluxcd.io/v1.DependencyReference">
[]DependencyReference
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>DependsOn may contain a meta.NamespacedObjectReference slice
<p>DependsOn may contain a DependencyReference slice
with references to Kustomization resources that must be ready before this
Kustomization can be reconciled.</p>
</td>
Expand Down
45 changes: 45 additions & 0 deletions docs/spec/v1/kustomizations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,51 @@ is running before deploying applications inside the mesh.
**Note:** Circular dependencies between Kustomizations must be avoided,
otherwise the interdependent Kustomizations will never be applied on the cluster.

#### Dependency Ready Expression

`.spec.dependsOn[].readyExpr` is an optional field that can be used to define a CEL expression
to determine the readiness of a Kustomization dependency.

This is helpful for when custom logic is needed to determine if a dependency is ready.
For example, when performing a lockstep upgrade, the `readyExpr` can be used to
verify that a dependency has a matching version label before proceeding with the
reconciliation of the dependent Kustomization.

```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: app-backend
namespace: apps
labels:
app/version: v1.2.3
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: app-frontend
namespace: apps
labels:
app/version: v1.2.3
spec:
dependsOn:
- name: app-backend
readyExpr: >
dep.metadata.labels['app/version'] == self.metadata.labels['app/version'] &&
dep.status.conditions.filter(e, e.type == 'Ready').all(e, e.status == 'True') &&
dep.metadata.generation == dep.status.observedGeneration
```

The CEL expression contains the following variables:

- `dep`: The dependency Kustomization object being evaluated.
- `self`: The Kustomization object being reconciled.

**Note:** When `readyExpr` is specified, the built-in readiness check is replaced by the logic
defined in the CEL expression. You can configure the controller to run both the CEL expression
evaluation and the built-in readiness check, with the `AdditiveCELDependencyCheck`
[feature gate](https://fluxcd.io/flux/components/kustomize/options/#feature-gates).

### Service Account reference

`.spec.serviceAccountName` is an optional field used to specify the
Expand Down
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ require (
github.com/fluxcd/pkg/testserver v0.11.0
github.com/fluxcd/source-controller/api v1.6.0
github.com/getsops/sops/v3 v3.10.2
github.com/google/cel-go v0.23.2
github.com/hashicorp/vault/api v1.20.0
github.com/onsi/gomega v1.37.0
github.com/opencontainers/go-digest v1.0.0
Expand Down Expand Up @@ -147,11 +148,10 @@ require (
github.com/go-openapi/jsonpointer v0.21.1 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
github.com/google/btree v1.1.3 // indirect
github.com/google/cel-go v0.23.2 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/go-containerregistry v0.20.6 // indirect
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -258,8 +258,8 @@ github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1v
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-test/deep v1.0.2 h1:onZX1rnHT3Wv6cqNgYyFOOlgVKJrksuCMCRvJStbMYw=
github.com/go-test/deep v1.0.2/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
Expand Down
5 changes: 4 additions & 1 deletion internal/controller/constants.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,7 @@ limitations under the License.

package controller

const OCIArtifactOriginRevisionAnnotation = "org.opencontainers.image.revision"
const (
OCIArtifactOriginRevisionAnnotation = "org.opencontainers.image.revision"
TerminalErrorMessage = "Reconciliation failed terminally due to configuration error"
)
Loading