It's not clear how to reduce "Dependencies do not meet ready condition" when there is a complex dependency graph.

[pkit]

The more dependsOn are there in the system the more the probability of one reconciliation hitting another at the same time grows.
For 10 dependencies I'm getting at least one Dependencies do not meet ready condition notification each time the reconcile happens.
Do I need to strategically place the interval on prime numbers? Or maybe there is a point in getting the interval jittered automatically?

[hiddeco]

Think this is is because we should be more thoughtful about when we are sending out external notifications versus internal Kubernetes events. Which is been taken into account as part of a bigger refactor we are doing across controllers to better align how we are working with dependencies, conditions, etc. (see: #1602).

Reason is, that all controllers live in a distributed space, and dependencies not being ready at certain times is kind of expected in this universe. Given this, the user likely only wants to be notified about it if e.g. the dependency isn't available for a longer period, which can be a sign that something is stuck. But they should likely use an alert rule for this.

Given this, I am of opinion that the following information should be recorded by the controller:

• Some type of Condition which is part of the API and sinked to a metric
  This makes the state observable, and allows setting an e.g. time based alert
• (Optionally) a Kubernetes Event (but not a notification) for a point in time record of the observation with e.g. dependency metadata added to it

This provides you with less information in e.g. Slack channels, but I think most users are only interested in persistent errors, or state changes, which I wouldn't categorize this as.

[pkit]

Yup, probably it's good to have a configuration entry that not ready should happen more than once or even more than 5 times before a human is notified.