Feature: ReconcilerRateLimiting

[sbueringer]

• One-line description: Enables reconciler rate-limiting (currently 1 request / 1 second)

• Feature gate state:

  • Alpha (disabled per default): v1.12
  • Beta (enabled per default): TBD
  • Stable (feature gate removed): TBD

• PRs:

  • TODO

• Next steps:

  • Export capicontrollerutil so it can be used in providers like CAPV as well
  • Check if we can de-duplicate requeue logic in the ClusterCache
  • Try to find a way to avoid duplicating the CR metrics (very likely would require changes to CR) (also consider if there are other things in the Reconcile wrapper that could be moved to CR or be made easier by changes to CR)

Today our reconcilers are reconciling on all incoming events which is way too often. The goal of this feature is to rate-limit our controllers to reduce overall resource consumption.

As we are close to code-freeze and in an abundance of caution we are adding this behind a feature gate.
We are planning to graduate quickly once appropriate scale testing has been done.