Reimplement release of workload pods in ComputeDomain in terms of BindingConditions

[klueska]

For ComputeDomains we need to do the following:

• Schedule a workload pod referencing a ResourceClaim to a node
• Label the node where the workload pod has been scheduled with a specific label
• Wait for a daemonset pod to be scheduled on the newly labeled node and become ready
• Release the original workload pod and allow it to start running

At present, we accomplish this by doing (2) in the PrepareResources() call for (1) and then failing in a loop until (3) has completed.

Using BindingConditions we can do (1), (2), and (3) without ever starting to create the workload pod on the node and only do so once (3) is complete.

This will lead to a simpler implementation and a snappier experience. Right now failing in the loop can sometimes result in significant delays for the workload pod to start (depending on the timing for (3) to complete).

Link to KEP describing BindingConditions:

• DRA: Device Binding Conditions kubernetes/enhancements#5007

[hase1128]

@klueska
CC: @ttsuuubasa @johnbelamaric @pohly
Do you plan to implement this feature? (For example, in a near milestone like K8s 1.36)
If not, would it be acceptable for my team member (the one who implemented DRA BindingConditions) to implement it and submit a PR? We want to gather feedback as one requirement for promoting BindingConditions to beta.