Pod selectors for kubelet plugin DaemonSet and controller Deployment are identical

[nojnhuh]

Both the DaemonSet and Deployment in the Helm chart define the following selector (using the default values for labels and selectors):

    selector:
      matchLabels:
        app.kubernetes.io/instance: nvidia-dra-driver-gpu
        app.kubernetes.io/name: nvidia-dra-driver-gpu

This doesn't have any tangible ill-effects that I've noticed except that commands like kubectl logs or kubectl exec will sometimes pick an incorrect Pod.

Given these pods:

% kubectl get pods -n nvidia-dra-driver-gpu --show-labels
NAME                                                READY   STATUS    RESTARTS       AGE   LABELS
nvidia-dra-driver-gpu-controller-5558c86b87-x2bkz   1/1     Running   0              79m   app.kubernetes.io/instance=nvidia-dra-driver-gpu,app.kubernetes.io/name=nvidia-dra-driver-gpu,pod-template-hash=5558c86b87
nvidia-dra-driver-gpu-kubelet-plugin-82gwf          2/2     Running   10 (69m ago)   71m   app.kubernetes.io/instance=nvidia-dra-driver-gpu,app.kubernetes.io/name=nvidia-dra-driver-gpu,controller-revision-hash=6fc465774f,pod-template-generation=1
nvidia-dra-driver-gpu-kubelet-plugin-xqlb4          2/2     Running   10 (74m ago)   76m   app.kubernetes.io/instance=nvidia-dra-driver-gpu,app.kubernetes.io/name=nvidia-dra-driver-gpu,controller-revision-hash=6fc465774f,pod-template-generation=1

This command for the Daemonset picks a Pod created from by the Deployment.

% kubectl logs -n nvidia-dra-driver-gpu ds/nvidia-dra-driver-gpu-kubelet-plugin 
Found 3 pods, using pod/nvidia-dra-driver-gpu-controller-5558c86b87-x2bkz
...

This one happened to pick the correct Pod, but still found that all three Pods matched the selector:

% kubectl logs -n nvidia-dra-driver-gpu deploy/nvidia-dra-driver-gpu-controller 
Found 3 pods, using pod/nvidia-dra-driver-gpu-controller-5558c86b87-x2bkz
...

I'd expect the Deployment's selector to match only the Pods it creates, and likewise for the Daemonset.

[jgehrcke]

Interesting! Thanks for the feedback.

I agree, we probably want to change that (both ways): The ds/nvidia-dra-driver-gpu-kubelet-plugin query (type/name specifier) should not yield controller logs, and the deploy/nvidia-dra-driver-gpu-controller query should not yield kubelet plugin logs.

[jgehrcke]

We are setting the same generic selector labels everywhere:

k8s-dra-driver-gpu/deployments/helm/nvidia-dra-driver-gpu/templates/_helpers.tpl

Line 75 in bf444c4

{{ include "nvidia-dra-driver-gpu.templateLabels" . }}

From:

The .spec.selector field defines how the created ReplicaSet finds which Pods to manage

That is, the purpose of this field is to select just the pods belonging to the wrapping deployment / daemonset. Proposing a patch via #359.