Skip to content

Status is not descriptive in case pod cannot be evicted #128

@rollandf

Description

@rollandf

For example if delete-emptydir-data was not specified and there are such pod, it is not reflected in the Status.

Note that 'GetPodsForDeletion' returns sometimes a pod list and an error. Maybe this can be used to reflect in the status.

See here:

2025-09-09T11:01:27.758502826Z	ERROR	controller/nodemaintenance_controller.go:166	failed to handle drain state for NodeMaintenance object	{"controller": "nodemaintenance", "controllerGroup": "maintenance.nvidia.com", "controllerKind": "NodeMaintenance", "NodeMaintenance": {"name":"network-operator-c-236-2-60-065","namespace":"default"}, "namespace": "default", "name": "network-operator-c-236-2-60-065", "reconcileID": "886c7c39-0c7c-4684-9b23-9b0e48cf0a0d", "error": "failed to update drain status. failed to get pods for deletion. cannot delete Pods with local storage (use --delete-emptydir-data to override): kube-system/rke2-metrics-server-69bdccfdd9-28jjq"}
2025-09-09T11:01:27.758564238Z	INFO	controller/nodemaintenance_controller.go:175	reconcile NodeMaintenance request end	{"controller": "nodemaintenance", "controllerGroup": "maintenance.nvidia.com", "controllerKind": "NodeMaintenance", "NodeMaintenance": {"name":"network-operator-c-236-2-60-065","namespace":"default"}, "namespace": "default", "name": "network-operator-c-236-2-60-065", "reconcileID": "886c7c39-0c7c-4684-9b23-9b0e48cf0a0d"}
2025-09-09T11:01:27.758606651Z	ERROR	controller/controller.go:316	Reconciler error	{"controller": "nodemaintenance", "controllerGroup": "maintenance.nvidia.com", "controllerKind": "NodeMaintenance", "NodeMaintenance": {"name":"network-operator-c-236-2-60-065","namespace":"default"}, "namespace": "default", "name": "network-operator-c-236-2-60-065", "reconcileID": "886c7c39-0c7c-4684-9b23-9b0e48cf0a0d", "error": "failed to update drain status. failed to get pods for deletion. cannot delete Pods with local storage (use --delete-emptydir-data to override): kube-system/rke2-metrics-server-69bdccfdd9-28jjq"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions