Extend the functionality of apply-exit to refresh CDI config #215

ArangoGutierrez · 2025-07-01T10:41:22Z

This patch adds extra functionality to the apply-exit hook, calling the nvidia-ctk cdi generate to refresh the CDI config on a MIG (re)configuration event

deployments/systemd/utils.sh

deployments/systemd/hooks-default.yaml

elezar · 2025-07-01T14:54:36Z

deployments/systemd/hooks.sh

 	fi
 }

+function refresh-cdi-config() {


This is no longer required.

deployments/systemd/hooks-default.yaml

ArangoGutierrez · 2025-07-02T11:32:30Z

PR Edited
@elezar / @klueska PTAL

elezar · 2025-07-10T09:47:22Z

@ArangoGutierrez how are we specifying WHERE the output should be created? Although @klueska had concerns about triggering the nvidia-cdi-refresh.service doing so would mean that we would ensure consistency in terms of the ouptut path.

Also, on:

Given that NVIDIA/nvidia-container-toolkit#1107 is merged, if I remove the --output from this PR that should be enough to allow the user to specify custom configuration via the config file

I don't think that specifying an output path in the config file makes sense since this would complicate the default behvaiour which is to output the spec to STDOUT.

ArangoGutierrez · 2025-07-10T12:19:00Z

@ArangoGutierrez how are we specifying WHERE the output should be created? Although @klueska had concerns about triggering the nvidia-cdi-refresh.service doing so would mean that we would ensure consistency in terms of the ouptut path.

I would like to call the nvidia-cdi-refresh.service from the hook, maybe something to discuss with @klueska

ArangoGutierrez · 2025-08-08T14:11:46Z

@klueska / @elezar Updated based on today's discussion

Copilot

Pull Request Overview

This PR extends the apply-exit hook functionality to refresh CDI (Container Device Interface) configuration after MIG reconfiguration events. The implementation adds a new hook that triggers CDI configuration updates through a systemd service.

Key changes:

Adds a new cdi-refresh hook to the hooks configuration
Creates a new script that conditionally starts the nvidia-cdi-refresh systemd service
Integrates CDI refresh functionality into the MIG manager's hook system

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
deployments/systemd/hooks-default.yaml	Adds new cdi-refresh hook configuration that executes the CDI refresh script
deployments/systemd/cdi-refresh.sh	New shell script that conditionally starts nvidia-cdi-refresh.service if available

deployments/systemd/cdi-refresh.sh

ArangoGutierrez · 2025-08-15T06:36:16Z

Refresher PING @klueska / @elezar

ArangoGutierrez · 2025-10-09T07:26:47Z

Now that we have Toolkit RC6 and are almost ready for proper release, and Systemd is stable, can we give this PR another try?
@klueska / @elezar

elezar · 2025-11-10T09:34:23Z

deployments/systemd/hooks-default.yaml

    - workdir: "/etc/nvidia-mig-manager"
      command: "/bin/bash"
      args: ["-x", "-c", "source hooks.sh; apply-exit"]
+    cdi-refresh:


This is not a valid hook name. I think this should probably just be an apply-exit hook.

elezar · 2025-11-10T09:35:39Z

deployments/systemd/cdi-refresh.sh

This file is not included in the rpm or deb packages.

elezar

This is not what we discussed. We should:

Define cdi-refresh in hooks.sh
Add a new hook to the apply-exit hooks hooks-default.yaml that calls the function in the same way as we do for the other hooks.

We ALSO discussed (not for this PR):

Updating the names of the other functions to show intent and not the EVENTS
Clarifying the API around envvars etc.

deployments/systemd/hooks.sh

ArangoGutierrez · 2025-11-10T11:07:08Z

This is not what we discussed. We should:

Define cdi-refresh in hooks.sh

Add a new hook to the apply-exit hooks hooks-default.yaml that calls the function in the same way as we do for the other hooks.

We ALSO discussed (not for this PR):

Updating the names of the other functions to show intent and not the EVENTS

Clarifying the API around envvars etc.

Done

deployments/systemd/hooks.sh

ArangoGutierrez · 2025-11-10T12:54:14Z

I have updated this PR to be in line with changes proposed at #275

elezar · 2025-11-12T08:25:28Z

deployments/systemd/hooks.sh

+    # Check if nvidia-cdi-refresh.service exists
+    if systemctl list-unit-files nvidia-cdi-refresh.service --quiet; then
+        echo "Found nvidia-cdi-refresh.service, calling systemctl..." >&2
+        if ! systemctl start nvidia-cdi-refresh.service; then


Question: Should this be restart instead of start?

You are right, will edit

Signed-off-by: Carlos Eduardo Arango Gutierrez <[email protected]>

ArangoGutierrez · 2025-11-19T10:16:09Z

@elezar I did a change after your approval PTAL

@elezar

@elezar I did a change after your approval PTAL

ArangoGutierrez requested review from Copilot, elezar and klueska July 1, 2025 10:41

ArangoGutierrez self-assigned this Jul 1, 2025

This comment was marked as outdated.

Sign in to view

klueska reviewed Jul 1, 2025

View reviewed changes

deployments/systemd/utils.sh Outdated Show resolved Hide resolved

ArangoGutierrez requested a review from klueska July 1, 2025 11:49

elezar reviewed Jul 1, 2025

View reviewed changes

deployments/systemd/hooks-default.yaml Outdated Show resolved Hide resolved

ArangoGutierrez requested a review from elezar July 1, 2025 12:18

ArangoGutierrez force-pushed the cdi_generate_hook branch from 8eff22b to d671054 Compare July 1, 2025 14:52

elezar reviewed Jul 1, 2025

View reviewed changes

deployments/systemd/hooks-default.yaml Outdated Show resolved Hide resolved

ArangoGutierrez force-pushed the cdi_generate_hook branch 2 times, most recently from cf9c0bd to 83b4d05 Compare July 1, 2025 15:08

ArangoGutierrez requested a review from elezar July 1, 2025 15:09

ArangoGutierrez changed the title ~~Add refresh-cdi hook~~ Extend the functionality of apply-exit to refresh CDI config Jul 2, 2025

ArangoGutierrez force-pushed the cdi_generate_hook branch from 83b4d05 to 97b7f03 Compare July 10, 2025 09:06

ArangoGutierrez requested a review from Copilot July 10, 2025 09:06

This comment was marked as outdated.

Sign in to view

ArangoGutierrez force-pushed the cdi_generate_hook branch from 97b7f03 to 077807f Compare August 8, 2025 14:11

ArangoGutierrez requested a review from Copilot August 8, 2025 14:11

This comment was marked as outdated.

Sign in to view

ArangoGutierrez force-pushed the cdi_generate_hook branch from 077807f to e624ef8 Compare August 8, 2025 14:15

ArangoGutierrez requested a review from Copilot August 8, 2025 14:15

Copilot AI reviewed Aug 8, 2025

View reviewed changes

deployments/systemd/cdi-refresh.sh Outdated Show resolved Hide resolved

deployments/systemd/cdi-refresh.sh Outdated Show resolved Hide resolved

elezar reviewed Nov 10, 2025

View reviewed changes

deployments/systemd/cdi-refresh.sh Outdated

Copy link

Member

elezar Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is not included in the rpm or deb packages.

ArangoGutierrez force-pushed the cdi_generate_hook branch from e624ef8 to 13e0185 Compare November 10, 2025 10:38

ArangoGutierrez requested a review from elezar November 10, 2025 10:39

elezar requested changes Nov 10, 2025

View reviewed changes

elezar reviewed Nov 10, 2025

View reviewed changes

deployments/systemd/hooks.sh Outdated Show resolved Hide resolved

ArangoGutierrez force-pushed the cdi_generate_hook branch 2 times, most recently from 4cac112 to cec05e8 Compare November 10, 2025 10:57

ArangoGutierrez requested a review from elezar November 10, 2025 11:07

elezar previously approved these changes Nov 10, 2025

View reviewed changes

elezar reviewed Nov 10, 2025

View reviewed changes

deployments/systemd/hooks.sh Outdated Show resolved Hide resolved

ArangoGutierrez force-pushed the cdi_generate_hook branch from cec05e8 to 0167933 Compare November 10, 2025 11:35

ArangoGutierrez requested a review from elezar November 10, 2025 11:36

ArangoGutierrez mentioned this pull request Nov 10, 2025

Use functional names for hook bash functions #275

Open

ArangoGutierrez force-pushed the cdi_generate_hook branch from 0167933 to 620e12f Compare November 10, 2025 12:53

elezar reviewed Nov 12, 2025

View reviewed changes

Call nvidia-ctk cdi generate on apply-exit hook

3e1a127

Signed-off-by: Carlos Eduardo Arango Gutierrez <[email protected]>

ArangoGutierrez force-pushed the cdi_generate_hook branch from 620e12f to 3e1a127 Compare November 12, 2025 09:43

ArangoGutierrez requested a review from elezar November 12, 2025 09:43

Extend the functionality of apply-exit to refresh CDI config #215

Are you sure you want to change the base?

Extend the functionality of apply-exit to refresh CDI config #215

Uh oh!

Conversation

ArangoGutierrez commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

elezar Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

ArangoGutierrez Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ArangoGutierrez commented Jul 2, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

elezar commented Jul 10, 2025

Uh oh!

ArangoGutierrez commented Jul 10, 2025

Uh oh!

ArangoGutierrez commented Aug 8, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

ArangoGutierrez commented Aug 15, 2025

Uh oh!

ArangoGutierrez commented Oct 9, 2025

Uh oh!

elezar Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

elezar Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

elezar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ArangoGutierrez commented Nov 10, 2025

Uh oh!

Uh oh!

ArangoGutierrez commented Nov 10, 2025

Uh oh!

elezar Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArangoGutierrez Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

ArangoGutierrez Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

ArangoGutierrez commented Nov 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ArangoGutierrez commented Jul 1, 2025 •

edited

Loading

elezar Nov 12, 2025 •

edited

Loading