Skip to content

Conversation

@cdesiniotis
Copy link
Collaborator

@cdesiniotis cdesiniotis commented Nov 11, 2025

This PR rewrites the vfio-manage script in Go. This sets the stage for a follow-up #128 which adds support for variant VFIO modules besides vfio-pci.

Testing

I acquired a machine that has one L40 and one L4. I configured the L40 GPU into graphics mode using https://developer.nvidia.com/displaymodeselector. Below is the initial state of the system. Notice that the L40 has an auxiliary PCI device.

# lspci -nnk -d 10de:
17:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD102GL [L40] [10de:26b5] (rev a1)
	Subsystem: NVIDIA Corporation AD102GL [L40] [10de:169d]
	Kernel modules: nvidiafb, nouveau
17:00.1 Audio device [0403]: NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
	Subsystem: NVIDIA Corporation AD102 High Definition Audio Controller [10de:169d]
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
3d:00.0 3D controller [0302]: NVIDIA Corporation AD104GL [L4] [10de:27b8] (rev a1)
	Subsystem: NVIDIA Corporation AD104GL [L4] [10de:16ca]
	Kernel modules: nvidiafb, nouveau

Bind

# ./vfio-manage bind --all
INFO[2025-11-20T00:10:12Z] Binding device 0000:17:00.0
INFO[2025-11-20T00:10:12Z] Binding device 0000:3d:00.0

#  lspci -nnk -d 10de:
17:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD102GL [L40] [10de:26b5] (rev a1)
	Subsystem: NVIDIA Corporation AD102GL [L40] [10de:169d]
	Kernel driver in use: vfio-pci
	Kernel modules: nvidiafb, nouveau
17:00.1 Audio device [0403]: NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
	Subsystem: NVIDIA Corporation AD102 High Definition Audio Controller [10de:169d]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel
3d:00.0 3D controller [0302]: NVIDIA Corporation AD104GL [L4] [10de:27b8] (rev a1)
	Subsystem: NVIDIA Corporation AD104GL [L4] [10de:16ca]
	Kernel driver in use: vfio-pci
	Kernel modules: nvidiafb, nouveau

Unbind

# ./vfio-manage unbind --all
INFO[2025-11-20T00:28:47Z] Unbinding device 0000:17:00.0
INFO[2025-11-20T00:28:47Z] Unbinding device 0000:3d:00.0

root@smc741ge-0014:/localhome/local-cdesiniotis# lspci -nnk -d 10de:
17:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD102GL [L40] [10de:26b5] (rev a1)
	Subsystem: NVIDIA Corporation AD102GL [L40] [10de:169d]
	Kernel modules: nvidiafb, nouveau
17:00.1 Audio device [0403]: NVIDIA Corporation AD102 High Definition Audio Controller [10de:22ba] (rev a1)
	Subsystem: NVIDIA Corporation AD102 High Definition Audio Controller [10de:169d]
	Kernel modules: snd_hda_intel
3d:00.0 3D controller [0302]: NVIDIA Corporation AD104GL [L4] [10de:27b8] (rev a1)
	Subsystem: NVIDIA Corporation AD104GL [L4] [10de:16ca]
	Kernel modules: nvidiafb, nouveau

@coveralls
Copy link

coveralls commented Nov 11, 2025

Pull Request Test Coverage Report for Build 19586910222

Details

  • 0 of 263 (0.0%) changed or added relevant lines in 4 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 0.0%

Changes Missing Coverage Covered Lines Changed/Added Lines %
cmd/vfio-manage/main.go 0 19 0.0%
cmd/vfio-manage/bind.go 0 62 0.0%
cmd/vfio-manage/unbind.go 0 62 0.0%
internal/nvpci/nvpci.go 0 120 0.0%
Totals Coverage Status
Change from base Build 19564974858: 0.0%
Covered Lines: 0
Relevant Lines: 1111

💛 - Coveralls

@cdesiniotis cdesiniotis marked this pull request as ready for review November 18, 2025 21:42
Signed-off-by: Christopher Desiniotis <[email protected]>
@cdesiniotis cdesiniotis force-pushed the rewrite-vfio-manager-in-go branch from c392090 to b5bce6d Compare November 19, 2025 22:54
@cdesiniotis cdesiniotis force-pushed the rewrite-vfio-manager-in-go branch from 2a41ed3 to e95a39b Compare November 20, 2025 01:12
Copy link
Member

@karthikvetrivel karthikvetrivel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for your effort on this, Chris.

@cdesiniotis cdesiniotis force-pushed the rewrite-vfio-manager-in-go branch from e95a39b to 7002214 Compare November 22, 2025 00:06
@cdesiniotis
Copy link
Collaborator Author

@tariq1890 @karthikvetrivel thanks for the reviews and approvals. I just updated this PR since I noticed I did not add the license / copyright header to the internal/nvcpi/nvpci.go file. As this is the only change made since your approval, will proceed to merge.

@cdesiniotis cdesiniotis merged commit ab47270 into main Nov 22, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants