nvidia-fs-ctr missing path to gcc-12 on v25.10.0

[kreeuwijk]

Describe the bug
When upgrading from GPU Operator 25.3.4 to 25.10.0, having the GDS option enabled in both versions, the nvidia-fs-ctr container fails to build the module due to gcc-12 not being found. The binary is available in the container under /run/nvidia/driver/usr/bin but this isn't part of the PATH.

To Reproduce
Install GPU Operator 25.10.0 and enable GDS with gds.enabled: true. Notice the nvidia-fs-ctr container of the driver pod goes into crashloop.

Expected behavior
The gcc binaries should be available by including /run/nvidia/driver/usr/bin in the PATH.
If I manually edit the args parameter for this container in the daemonset to augment the PATH variable, it works:

args:
  - until [ -d /run/nvidia/driver/usr/src ] && lsmod | grep nvidia; do echo  Waiting for nvidia-driver to be installed...; sleep 10; done; PATH=/run/nvidia/driver/usr/bin:$PATH; exec nvidia-gds-driver install

Environment (please provide the following information):

• GPU Operator Version: v25.10.0
• OS: Ubuntu22.04
• Kernel Version: 6.5.0-generic
• Container Runtime Version: containerd 2.1.4
• Kubernetes Distro and Version: Kubeadm 1.33.4

[kreeuwijk]

Adjusting the gds: section of the chart to override the PATH variable also worked for me:

gds:
  enabled: true
  repository: nvcr.io/nvidia/cloud-native
  image: nvidia-fs
  version: "2.26.6"
  imagePullPolicy: IfNotPresent
  imagePullSecrets: []
  env:
    # Fix "gcc-12: file not found" error as gcc-12 is in /run/nvidia/driver/usr/bin which isn't in the PATH by default
    - name: PATH
      value: /run/nvidia/driver/usr/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  args: []

[tariq1890]

Thanks for reporting this @kreeuwijk . We are investigating this and we are looking into installing gcc-12 and making that the default gcc version in the nvidia-fs container