docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

[niktheod]

I have docker and nvidia-container-toolkit installed on my system (Ubuntu 24.04) but I keep getting the error on the title when I try to run any image that needs access to gpu.

Here are some more details for my setup:

(base) nikostheodoridis@Aurora1:\~$ docker --version
Docker version 27.4.1, build b9d17ea

(base) nikostheodoridis@Aurora1:\~$ dpkg -l | grep nvidia-container-toolkit
ii nvidia-container-toolkit 1.17.3-1 amd64 NVIDIA Container toolkit
ii nvidia-container-toolkit-base 1.17.3-1 amd64 NVIDIA Container Toolkit Base

(base) nikostheodoridis@Aurora1:\~$ cat /etc/docker/daemon.json
{
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}

(base) nikostheodoridis@Aurora1:\~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nvidia/cuda 12.6.3-cudnn-devel-ubuntu24.04 50efab398f76 6 weeks ago 12.9GB
airsplay/bottom-up-attention latest 66d17015f76c 5 years ago 5.92GB

(base) nikostheodoridis@Aurora1:~$ docker run --rm --gpus all nvidia/cuda:12.6.3-cudnn-devel-ubuntu24.04 nvidia-smi
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].`

Any idea what could be wrong?

[irreversible-paths]

I am having same issue

[SebastianBitsch]

I had the same issue, and i can see you are also on Ubuntu. Maybe you have Docker Desktop installed alongside your CLI installation which is overwriting the context - check with docker context ls. Docker Desktop currently only supports GPU on Windows and WSL2, see: https://docs.docker.com/desktop/features/gpu/ . For me switching to the default context with docker context use default was the solution

[djayvp]

I had the same issue, and i can see you are also on Ubuntu. Maybe you have Docker Desktop installed alongside your CLI installation which is overwriting the context - check with docker context ls. Docker Desktop currently only supports GPU on Windows and WSL2, see: https://docs.docker.com/desktop/features/gpu/ . For me switching to the default context with docker context use default was the solution

@SebastianBitsch : Thank you very much. I was also facing the same issue on Fedora 41. Changing the context to default solved the issue.

[github-actions]

This issue is stale because it has been open 90 days with no activity. This issue will be closed in 30 days unless new comments are made or the stale label is removed. To skip these checks, apply the "lifecycle/frozen" label.

[ArangoGutierrez]

I had the same issue, and i can see you are also on Ubuntu. Maybe you have Docker Desktop installed alongside your CLI installation which is overwriting the context - check with docker context ls. Docker Desktop currently only supports GPU on Windows and WSL2, see: https://docs.docker.com/desktop/features/gpu/ . For me switching to the default context with docker context use default was the solution

Thanks for your help answering this issue.

I am closing this issue. If any problems persist using the latest version of the NVIDIA Container Toolkit, please open a new issue.