Running Docker-in-Docker with Minimal or No Privileges in GitHub Self-Hosted ARC Runners

[jerrinfrancis]

We are running self-hosted GitHub ARC-based runners to support Docker-based workloads. Currently, we are using Docker-in-Docker (DIND) base runners to achieve this. However, DIND requires:

securityContext:
  privileged: true

We are exploring whether there is a way to run DIND without privileged: true, or at least restrict the securityContext to the minimum set of privileges required to make it work.

In Docker's original announcement, it was mentioned that newer versions of Docker support fine-grained permission control, potentially enabling DIND without full privileges.

Could you please confirm:

• Is it possible to run DIND without privileged: true using the latest Docker version?
• If so, what are the minimal security context settings required?
• Are there example configurations or guidelines for setting this up securely?

Any guidance or best practices would be greatly appreciated.

[tianon]

Are there example configurations or guidelines for setting this up securely?

I don't think this is a thing because dockerd requires pretty aggressive permissions to function properly such that providing them all necessarily requires a level of insecurity that's somewhat unavoidable. See also the saga with unprivileged user namespaces being unsupported by default in several major distros because it's inherently dicey from a security perspective within the kernel itself (and my understanding is that the pressure from browsers like Chrome is what turned that tide, not the security posture changing dramatically).

That being said, moby/moby#48828 was the primary thing I'm aware of that --privileged was still the only escape hatch for getting access to, so you might be able to get more specific on flags but I think it's still going to be pretty verbose.

You might also be interested in the saga of https://github.com/canonical/snapd/commits/2ec1611b5989a16d66ed5e857a1a29ecccc55653/interfaces/builtin/docker_support.go (and https://github.com/canonical/snapd/commits/16510ed7470630d352e5197a5d2a6dcdea8551cd/interfaces/builtin/docker.go even further back), which is one of the main efforts I'm aware of where folks have tried to "limit" the access that dockerd has (although even then the end result is still very, very loose).

[tianon]

With the caveat that this is 100% not supported and not recommended, here's a starting point that gets me a successfully running daemon, although trying to run containers fails (and of course with apparmor=unconfined and both SYS_ADMIN and NET_ADMIN this is already extremely wide permissions and can technically do some pretty heinous things):

docker run -it --rm --security-opt writable-cgroups=true --security-opt apparmor=unconfined --cap-add SYS_ADMIN --cap-add NET_ADMIN --name dind docker:dind dockerd

Again, this is kind of a fraught exercise IMO because running containers (regardless of which runtime you use to do so) inherently requires access to privileged interfaces. If you'd like more "secure" ways of running containers (for various values of "secure"), you might want to look into VM-isolation based container runtimes instead (https://github.com/firecracker-microvm/firecracker-containerd, for example).