Docker in Docker is messing up Kubernetes metrics.

[erik-lbg]

Describe the bug
Memory and CPU usage is reported incorrectly when using Docker in Docker.

To Reproduce

1. Set up self-hosted GitHub actions runners in Kubernetes using: https://github.com/actions/actions-runner-controller
2. Use the run in container feature to run your job: https://docs.github.com/en/actions/writing-workflows/choosing-where-your-workflow-runs/running-jobs-in-a-container
3. Check memory and CPU usage of your runner pod in Kubernetes.

Expected behavior
You can see the memory and CPU usage of your runner.

Runner Version and Platform

Version of your runner? 2.324.0

OS of the machine running the runner? Linux

What's not working?

The memory and CPU used inside the Docker in Docker container is not reported with the usage of the runner pod. The runner pods all look pretty idle, but in reality they are running really heavy jobs with Docker in Docker. Somehow the OEM reaper doesn't get fooled and is still able to kill the pod for using too much memory.

Job Log Output

N/A

Runner and Worker's Diagnostic Logs

N/A

Humble suggestion
If the container could be added to the pod, instead of using Docker in Docker, that container's resource usage should be reported correctly in Kubernetes. (I'm assuming that would be difficult to accomplish with the current architecture, since the runner doesn't now what job it will pick up when it's started and you can't add containers to a running pod.)

If there are ways to configure Docker in Docker that makes it report resource usage properly, that's obviously a perfectly fine solution as well.