runsc delete -force is slow with non-detached container

[stepancheg]

Description

When runsc run -detach is used, runsc delete -force immediately kills the process.

But without -detach flag, this command waits for 5 seconds, then prints "destroying container: stopping container: gofer is still running", and terminates the container.

Steps to reproduce

Using default config.json.

First, properly working version

$ sudo bazel-bin/runsc/runsc_/runsc run -detach -bundle bu xx

$ sudo bazel-bin/runsc/runsc_/runsc list
ID          PID         STATUS      BUNDLE      CREATED                          OWNER
xx          6005        running     bu          2025-08-16T07:40:18.351042194Z   root

$ sudo bazel-bin/runsc/runsc_/runsc delete -force xx
(works fast)

sudo bazel-bin/runsc/runsc_/runsc list
ID          PID         STATUS      BUNDLE      CREATED     OWNER

Now slow version

sudo bazel-bin/runsc/runsc_/runsc run -bundle bu xx

then in another terminal

$ sudo ./bazel-bin/runsc/runsc_/runsc list
ID          PID         STATUS      BUNDLE      CREATED                          OWNER
xx          6209        running     bu          2025-08-16T07:41:44.132236625Z   root

$ sudo bazel-bin/runsc/runsc_/runsc delete -force xx
(waits for 5 seconds then prints the message and exits with error)
destroying container: stopping container: gofer is still running

$ sudo ./bazel-bin/runsc/runsc_/runsc list
ID          PID         STATUS      BUNDLE      CREATED     OWNER

I consider this a bug because:

• -detach flag should not affect how process is terminated
• container process can be terminated faster than in 5 seconds
• container was terminated successfully, so exit code should be zero

It is not a blocker, because there's runsc kill.

runsc version

master

uname

Linux st-devbox.xxx 6.14.0-1012-gcp #12-Ubuntu SMP Wed Jul 16 03:18:29 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

repo state (if built from source)

release-20250804.0-48-g5077500e6

[ayushr2]

Thanks for the bug report. Here is what is happening:

• When you do runsc run without -detach flag, the runsc run process (which is the foreground process in the first terminal from your example) tries to Destroy() the container once the application exits. See the attached code. Notice that when args.Attached == true, cu is not released. Instead, we wait for the sandbox to exit and try deleting the container from there.

  gvisor/runsc/container/container.go

  Lines 522 to 552 in dc0c437

  	func Run(conf *config.Config, args Args) (unix.WaitStatus, error) {
  	log.Debugf("Run container, cid: %s, rootDir: %q", args.ID, conf.RootDir)
  	c, err := New(conf, args)
  	if err != nil {
  	return 0, fmt.Errorf("creating container: %v", err)
  	}
  	// Clean up partially created container if an error occurs.
  	// Any errors returned by Destroy() itself are ignored.
  	cu := cleanup.Make(func() {
  	c.Destroy()
  	})
  	defer cu.Clean()
  	if err := c.Start(conf); err != nil {
  	return 0, fmt.Errorf("starting container: %v", err)
  	}
  	// If we allocate a terminal, forward signals to the sandbox process.
  	// Otherwise, Ctrl+C will terminate this process and its children,
  	// including the terminal.
  	if c.Spec.Process.Terminal {
  	stopForwarding := c.ForwardSignals(0, true /* fgProcess */)
  	defer stopForwarding()
  	}
  	if args.Attached {
  	return c.Wait()
  	}
  	cu.Release()
  	return 0, nil
  	}

• When you call runsc delete from another terminal, it first takes a filesystem mutex lock. It then kills the sandbox process, after which the gofer process also exits. Then it tries to wait for the gofer process to exit with a 5 second timeout (which you are observing):

  gvisor/runsc/container/container.go

  Lines 1181 to 1191 in dc0c437

  	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
  	defer cancel()
  	b := backoff.WithContext(backoff.NewConstantBackOff(100*time.Millisecond), ctx)
  	op := func() error {
  	if err := unix.Kill(c.GoferPid, 0); err == nil {
  	return fmt.Errorf("gofer is still running")
  	}
  	c.GoferPid = 0
  	return nil
  	}
  	return backoff.Retry(op, b)

• The gofer process has already exited, however, unix.Kill(c.GoferPid, 0) keeps returning a nil error (presumably because the parent process of the gofer, which is the runsc run process is still alive).
• The runsc run process is stuck waiting for the above-mentioned filesystem lock so that it can also try to Destroy() the container. Once the 5 second timeout expires, runsc delete fails with "gofer is still running", when it is actually not. It releases the filesystem lock.
• Then runsc run is able to proceed. Since it knows that the gofer is its child, it is able to wait for it properly and exit cleanly:

  gvisor/runsc/container/container.go

  Lines 1171 to 1179 in dc0c437

  	if c.goferIsChild {
  	// The gofer process is a child of the current process,
  	// so we can wait it and collect its zombie.
  	if _, err := unix.Wait4(int(c.GoferPid), nil, 0, nil); err != nil {
  	return fmt.Errorf("error waiting the gofer process: %v", err)
  	}
  	c.GoferPid = 0
  	return nil
  	}

Two ways to fix this that I can think of:

• Do not make runsc run Destroy() the container when --detach is not specified. The caller is expected to explicitly destroy the container. (This will be a breaking change...)
• Fix the unix.Kill(c.GoferPid, 0) check which takes into account potential parent processes of the gofer which are still alive.

[ayushr2]

#12076 fixes this issue, @stepancheg let me know.