FUSE mounts inside a sandbox cannot be unmounted — umount2() returns ENOENT

[qiffang]

FUSE mounts inside a sandbox cannot be unmounted — umount2() returns ENOENT

Summary

We've been running some FUSE-based tooling inside Daytona sandboxes and noticed that unmounts don't seem to work while the sandbox is live. Every unmount path we tried (fusermount3, umount, direct umount2() syscall) returns ENOENT, even though the mount is present in /proc/self/mountinfo and reads/writes work normally. The mount table entry only clears when the sandbox is destroyed.

The symptoms line up closely with a sysbox-fs issue that was fixed by nestybox/sysbox-fs#103 (merged 2025-01-17, tracking nestybox/sysbox#854). We're wondering if the sysbox-fs component on the Daytona runners might be from before that fix landed. Happy to be corrected if we've misdiagnosed — the reproduction below should let you check on your side.

Why this matters for us

Any tool that mounts FUSE inside a sandbox seems to run into this — sshfs, rclone, s3fs, juicefs, our own drive9 mount, and even a plain libfuse "hello" example. A few things this makes awkward:

• umount / fusermount3 -u don't clean up, so the mount table entry stays around until the sandbox is destroyed.
• Re-mounting the same path in a long-lived sandbox fails with "already mounted".
• Tools that check "did the unmount succeed?" before continuing (backup, snapshot, log rotation) end up blocking or erroring out.
• FUSE storage-client test suites can't exercise the mount/unmount lifecycle inside a sandbox.

For short-lived sandboxes this is easy to live with (see workaround below); for long-running ones it's a bit of a papercut.

Environment

• Daytona API endpoint: https://app.daytona.io/api
• Sandbox snapshot: daytona-small (image daytonaio/sandbox:0.9.0, sandboxClass: container). Also reproduced on daytona-medium.
• Host kernel (varies by runner): Linux 6.17.0-42-generic, Linux 6.8.0-139-generic
• Guest OS: Debian 13 (trixie); fuse3 3.17.2; util-linux 2.41
• Sandbox user: daytona (uid 1001)
• Sandbox process 1: daytona (uid 1001), Seccomp: 2 (SECCOMP_MODE_FILTER), 1 filter loaded

Evidence the sandbox runtime is sysbox

• sysboxfs FUSE mounts are present on /proc/sys, /proc/uptime, /proc/swaps, /sys/kernel, /sys/devices/virtual, /sys/module/nf_conntrack/parameters:

  fuse sysboxfs rw,user_id=0,group_id=0,default_permissions,allow_other
  

• unshare --user --map-root-user echo ok fails with write failed /proc/self/uid_map: Operation not permitted (sysbox restricts this even though the container has its own user namespace — a documented sysbox behavior).
• Container runs in a user namespace with /proc/self/uid_map = 0 <host-base> 65536.

Minimal reproduction

# Inside a sandbox as user `daytona` (uid 1001).
sudo apt-get install -y fuse3 strace

# Mount any FUSE filesystem. The bug is generic to FUSE — this can be sshfs,
# s3fs, libfuse "hello", etc. We used our own `drive9 mount` while investigating.
mkdir -p ~/mnt
<any-fuse-tool> mount ~/mnt

# Confirm mount is live and readable
grep fuse /proc/self/mountinfo
ls ~/mnt          # succeeds

# Every unmount path fails:
fusermount3 -u ~/mnt
# → fusermount3: failed to unmount /home/daytona/mnt: No such file or directory

sudo strace -e trace=umount2 -f umount ~/mnt 2>&1 | grep umount2
# → umount2("/home/daytona/mnt", 0) = -1 ENOENT (No such file or directory)
# followed by: umount: /home/daytona/mnt: no mount point specified.

# Verify the mount is still present after every attempt:
grep fuse /proc/self/mountinfo   # still shows the FUSE mount

Reproduced 3/3 with fresh sandboxes.

Control: non-FUSE mounts unmount fine

Same sandbox, same user, same syscall path:

sudo mount -t tmpfs none /tmp/tmpfs
sudo strace -e trace=umount2 -f umount /tmp/tmpfs 2>&1 | grep umount2
# → umount2("/tmp/tmpfs", 0) = 0

tmpfs, ext4 bind mounts, and other in-kernel filesystems unmount successfully. The failure is specific to FUSE-type mounts.

Why we suspect this is nestybox/sysbox-fs#103

The PR description states:

In the umount syscall processing, rather than performing the PathAccess() check upfront [...] delay it until we know it refers to a sysbox-fs managed mount [...] This means that for other mountpoints (i.e., those not managed by sysbox-fs), we let the kernel do the path access check since it's the kernel that processes those mounts anyway.

Why this change? Because this allows us to work-around a problem where while processing the umount syscall, PathAccess() unexpectedly fails on a FUSE-backed mount inside the container [...] The failure is caused by the FUSE-backed mount not showing up properly under /proc/<pid>/root/<path-to-fuse-mountpoint> [...] This behavior does not reproduce on other mountpoint types, which is why we didn't detect this problem before (e.g., I tried with a tmpfs mount inside the container, and verified that /proc/<pid>/root/<path-to-tmpfs-mountpoint> works fine).

This lines up with what we're seeing:

• On an ordinary kernel path, umount2() on a real mountpoint returns 0 or EPERM, not ENOENT. Something in userspace has to be intercepting it to get ENOENT on a mount that clearly exists in /proc/self/mountinfo.
• The failure is limited to FUSE mounts — tmpfs unmounts succeed under identical conditions in the same sandbox.
• The sandbox has the sysbox-fs fingerprints listed above.

If the sysbox-fs build shipped with the runners already contained PR #103, we'd expect umount2() to fall through to the kernel and succeed. Since it doesn't, our best guess is that the runners are on a pre-#103 sysbox-fs. Very possible we're missing something though — this is inference from the outside.

What would help

If it's easy to check, could you confirm the sysbox / sysbox-fs version running on the sandbox runners, and if applicable, plan an upgrade to a release that includes PR #103 (anything tagged after 2025-01-17)? Happy to run any additional diagnostics on our side if that helps narrow it down.

Current workaround

For our use case we've been getting by with sandbox.delete() — tearing the sandbox down on the host side cleans everything up, since that path doesn't go through the in-container umount. That's fine for short-lived, single-use sandboxes. We haven't found anything that works from inside a long-lived sandbox: killing the FUSE server aborts the connection but leaves the mount entry behind (Transport endpoint is not connected), and every umount syscall path hits the ENOENT above.

References

• sysbox-fs PR: Fix for issue 854 nestybox/sysbox-fs#103
• sysbox tracking issue: umount2 fails with ENOENT (No such file or directory) nestybox/sysbox#854