Tolerate host mounts where a create is not immediately visible to lookup - #160
Tolerate host mounts where a create is not immediately visible to lookup#160ezufelt wants to merge 3 commits into
Conversation
On a Docker Desktop / virtiofs bind mount of a macOS host, mkdir, writeFile, and symlink return success but lstat/realpath/existsSync on the path they just created report ENOENT for the next 1-2ms. open, read, and write on that path work immediately -- the lookup path lags, not the data. Measured visibility after mkdir: 0/10 at 0ms, 9/10 at 1ms, 10/10 at 2ms. Sandboxed agent environments commonly mount the workspace this way. deepsec assumed POSIX durability-on-return in a few places, so init failed with "ENOENT: no such file or directory, mkdir 'data/<projectId>'" -- recursive mkdir creates data/, then cannot see it when it looks up the child. Adds settleFs, existsSettled, and mkdirSettled: a flat 100ms wait after creating a path, tunable via DEEPSEC_FS_SETTLE_MS (0 opts out). Not a poll-until-visible loop -- this is a handful of call sites on one code path, and the simplicity is worth more than the milliseconds. mkdirSettled retries the mkdir once on ENOENT, since that is where the failure surfaces; any other errno is a real error and is rethrown untouched. Used only where this process just created the path: ensureProject's data dir, the init workspace dir, init-project's data dir, and the post-install node_modules probe. Resume detection, atomic-file temp cleanup, and the mkdir lock primitive are left alone -- those paths were created by an earlier process or are legitimately absent most of the time, where waiting is pure cost. Refs vercel-labs#159 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@ezufelt is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
|
Clanker feedback: • Found one blocking issue:
The new core tests pass locally, but they don’t exercise this installer path. The PR is currently blocked by unauthorized Vercel deployment |
…le-non-coherent-mounts
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional Suggestion:
On a lagging/non-coherent mount, ensureWorkspaceInstall's post-install probe uses raw existsSync on files the install child just wrote, so a fresh deepsec init can spuriously throw "Install completed but workspace is unusable".
existsSettled only guarded the install checkpoint check, which runs when a checkpoint already exists -- so a fresh init skipped it and then probed the just-written node_modules with a raw existsSync, where the same create-to-lookup lag reports "Install completed but workspace is unusable" for a good install. probeWorkspaceInstall takes a settle option that routes its existence checks through existsSettled; the post-install call passes it. The pre-install probe keeps the raw check, since an absent node_modules is the normal case there and waiting on it is pure cost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Thanks for the feedback, I pushed a new commit. |
2 participants
On a Docker Desktop / virtiofs bind mount of a macOS host, mkdir,
writeFile, and symlink return success but lstat/realpath/existsSync on
the path they just created report ENOENT for the next 1-2ms. open, read,
and write on that path work immediately -- the lookup path lags, not the
data. Measured visibility after mkdir: 0/10 at 0ms, 9/10 at 1ms, 10/10 at
2ms. Sandboxed agent environments commonly mount the workspace this way.
deepsec assumed POSIX durability-on-return in a few places, so init
failed with "ENOENT: no such file or directory, mkdir 'data/'"
-- recursive mkdir creates data/, then cannot see it when it looks up the
child.
Adds settleFs, existsSettled, and mkdirSettled: a flat 100ms wait after
creating a path, tunable via DEEPSEC_FS_SETTLE_MS (0 opts out). Not a
poll-until-visible loop -- this is a handful of call sites on one code
path, and the simplicity is worth more than the milliseconds. mkdirSettled
retries the mkdir once on ENOENT, since that is where the failure
surfaces; any other errno is a real error and is rethrown untouched.
Used only where this process just created the path: ensureProject's data
dir, the init workspace dir, init-project's data dir, and the post-install
node_modules probe. Resume detection, atomic-file temp cleanup, and the
mkdir lock primitive are left alone -- those paths were created by an
earlier process or are legitimately absent most of the time, where
waiting is pure cost.
Refs #159
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com