fix: make a meta write safe against a second writer (T2.1.13) - #44
Merged
Conversation
Two overlapping PATCHes to one slug lost a field on the first round and left an unparseable meta.json from the second, which took the artifact out of the list, 404'd /a/<slug> and made it refuse its own DELETE. Two halves: storage/local.js writes a scratch file and renames it into place, so a reader sees either every old byte or every new one. It carries the target's existing mode over, because a rename brings a new inode and would have handed auth.json back at 0644 after an operator ran the chmod 600 in docs/deploy.md. s3, postgres and sqlite were already whole-object writes; the git backend reuses local. server.js gained withMetaChain, a per-slug write queue shaped like the one lib/auth.js uses for auth.json. Publish, replace, zip deploy, duplicate, patch and delete all re-read the record inside it, so a write changes what the backend holds rather than what the request found on arrival. A rename and a copy hold both names through withMetaChains, sorted so two renames that cross cannot wait on each other. A slug from a JSON body is settled to a string first: 123 and "123" named one directory and keyed two queues. null still means the caller left it out. npm test 49 to 55, smoke 154 to 162 ok-lines.
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What the item was
T2.1.13.
storage/local.jswrote an object with a barefs.writeFile, and every meta write was afull-record rewrite. Two PATCHes to one slug that overlapped interleaved their bytes: the shorter
write landed inside the longer one,
meta.jsonstopped parsing,readMetaswallowed the parseerror and answered
null. The artifact then vanished fromGET /api/artifacts, answered 404 on/a/<slug>, and refused its own DELETE with a 404. The only way back was deleting the directory byhand.
Reproduced red before touching anything, five rounds of two overlapping PATCHes on
origin/main:Same five rounds on this branch:
tags=[raced] project=[Race]every round, serve 200, delete 200.What changed
The object write lands whole.
storage/local.jswrites a scratch file beside the target andrenames it into place. Rename inside one filesystem swaps the whole object, so a reader gets either
every old byte or every new one. The scratch file is created
wxat mode 0600, then takes thetarget's existing mode before the rename: a rename brings a new inode, and without that step the
chmod 600 auth.jsonindocs/deploy.mdcame undone on the next ordinary write, because a managedkey's
lastUsedAtgoes through the same path. s3 (one PUT) and postgres/sqlite (one upsert) werealready whole-object writes and are documented as such. The git backend reuses the local store.
Meta writes merge instead of replacing.
server.jsgainedwithMetaChain, a per-slug writequeue in the shape of the one
lib/auth.jsuses forauth.json. Publish, replace, zip deploy,duplicate, patch and delete all re-read the record inside it, so a write changes what the backend
holds rather than what the request found when it arrived. A rename and a copy write under two names,
so they hold both through
withMetaChains, sorted and de-duplicated: two renames that cross(
atobwhilebtoa) would otherwise take the two queues in opposite orders and wait oneach other forever.
A slug from a JSON body is settled to a string first.
SLUG_RE.test(123)coerced on the waythrough, so
123and"123"named one directory and keyed two independent queues, which put bothwriters back to believing they were alone.
nullstill means the caller left it out.Scratch files never travel. A killed process leaves one behind holding the whole record it was
writing, which for
meta.jsonincludes the view-password hash. They are swept at startup (root andone level down, where
auth.jsonand everymeta.jsonlive), skipped bycopySlugso a duplicatecannot inherit the source's password hash, and filtered out of the git backend's
statusMatrixsoone is never committed and pushed.
Tests
npm test49 to 55. Six new tests intest/storage-local.test.jscover the torn write, theleftover scratch file, the failed write, and the file mode.
bash .github/workflows/smoke.sh154 to 162 ok-lines, all green against the local backend. Newcases: PATCH+PATCH, PUT+PATCH, DELETE+PATCH, a rename racing a publish for one destination, two
renames that cross, and a
nullslug. Every backgrounded call carries--max-time 15, so a queuethat stopped settling fails the job instead of hanging it.
Browser pass on the local dashboard at 1280x900: created the admin, opened a row menu, moved
release-notesinto the Acme project through one PATCH. The row moved, the group count went to 2,the
docstag survived, the timestamp updated. Console reported 0 errors and 0 warnings.Review
Four lenses (adversarial twice, security, QA) on a 381-line diff, 27 items. Two were regressions
this change introduced and both are fixed:
patch's scratch write and its rename, which answered 500, or recreated
meta.jsonunder anemptied directory and left a listed row serving 404.
origin/mainscored 0 in 20; the extrasyscall opened it. Putting
deleteArtifacton the same queue closes it: 0 in 20, and the smokesuite now covers it.
process.umask()called per write. Its no-argument form isumask(0)thenumask(old), sothe mask is 0 for a moment and any directory created in that window comes out world-writable.
6000 concurrent publishes produced four artifact directories at 0777, and
meta.jsonlives inone of those. Read once at module load now. 1200 concurrent publishes: 0755 every time.
Fixed inline, all pre-existing or introduced above:
{"slug": null}published an artifact literally namednulland 409'd every publish after it.nullmeans the caller left it out again, which is how a JS client writes{ slug: form.slug || null }.check and the move: 19 rounds in 20 answered 500 on
origin/main, and when the move won insteadit left a private artifact's bytes under the publish's public record. Now 200/409 or 409/201,
never both, over 15 rounds.
loser's bytes stayed on disk with nothing serving them. Now exactly one winner over 15 rounds.
duplicateArtifactdid the same read-then-write on its 409 guard.copySlugand the git backend carried scratch files, so a duplicate could inherit the source'spassword hash and a push could put one in history that outlives the artifact.
fs.chmodwas passed the file-type bits fromstat().mode, andfs.stat().catch(() => null)swallowed a permission error and downgraded a hardened file to 0644.
a non-slug briefly became a queue key.
puta numberthrows in argument validation before a file exists, so the cleanup branch never ran. It now makes
the rename fail instead.
storage/sqlstore.js, one run-on in the newdocs/deploy.mdparagraph, and a comment ondeleteArtifactthat described the wrong symptom.Filed rather than fixed:
has no ceiling and no timeout, and
storage/s3.jspasses noAbortSignalon any fetch, so astalled endpoint parks every later write on that slug for the life of the process. Before the
queue a hung write cost one request. This needs two calls only Z can make, how long a storage
call may take and whether the answer is a 503, so it is a ticket rather than a guess.
Recorded, no action:
fsyncon the file or its parent directory before the rename, so a power cut can still leavea zero-length
meta.jsonon ext4 or xfs. Pre-existing (a barewriteFilenever synced either);the item asked for atomic, not durable.
storage/index.jsnow says so plainly rather thanimplying more.
wrong in two ways reads a different 400 message. Both are 400 and nothing asserts either string.
PUT /api/artifacts/ABanswers 400 for a path segment that is not a slug whilePATCHandDELETEanswer 404 for the same thing. Pre-existing inconsistency, out of scope here.{"slug": false}answered 201 with a generated slug and now answers 400.docs/api.mddocumentsslugas a string, and nothing in the CLI, the MCP tools or the dashboard can send a boolean.Reasoning in AGENT-DECISIONS.md, 2026-08-12.
Docs
docs/deploy.mdgained a section on two writes to one artifact, next to the one that alreadyexplains the same shape for
auth.json, including what a fleet still cannot rely on.No screenshot artifact: the diff changes no UI, so the browser pass above is a regression check
rather than something to look at.