Skip to content

fix(i18n): restore manifest change-tracking, dead since the manifest passed 1 MiB - #1973

Merged
zksquirrel merged 1 commit into
ZecHub:mainfrom
bloxster:i18n/fix-invariants-fail-open
Aug 17, 2026
Merged

fix(i18n): restore manifest change-tracking, dead since the manifest passed 1 MiB#1973
zksquirrel merged 1 commit into
ZecHub:mainfrom
bloxster:i18n/fix-invariants-fail-open

Conversation

@bloxster

Copy link
Copy Markdown
Contributor

The gate has been dead for two days

manifest-invariants stopped enforcing its change-tracking invariants on 2026-08-15, and kept reporting green the whole time.

check-invariants.mjs read the base manifest through execFileSync with no maxBuffer, so it inherited Node's 1 MiB default. translation/sync-state.json crossed that threshold at 2dd8fb0f — the 18-page sync merged in #1964 — going 1,001,048 → 1,094,558 bytes. From that commit on, git show threw ENOBUFS on every run.

The read sat in a catch that treated any failure as "the manifest did not exist at the base — first introduction, nothing to change-track". So both directions were skipped on every pull request, including #1965 and #1966. The bijection checks don't touch that read, which is precisely why nothing looked wrong.

What was unenforced

  • direction 1 — a translated file could change with no provenance trace. That invariant is what stops a hand-edit from being indistinguishable from a page nobody re-synced.
  • direction 2src could be bumped without the file changing, marking a stale translation fresh forever. That is the single lie the manifest exists to prevent, and by design it has no escape hatch.

The fix, and two adjacent instances of the same mistake

  • Bound every git read at 64 MiB, matching what scripts/check-protected-terms.mjs already does. git ls-files translations/ is 210 KB today and scales with locales × pages, so it is on the same path.

  • Establish absence instead of inferring it, with git ls-tree rather than git cat-file -e. Under ls-tree, absent and present are both exit 0 and are told apart by the output, so a non-zero exit is unambiguously an error. cat-file -e cannot draw that line: it exits non-zero both for an absent path and for a blob it could not obtain, so under --filter=blob:none an unreachable promisor remote would read as "absent" and skip the gate again — the same silent skip with a network trigger. ls-tree also never needs the blob at all.

  • Reject a base manifest that parses but is not an object. null, an array or a string would parse and then throw on the first baseManifest[loc].

  • Fail on --base with no ref value. It resolved to undefined, which read as "no base requested" and skipped change-tracking with a notice and exit 0. CI invokes --base "$BASE_REF_OUT", so an empty variable would have landed exactly there.

Verification

Every case run on this tree against base=origin/main, comparing the checker at origin/main with this one:

case before after
clean tree "no manifest", skipped change-tracked, holds
translation edited, manifest untouched passed green fails, direction 1
src bumped, file untouched passed green fails, direction 2
manifest genuinely absent at base notice, skip notice, skip (unchanged)
base manifest is null TypeError stack trace fails, names the reason
--base with no value notice, exit 0 fails, exit 1
no --base at all notice, exit 0 notice, exit 0 (unchanged)

Review

Two independent adversarial passes, both of which changed the patch.

One reproduced the mask on a real violation, and settled the cat-file -e question empirically in a synthetic blobless clone: with the promisor reachable cat-file -e lazy-fetches and exits 0, but with it unreachable it exits 128 for a blob that exists — indistinguishable from a genuine absence, and GIT_NO_LAZY_FETCH=1 gives exit 1 with the same misclassification. It then verified that ls-tree answers correctly from the tree objects with the blob missing and the promisor unreachable. That finding is why this PR uses ls-tree.

The other found the --base and base-shape defects, neither of which I had looked for.

Not fixed here

Filed rather than folded in, none of them live:

  • unbounded git reads in detect-staleness.mjs (:89, :125, :219), seed-sync-state.mjs (:87) and gen-menu-titles.mjs (:52) — all 30–2000× under the ceiling, and scaling linearly rather than with the locale × page product;
  • this file's ls-files calls don't use -z, so core.quotePath would mangle a non-ASCII path. seed-sync-state.mjs:85 documents that exact trap and uses -z; the corpus is ASCII today;
  • the manifest-invariants push branch passes $EVENT_BEFORE with no zero-SHA fallback, while the protected-terms job has one, so a force-push to main goes red under the fail-closed rev-parse. That predates this change.

Suggested merge order

This one first. It restores a required check, and it is independent of #1966 and of the contributor-doc PR.

…passed 1 MiB

`check-invariants.mjs` read the base manifest with `execFileSync` and no
`maxBuffer`, so it inherited Node's 1 MiB default. `translation/sync-state.json`
crossed that threshold at 2dd8fb0 (the 18-page sync merged in ZecHub#1964, 1,001,048
-> 1,094,558 bytes), and from that commit on `git show` threw ENOBUFS on every
run.

The read sat in a catch that treated any failure as "the manifest did not exist
at the base — first introduction, nothing to change-track". So both
change-tracking directions were skipped on every pull request since 2026-08-15
while the job printed `Manifest invariants hold` and stayed green. The bijection
checks don't touch that read, which is precisely why nothing looked wrong.

What was unenforced in that window:

- direction 1 — a translated file could change with no provenance trace, the
  invariant that stops a hand-edit from being indistinguishable from a page
  nobody re-synced;
- direction 2 — `src` could be bumped without the file changing, marking a stale
  translation fresh forever. That is the single lie the manifest exists to
  prevent, and by design it has no escape hatch.

The fix, and two adjacent instances of the same mistake found while reviewing it:

- Bound every git read in this file at 64 MiB, matching what
  scripts/check-protected-terms.mjs already does. `git ls-files translations/`
  is 210 KB today and scales with locales x pages, so it is on the same path.

- Establish absence instead of inferring it, using `git ls-tree` rather than
  `git cat-file -e`. Absent and present are both exit 0 under `ls-tree` and are
  told apart by the output, so a non-zero exit is unambiguously an error.
  `cat-file -e` cannot draw that line: it exits non-zero both for an absent path
  and for a blob it could not obtain, so under `--filter=blob:none` an
  unreachable promisor remote would read as "absent" and skip the gate again —
  the same silent skip with a network trigger. `ls-tree` also never needs the
  blob at all.

- Reject a base manifest that parses but is not an object. `null`, an array or a
  string would parse and then throw on the first `baseManifest[loc]`.

- Fail on `--base` with no ref value. It resolved to `undefined`, which read as
  "no base requested" and skipped change-tracking with a notice and exit 0. CI
  invokes `--base "$BASE_REF_OUT"`, so an empty variable would have landed
  exactly there. Asking for the gate and silently not getting it is the failure
  this whole block exists to prevent.

Verified on this tree, base origin/main:

| case | before | after |
|---|---|---|
| clean tree | "no manifest", skipped | change-tracked, holds |
| translation edited, manifest untouched | passed green | fails, direction 1 |
| `src` bumped, file untouched | passed green | fails, direction 2 |
| manifest genuinely absent at base | notice, skip | notice, skip (unchanged) |
| base manifest is `null` | TypeError stack trace | fails, names the reason |
| `--base` with no value | notice, exit 0 | fails, exit 1 |
| no `--base` at all | notice, exit 0 | notice, exit 0 (unchanged) |

Reviewed by two independent adversarial passes. One reproduced the mask on a
real violation and established the `cat-file -e` / `ls-tree` distinction
empirically in a synthetic blobless clone: with the promisor unreachable,
`cat-file -e` exits 128 for a blob that exists, indistinguishable from a genuine
absence, while `ls-tree` answers correctly from the tree objects. The other found
the `--base` and base-shape defects. Both are fixed here.

Not fixed, filed instead: unbounded git reads in detect-staleness.mjs,
seed-sync-state.mjs and gen-menu-titles.mjs (all 30-2000x under the limit and
scaling linearly rather than with the locale x page product), and this file's
`ls-files` calls not using `-z`, which seed-sync-state.mjs documents as a
non-ASCII filename trap.
@bloxster

Copy link
Copy Markdown
Contributor Author

Confirmed in CI, side by side

The two pull requests I opened minutes apart give the before/after directly, because #1974 runs the checker as it exists on main and this one runs the fixed checker. Same base, same corpus, same minute:

#1974manifest-invariants, checker from main:

notice: base cf08a502293c has no manifest — first introduction, nothing to change-track.
Manifest invariants hold: 203 curated pages, 18 locales.

That notice is the bug. The manifest plainly exists at cf08a502 — the job simply could not read it past the 1 MiB default, and reported the failure as absence. Both change-tracking directions were skipped, and the job went green.

#1973 — same job, this branch:

Manifest invariants hold: 203 curated pages, 18 locales.

No notice, because there was nothing to skip. Change-tracking ran.

@zksquirrel
zksquirrel merged commit 18856fb into ZecHub:main Aug 17, 2026
4 checks passed
zksquirrel pushed a commit that referenced this pull request Aug 19, 2026
`pages build and deployment` has been failing since 2026-08-14, and the published
GitHub Pages site (https://zechub.github.io/zechub/, status `errored`) has been
frozen on that day's build ever since.

The LLM sync prepends a `---` that the English source does not have. Jekyll reads
a leading `---` as a YAML front-matter opener, runs to the next `---` far down the
page, and fails to parse. The build dies on
translations/ja/site/Zcash_Tech/What_a_Block_Explorer_Can_See.md.

Emulating Jekyll's front-matter parse across the whole tree: 251 translated pages
break, 0 English pages do. Every breaking file is a translation, and in all 251
the English source has no leading `---`, so there is no judgment call — the
dashes are simply not supposed to be there.

Traced by counting leading-`---` files at each merge:

| commit | files | pages build |
|---|---|---|
| d951213 (#1948) | 96 | last success |
| c301ff2 (#1960) | 127 | first failure |
| 636861a (#1964) | 356 | failing |

96 such files predate this and the build was green, so a leading `---` is not
automatically fatal — it breaks when the resulting pseudo-front-matter is invalid
YAML, which prose in these locales reliably produces. #1960 tipped it over and
#1964 made it much worse.

This strips the stray line from all 260 pages where the English source lacks it,
not only the 251 that currently break, since the other 9 are the same defect and
a latent break.

Reader-facing impact was limited: the frontend calls next-mdx-remote's
`serialize(..., {})`, whose `parseFrontmatter` defaults to false, and gray-matter
— though in package.json — is imported nowhere in src/, so the dashes rendered as
a stray horizontal rule rather than swallowing content. Worth noting gray-matter
DOES throw on these files, so anything that starts using it would break hard.

Two sidebar labels were wrong, though, and this fixes them: `Using_Zcash/Zecmap.md`
read "Kurzfassung" in de and "Resumo" in pt — both meaning "summary" — because the
stray `---` made the title extractor skip `# ZecMap` and pick up `## TL;DR`. Both
now read "ZecMap". Menu-title manifests regenerated accordingly.

No prose changed: the only content line touched anywhere is the bare `---`.
`src` is untouched because it hashes the ENGLISH source, which has not moved.
`edited` stays false throughout — this is a mechanical pass, and flipping it
would remove these pages from automated sync permanently.

Verification, all on this branch:

| check | result |
|---|---|
| Jekyll front-matter emulation | 0 breaking pages, was 251 |
| protected-terms (scoped) | passed, 3,654 pages |
| manifest-invariants | holds, 203 x 18 |
| translation lib tests | 51 pass / 0 fail |
| menu-titles-fresh | up to date, 19 files |
| non-`---` content lines changed | 0 |

Incidentally a first real exercise of #1973: with the identical content repair but
the manifest left untagged, `manifest-invariants` now fails with 260 violations.
Before #1973 it passed green, so this repair could have landed with no provenance
trace at all.

Not done here, having checked and found nothing wrong: the 108 translated pages
using the markdown badge form `[![Edit](...)](...)` rather than the HTML
`<a><img/></a>` form. All 108 match their English source exactly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants