Skip to content

fix(i18n): repair broken front matter in 302 translated pages - #1975

Merged
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/repair-frontmatter-and-editbadge
Aug 19, 2026
Merged

fix(i18n): repair broken front matter in 302 translated pages#1975
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/repair-frontmatter-and-editbadge

Conversation

@bloxster

@bloxster bloxster commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

fix(i18n): strip a stray leading --- from 260 translated pages

pages build and deployment has been failing since 2026-08-14, and the published
GitHub Pages site (https://zechub.github.io/zechub/, status errored) has been
frozen on that day's build ever since.

The LLM sync prepends a --- that the English source does not have. Jekyll reads
a leading --- as a YAML front-matter opener, runs to the next --- far down the
page, and fails to parse. The build dies on
translations/ja/site/Zcash_Tech/What_a_Block_Explorer_Can_See.md.

Emulating Jekyll's front-matter parse across the whole tree: 251 translated pages
break, 0 English pages do. Every breaking file is a translation, and in all 251
the English source has no leading ---, so there is no judgment call — the
dashes are simply not supposed to be there.

Traced by counting leading---- files at each merge:

commit files pages build
d951213 (#1948) 96 last success
c301ff2 (#1960) 127 first failure
636861a (#1964) 356 failing

96 such files predate this and the build was green, so a leading --- is not
automatically fatal — it breaks when the resulting pseudo-front-matter is invalid
YAML, which prose in these locales reliably produces. #1960 tipped it over and
#1964 made it much worse.

This strips the stray line from all 260 pages where the English source lacks it,
not only the 251 that currently break, since the other 9 are the same defect and
a latent break.

Reader-facing impact was limited: the frontend calls next-mdx-remote's
serialize(..., {}), whose parseFrontmatter defaults to false, and gray-matter
— though in package.json — is imported nowhere in src/, so the dashes rendered as
a stray horizontal rule rather than swallowing content. Worth noting gray-matter
DOES throw on these files, so anything that starts using it would break hard.

Two sidebar labels were wrong, though, and this fixes them: Using_Zcash/Zecmap.md
read "Kurzfassung" in de and "Resumo" in pt — both meaning "summary" — because the
stray --- made the title extractor skip # ZecMap and pick up ## TL;DR. Both
now read "ZecMap". Menu-title manifests regenerated accordingly.

No prose changed: the only content line touched anywhere is the bare ---.
src is untouched because it hashes the ENGLISH source, which has not moved.
edited stays false throughout — this is a mechanical pass, and flipping it
would remove these pages from automated sync permanently.

Verification, all on this branch:

check result
Jekyll front-matter emulation 0 breaking pages, was 251
protected-terms (scoped) passed, 3,654 pages
manifest-invariants holds, 203 x 18
translation lib tests 51 pass / 0 fail
menu-titles-fresh up to date, 19 files
non---- content lines changed 0

Incidentally a first real exercise of #1973: with the identical content repair but
the manifest left untagged, manifest-invariants now fails with 260 violations.
Before #1973 it passed green, so this repair could have landed with no provenance
trace at all.

Not done here, having checked and found nothing wrong: the 108 translated pages
using the markdown badge form [![Edit](...)](...) rather than the HTML
<a><img/></a> form. All 108 match their English source exactly.


Second commit: two residual front-matter defects (42 more pages)

A sweep of all 4800 .md files — rather than only the pages the last resync
touched — turned up two more defects of the same family. Neither breaks the
Jekyll build on its own; both are invisible until someone reads the page.

24 pages carry a duplicated front-matter opener. On disk:

---
---
published: 2025-08-19
---

Jekyll consumes the first, empty block as the front matter, so the date leaks
into the body — in 22 of them the following --- turns it into a setext H2
heading, and in the 2 with a blank line between, a paragraph followed by a
rule. The extra delimiter is dropped; the value is untouched.
Locales: ru 4, ja 3, ko 2, hi 4, fr 1, uk 5, tr 5.

18 pages had the front-matter key itself machine-translated — ee wotae:,
sw iliyochapishwa:, ak wotintimii:, six Research pages each — so no
consumer could read the publication date at all. The published: key is
restored, value untouched. (published is a reserved Jekyll key, but every
value is a date and therefore truthy, so no page's visibility changes.)

Provenance for all 42 is recorded in tool
(+collapse-empty-frontmatter / +restore-frontmatter-key), the same
convention this branch already uses for +strip-stray-frontmatter. src is
deliberately not bumped — these pages are no fresher against their English
source than before — and edited stays false, since flipping it for a
mechanical pass would hold the page out of automated re-translation forever.

Gates run locally against upstream/main as the base, so change-tracking is
live rather than skipped:

gate result
check-invariants --base upstream/main invariants hold: 203 curated pages, 18 locales (302 changed files, all with provenance)
gen-menu-titles --check up to date (19 files)
node --test translation/lib/*.test.mjs 51 pass / 0 fail
check-protected-terms --base passed for 3654 pages (pre-existing drift reported, none in the 42)

After this, no .md in the tree opens with an empty front-matter block, an
unterminated block, or a front-matter key other than published.

Follow-up, not in this PR: the generator's own guard cannot catch either
defect. It compares only the first line of the translation against the first
line of the source, so a doubled opener on a page whose source legitimately
has front matter passes, and a translated key is never checked. Without
tightening that gate to compare the parsed block (delimiter count + key set),
a future resync can reintroduce both.

Third commit: a third shape, found by the tightened generator guard (14 pages)

The guard in the generator was rewritten to compare the parsed front-matter
BLOCK against the source's instead of only the first line. Pointed at the whole
corpus it found 14 pages that the scans above could not see, because those only
examined files that HAVE front matter:

12 Research pages (ig 6, yo 6) lost the opening delimiter while the key was
translated and the closing --- survived — bipụtara: 2025-08-02,
tí a tẹ̀ jáde: 2026-04-14. The page therefore has no front matter at all and
the line renders as a setext H2 heading. The source block is restored and the
mangled line dropped.

2 ja pages kept a blank line inside the block after the second commit
collapsed their duplicated opener. Valid YAML with the right value either way;
normalised so every translated block is byte-identical to its source.

Replaying the new guard over all 54 pre-fix files from this branch's history
reproduces the fixes in these three commits byte-for-byte, and the repaired
corpus is a fixed point: 3654 pages, 0 repairs, 0 failures.

One deliberate non-change: six of the restored values keep a trailing space
(published: 2023-10-23 ) because the English source has it, and one source is
CRLF. Stripping it in the translations alone would make them the only pages that
deviate from their source, which the new guard would then repair straight back.
Whitespace in the English sources is a separate cleanup.

`pages build and deployment` has been failing since 2026-08-14, and the published
GitHub Pages site (https://zechub.github.io/zechub/, status `errored`) has been
frozen on that day's build ever since.

The LLM sync prepends a `---` that the English source does not have. Jekyll reads
a leading `---` as a YAML front-matter opener, runs to the next `---` far down the
page, and fails to parse. The build dies on
translations/ja/site/Zcash_Tech/What_a_Block_Explorer_Can_See.md.

Emulating Jekyll's front-matter parse across the whole tree: 251 translated pages
break, 0 English pages do. Every breaking file is a translation, and in all 251
the English source has no leading `---`, so there is no judgment call — the
dashes are simply not supposed to be there.

Traced by counting leading-`---` files at each merge:

| commit | files | pages build |
|---|---|---|
| d951213 (ZecHub#1948) | 96 | last success |
| c301ff2 (ZecHub#1960) | 127 | first failure |
| 636861a (ZecHub#1964) | 356 | failing |

96 such files predate this and the build was green, so a leading `---` is not
automatically fatal — it breaks when the resulting pseudo-front-matter is invalid
YAML, which prose in these locales reliably produces. ZecHub#1960 tipped it over and
ZecHub#1964 made it much worse.

This strips the stray line from all 260 pages where the English source lacks it,
not only the 251 that currently break, since the other 9 are the same defect and
a latent break.

Reader-facing impact was limited: the frontend calls next-mdx-remote's
`serialize(..., {})`, whose `parseFrontmatter` defaults to false, and gray-matter
— though in package.json — is imported nowhere in src/, so the dashes rendered as
a stray horizontal rule rather than swallowing content. Worth noting gray-matter
DOES throw on these files, so anything that starts using it would break hard.

Two sidebar labels were wrong, though, and this fixes them: `Using_Zcash/Zecmap.md`
read "Kurzfassung" in de and "Resumo" in pt — both meaning "summary" — because the
stray `---` made the title extractor skip `# ZecMap` and pick up `## TL;DR`. Both
now read "ZecMap". Menu-title manifests regenerated accordingly.

No prose changed: the only content line touched anywhere is the bare `---`.
`src` is untouched because it hashes the ENGLISH source, which has not moved.
`edited` stays false throughout — this is a mechanical pass, and flipping it
would remove these pages from automated sync permanently.

Verification, all on this branch:

| check | result |
|---|---|
| Jekyll front-matter emulation | 0 breaking pages, was 251 |
| protected-terms (scoped) | passed, 3,654 pages |
| manifest-invariants | holds, 203 x 18 |
| translation lib tests | 51 pass / 0 fail |
| menu-titles-fresh | up to date, 19 files |
| non-`---` content lines changed | 0 |

Incidentally a first real exercise of ZecHub#1973: with the identical content repair but
the manifest left untagged, `manifest-invariants` now fails with 260 violations.
Before ZecHub#1973 it passed green, so this repair could have landed with no provenance
trace at all.

Not done here, having checked and found nothing wrong: the 108 translated pages
using the markdown badge form `[![Edit](...)](...)` rather than the HTML
`<a><img/></a>` form. All 108 match their English source exactly.
@bloxster
bloxster marked this pull request as draft August 17, 2026 12:46
@bloxster
bloxster marked this pull request as ready for review August 17, 2026 14:37
…earch pages

Companion to the stray-`---` strip in this branch, from a sweep of all 4800
`.md` files rather than the pages the resync last touched.

24 pages carried a DUPLICATED front-matter opener (`---` / `---` / `published:
<date>` / `---`). Jekyll consumes the first, empty block as the front matter,
so the date leaks into the body: in 22 of them the following `---` makes it a
setext H2 heading, in the 2 with a blank line between, a paragraph and a rule.
Drops the extra delimiter; the value is untouched. ru, ja, ko, hi, fr, uk, tr.

18 pages had the front-matter KEY itself machine-translated — ee `wotae:`,
sw `iliyochapishwa:`, ak `wotintimii:` — so no consumer could read the
publication date at all. Restores `published:`, value untouched.

Provenance for all 42 pages is recorded in `tool`
(`+collapse-empty-frontmatter` / `+restore-frontmatter-key`), matching the
convention used for the strip. `src` is deliberately NOT bumped: these pages
are no fresher against their English source than before, and `edited` stays
false so they remain in automated sync.

Neither defect blocks the Jekyll build on its own — the stray `---` stripped
in the previous commit is what breaks it — but both are the same class of
damage from the same resync, and they are invisible in the rendered page only
until someone reads it.
@bloxster bloxster changed the title fix(i18n): strip a stray leading --- from 260 translated pages fix(i18n): repair broken front matter in 302 translated pages Aug 18, 2026
Found by the tightened generator guard, which compares the translation's
front-matter BLOCK against the source's rather than only the first line.

On 12 Research pages (ig 6, yo 6) the OPENING `---` was lost and the key
translated — `bipụtara: 2025-08-02`, `tí a tẹ̀ jáde: 2026-04-14` — while the
closing delimiter survived. The page therefore has no front matter at all and
the line renders as a setext H2 heading: the same visible damage as the
duplicated opener fixed in the previous commit, from a different cause. The
source block is restored verbatim and the mangled line dropped.

The 2 ja pages kept a blank line inside the block after the previous commit
collapsed their duplicated opener. Valid YAML and the right value either way;
normalised so every translated block is byte-identical to its source.

`tool` records `+restore-frontmatter` / `+normalize-frontmatter`; `src` and
`edited` are untouched, as before.

No .md under translations/ now deviates from its source's front matter — all
3654 translated pages pass the new gate.
bloxster pushed a commit to bloxster/zechub that referenced this pull request Aug 18, 2026
Review found the scoping too coarse to catch the very thing it was written for.
A page already broken at the base could have its block rewritten into a
DIFFERENT broken block of the same kind — `wotintimii: 2024-01-12` to
`wotintimii: 9999-99-99` — and be waved through as inherited, because only the
violation's category was compared. A resync re-damaging pages is precisely the
2026-08-14 shape, and all 314 pages are in that state until ZecHub#1975 lands, so the
window was open exactly where it mattered. The front-matter region itself must
now be unchanged on both sides before damage counts as somebody else's.

Four narrower corrections, each a page Jekyll accepts that the check rejected,
or the reverse:

- A non-ASCII key is valid YAML. `bipụtara: 2025-08-19` was reported as an
  unparseable block, with a message claiming the build would fail — it does not,
  and the taxonomy contradicted the doc. It reads as a key mismatch now.
- An empty block is valid front matter. Only call it a duplicated opener when
  the source actually declares something the page then repeats as body.
- `-` and `?` open a block sequence and a complex key; Psych rejects both, so a
  source and translation sharing `published: - x` really would fail the build.
- An opener the source never closes is not front matter to Jekyll: it copies
  both files through untouched. A translation mirroring that is not a stray
  opener, and saying so was wrong twice over.

Scope narrows to the translations a change touched. Adding front matter to an
English page leaves its translations out of step, but that is the sync
pipeline's work and the staleness dashboard's report — not 17 red X's for
whoever edited the English page. The contributor doc no longer claims this
scopes identically to the terminology gate, because it does not.

86 tests pass.
@zksquirrel
zksquirrel merged commit 960120c into ZecHub:main Aug 19, 2026
4 checks passed
zksquirrel pushed a commit that referenced this pull request Aug 19, 2026
…urce

On 2026-08-14 a resync gave 260 translated pages a leading `---` their source
does not have. Jekyll reads that as a front-matter opener, scans to the next
`---` far down the page, and fails the build for the whole site; the published
GitHub Pages copy stayed frozen for four days. Nothing in CI could have stopped
it: the Pages build runs only on push to `main`, never on a pull request.

Three quieter shapes came in with it and are worse in one respect — no build
fails, so nothing reports them at all: an empty block ahead of the real one, the
key itself translated (`wotae:`, `iliyochapishwa:`, `wotintimii:`, `bipụtara:`),
and a lost opening delimiter with the closing one still in place. Two of those
render the publication date as a heading in the page; the third loses it. 314
pages were damaged in total.

The rule this adds is the one the corpus already follows: a translated page
declares the same leading YAML block as its English source, copied verbatim.
Nothing about a publication date changes between languages.

- translation/lib/frontmatter.mjs — parse and compare, BOM-aware and blind to
  line endings, delimiters anchored at column 1 the way Jekyll anchors them.
  Also rejects a block that is not flat `key: value`, because two files can
  share a block that Jekyll cannot parse and parity alone would pass it.
- scripts/check-frontmatter.mjs — with `--base`, blocking only on a violation
  this change introduced; anything already broken prints as a notice, so an
  unrelated pull request stays green. Bare, every page is blocking.
- frontmatter-parity job — scoped on a pull request, a full sweep on push to
  `main`, so damage that arrives by force-push or by a merge predating this job
  cannot sit unreported on the branch Pages builds from.
- The contributor doc gains the rule and the reason.

Tests live in translation/lib/frontmatter.test.mjs, picked up by the existing
hash-lib-tests job: 77 pass across the four lib modules.

Note on merge order: run bare against `main` today and it reports the 314 pages
still to be repaired in #1975. Merging that first leaves this job green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants