From e598e5647b0364d45d9be18fcf37c433be466984 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Sat, 15 Aug 2026 11:50:19 +0200 Subject: [PATCH 1/2] i18n(terms): protect two key forms, four case variants, and a singular MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three gaps found while auditing this list against the categorized list in the frontend repo. 1. Key and address compounds are only half protected. `Unified Address` and `Viewing Key` are enforced, the other expanded forms are not, so a translation can localize them and nothing complains: Full Viewing Key 5 English pages, 0 translations missing it Incoming Viewing Key 2 English pages, 0 translations missing it 2. The check is case-sensitive, so a term protected in one casing is unprotected in the other. The list carries `Free2Z`, `Zcash.Me`, `MetaMask Snap` and `Zingo`, while the variants that actually appear in the wiki went unenforced: Zcash.me 1 English page, 0 translations missing it Zingo! 20 English pages, 0 translations missing it ZGo 27 English pages, 0 translations missing it Zgo 9 English pages, 0 translations missing it Same trap as `Zodl` vs `ZODL`, which is still open. 3. `ZK-SNARKs` is enforced but its singular is not: ZK-SNARK 1 English page, 0 translations missing it All seven are free: every existing translation already keeps them verbatim, so this locks in current behaviour and prevents a future regression. Verified — the whole-tree audit reports 72 violations both before and after, and none names a newly added term. (Those 72 are pre-existing drift from #1948 and #1957, cleared by the pending backfill/re-sync PR.) WHAT IS NOT HERE, AND WHY IT SHOULD NOT BE ADDED LATER EITHER An earlier draft of this change listed `Unified Addresses` as merely expensive — 94 violations, 113 after the pending re-sync. Measuring what those violations actually are changed the conclusion: protecting an English PLURAL is wrong in principle, not just costly. Of the 113, 70% already carry the protected singular `Unified Address`. They did not lose the concept; they declined to pluralize an English loanword. English "for inspecting Unified Addresses" becomes Italian "per ispezionare gli Unified Address", which is correct Italian — Italian does not add -s to English loanwords. Japanese writes `Unified Addressを調べる` because Japanese does not inflect plurals at all, and 23 of the 113 sit in ja/ko/zh, none of which do. Enforcing the plural would push a grammatical error into Italian, French, Spanish and others, and a meaningless string into three more, to satisfy a check the singular already satisfies conceptually. The principle: protect singulars, not plurals. Languages differ in whether and how they pluralize borrowed nouns, so a protected plural imposes English grammar on all 18 locales. This is not hypothetical. `ZK-SNARKs` is enforced today, and on all four English pages that use it, ja/ko/zh/it carry the English plural verbatim — they had no choice. A follow-up will teach the checker that a plural is satisfied by its protected singular and retire the plural entries; this change adds the singular so that follow-up has something to fall back to. Also still held back, each priced against both main and the pending re-sync tree, because adding them creates violations no queued sync repairs (the terms list is not part of staleness detection, so the debt is invisible to the dashboard and unreachable by the sync agent): TEX Address 39, Dev Fund 25/29, Diversified Address 14, Spending Key 7, Metamask Snap 2, zk-SNARK 2, Free2z 1. Skipped entirely: `Unified Viewing Key` and `Outgoing Viewing Key` are in the categorized list but appear on no English page. --- translation/protected-terms.json | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/translation/protected-terms.json b/translation/protected-terms.json index 136de1c13..5c926a439 100644 --- a/translation/protected-terms.json +++ b/translation/protected-terms.json @@ -81,7 +81,14 @@ "Zcash Engineering Office Hours", "NU6.1", "NU6.2", - "NU7" + "NU7", + "Full Viewing Key", + "Incoming Viewing Key", + "Zcash.me", + "Zingo!", + "ZGo", + "Zgo", + "ZK-SNARK" ], "glossaryOnly": [ "token", From 90b460ce72d1f5cce69625a7c33667cb5e548f61 Mon Sep 17 00:00:00 2001 From: Bloxster Date: Sat, 15 Aug 2026 15:05:57 +0200 Subject: [PATCH 2/2] ci(i18n): let a protected term be satisfied by a declared equivalent form MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some protected terms are the same word in two shapes — a singular and its plural. Whether an English loanword takes an -s is a decision each language makes for itself, not one English makes for it: Italian writes "gli Unified Address" because Italian does not pluralize borrowed nouns, and Japanese, Korean and Chinese do not inflect plurals at all. Demanding one exact shape forces English grammar into 18 languages that do not share it, to assert a term the other shape already carries. Not hypothetical: `ZK-SNARKs` is protected and its singular was not, so on all four English pages using it, ja/ko/zh/it carry the English plural verbatim. They had no choice — the gate required it. A translation may now satisfy a term with any form DECLARED equivalent to it, via a new optional `equivalentForms` key: "equivalentForms": [ ["ZK-SNARK", "ZK-SNARKs"] ] The pairing is declared, never inferred from spelling. An earlier draft derived it by stripping a trailing "s"/"es", which two reviewers independently rejected: that makes `Argos` interchangeable with `Argo` and `zec.rocks` with `zec.rock` — different things that merely look related — breaks on acronyms (`CSS` -> `CS`), and silently misses irregular plurals like `Technologies` / `Technology`. It also got its own motivating case wrong, deriving "Unified Addresse" from "Unified Addresses". A maintainer states which forms are one word; the checker never guesses. Membership is symmetric because a group asserts "these are one term", so any member satisfies it. That is now a declared property rather than an accident of the derivation. The source side still tests the exact term — the English page really does use that form. Only the translation side accepts an equivalent. Every malformed declaration is fatal: a member absent from preserveVerbatim, a term in two groups, a duplicate, a group of one, a non-array. A group that was quietly ignored would weaken this gate in the way least likely to be noticed. `satisfiedIn` is applied to classify()'s baseline test as well, and that is load-bearing in the opposite direction from the obvious one: it TIGHTENS the gate. If a base translation carries only the singular and a change strips it, the previous exact-match baseline made violatedAtBase true, bucketing the result "pre-existing" and exiting 0 — silently passing a change that removed the last protected form. It is now correctly "introduced", exit 1. Logic lives in translation/lib/term-forms.mjs (pure, no fs/process/clock) so the existing hash-lib-tests job picks up its 11 unit tests with no workflow change: the declared pair in both directions, the -es case the derivation got wrong, look-alikes that must NOT pair, and each validation failure. termRegExp now memoizes; it was recompiling per (page, locale, term). Note this changes NOTHING on the current corpus — 72 violations before and after — because the harm already landed: the locales were forced to carry the plural, so no violations remain to clear. The value is preventive, and it is why the companion terms PR adds `ZK-SNARK` rather than removing `ZK-SNARKs`. --- scripts/check-protected-terms.mjs | 57 +++++++++++++-- translation/lib/term-forms.mjs | 104 +++++++++++++++++++++++++++ translation/lib/term-forms.test.mjs | 105 ++++++++++++++++++++++++++++ translation/protected-terms.json | 3 + 4 files changed, 265 insertions(+), 4 deletions(-) create mode 100644 translation/lib/term-forms.mjs create mode 100644 translation/lib/term-forms.test.mjs diff --git a/scripts/check-protected-terms.mjs b/scripts/check-protected-terms.mjs index 0d840d1f3..0308f1aaf 100644 --- a/scripts/check-protected-terms.mjs +++ b/scripts/check-protected-terms.mjs @@ -40,6 +40,8 @@ import { execFileSync } from "node:child_process"; import { existsSync, readdirSync, readFileSync, statSync } from "node:fs"; import { join, relative } from "node:path"; +import { buildEquivalence, formsFor } from "../translation/lib/term-forms.mjs"; + const root = new URL("../", import.meta.url).pathname; const config = JSON.parse( readFileSync(join(root, "translation/protected-terms.json"), "utf8"), @@ -87,16 +89,41 @@ function sourceForTranslation(absTranslated) { return sourceRel; } +const reCache = new Map(); function termRegExp(term) { + const hit = reCache.get(term); + if (hit) return hit; // Word-boundary match so short terms don't false-positive inside other // words (e.g. "mining" must not match "deter*mining*"; "chain" is still // satisfied by "block*chain*" only via the standalone token's boundaries). const escaped = term.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); - return new RegExp(`(? termRegExp(f).test(translated)); +} + // ---- pass 1: every violation in the working tree --------------------------- const violations = []; @@ -125,8 +152,7 @@ for (const translatedAbs of walk(translationsDir)) { checked += 1; for (const term of terms) { - const re = termRegExp(term); - if (re.test(source) && !re.test(translated)) { + if (termRegExp(term).test(source) && !satisfiedIn(translated, term)) { violations.push({ translatedRel, sourceRel, term }); } } @@ -234,6 +260,29 @@ function baseContentAt(basePath) { return content; } +// Base content must be judged by the BASE declaration, not this change's. +// The config comes from the working tree while base translations come from +// git, so a change that removes an equivalence group would otherwise make the +// base look as though it had already been violating — bucketing a genuine +// regression as "pre-existing" and exiting 0. A base config that cannot be +// read, parsed or validated is not this change's fault, so fall back to the +// head declaration rather than failing the PR over history. +let baseEquivalence = equivalence; +if (mergeBase) { + const rawBaseConfig = baseContentAt("translation/protected-terms.json"); + if (rawBaseConfig !== null) { + try { + const baseConfig = JSON.parse(rawBaseConfig); + baseEquivalence = buildEquivalence( + baseConfig.preserveVerbatim ?? [], + baseConfig.equivalentForms, + ); + } catch { + baseEquivalence = equivalence; + } + } +} + function classify(v) { if (!mergeBase) return "introduced"; // no base requested → whole-tree audit // Judge pre-existence against the pairing that ACTUALLY existed at the base: @@ -253,7 +302,7 @@ function classify(v) { baseTranslated !== null && baseSource !== null && termRegExp(v.term).test(baseSource) && - !termRegExp(v.term).test(baseTranslated); + !satisfiedIn(baseTranslated, v.term, baseEquivalence); if (violatedAtBase) return "pre-existing"; return changedTranslations.has(v.translatedRel) ? "introduced" : "stale-source"; } diff --git a/translation/lib/term-forms.mjs b/translation/lib/term-forms.mjs new file mode 100644 index 000000000..cefce599a --- /dev/null +++ b/translation/lib/term-forms.mjs @@ -0,0 +1,104 @@ +// Equivalent forms of a protected term. +// +// Some protected terms are the same word in two shapes — a singular and its +// plural. Whether an English loanword takes an -s is a decision each language +// makes for itself, not one English makes for it: Italian writes "gli Unified +// Address" because Italian does not pluralize borrowed nouns, and Japanese, +// Korean and Chinese do not inflect plurals at all. Demanding one exact shape +// therefore forces English grammar into 18 languages that do not share it, to +// assert a term the other shape already carries. +// +// So a translation may satisfy a protected term with any form declared +// equivalent to it. The pairing is DECLARED, never inferred from spelling. +// Deriving it by stripping a trailing "s" would make `Argos` interchangeable +// with `Argo` and `zec.rocks` with `zec.rock` — different things that merely +// look related — and would silently miss irregular plurals such as +// `Technologies` / `Technology`. A maintainer states which forms are the same +// word; the checker never guesses. +// +// Shape in translation/protected-terms.json (the key is optional): +// +// "equivalentForms": [ +// ["ZK-SNARK", "ZK-SNARKs"] +// ] +// +// Membership is symmetric by construction: the group says "these are one +// term", so any member asserts it. This module is pure — no filesystem, no +// process, no clock — so it can be unit tested directly. + +export class TermFormsError extends Error {} + +/** + * Build a lookup of term -> Set of forms that satisfy it. + * + * Every group is validated against the protected list, and every failure is + * loud. A malformed group that was quietly ignored would weaken the gate in + * exactly the way nobody would notice. + * + * @param {string[]} terms the preserveVerbatim list + * @param {string[][]} groups the equivalentForms list (may be undefined) + * @returns {Map>} + */ +export function buildEquivalence(terms, groups) { + const protectedSet = new Set(terms); + const map = new Map(); + if (groups === undefined) return map; + + if (!Array.isArray(groups)) { + throw new TermFormsError("equivalentForms must be an array of groups"); + } + + const seen = new Map(); // term -> index of the group that already claimed it + groups.forEach((group, i) => { + if (!Array.isArray(group) || group.length < 2) { + throw new TermFormsError( + `equivalentForms[${i}]: a group needs at least two terms`, + ); + } + if (new Set(group).size !== group.length) { + throw new TermFormsError(`equivalentForms[${i}]: contains a duplicate`); + } + for (const term of group) { + // Reject anything that is not a clean, non-blank string. A blank or + // padded entry is the one shape that could do real damage: paired with a + // junk entry in preserveVerbatim, a term of " " matches nearly every + // page and would silently satisfy whatever it was grouped with. + if (typeof term !== "string" || !term.trim() || term.trim() !== term) { + throw new TermFormsError( + `equivalentForms[${i}]: entries must be non-empty strings without surrounding whitespace`, + ); + } + if (!protectedSet.has(term)) { + // Otherwise a typo would create a group that silently relaxes nothing, + // or worse, relaxes a term nobody protects. + throw new TermFormsError( + `equivalentForms[${i}]: "${term}" is not in preserveVerbatim`, + ); + } + if (seen.has(term)) { + throw new TermFormsError( + `equivalentForms[${i}]: "${term}" is already in group ${seen.get(term)}`, + ); + } + seen.set(term, i); + } + const forms = new Set(group); + for (const term of group) map.set(term, forms); + }); + + return map; +} + +/** + * The forms that satisfy `term`, always including the term itself first so the + * common case costs a single test. + * + * @param {string} term + * @param {Map>} equivalence + * @returns {string[]} + */ +export function formsFor(term, equivalence) { + const group = equivalence.get(term); + if (!group) return [term]; + return [term, ...[...group].filter((f) => f !== term)]; +} diff --git a/translation/lib/term-forms.test.mjs b/translation/lib/term-forms.test.mjs new file mode 100644 index 000000000..b2b9f7685 --- /dev/null +++ b/translation/lib/term-forms.test.mjs @@ -0,0 +1,105 @@ +import { strict as assert } from "node:assert"; +import { test } from "node:test"; + +import { buildEquivalence, formsFor, TermFormsError } from "./term-forms.mjs"; + +const TERMS = [ + "ZK-SNARK", "ZK-SNARKs", + "Unified Address", "Unified Addresses", + "Argos", "Zcash", "CSS", +]; + +test("no equivalentForms key leaves every term alone", () => { + const eq = buildEquivalence(TERMS, undefined); + assert.equal(eq.size, 0); + assert.deepEqual(formsFor("ZK-SNARKs", eq), ["ZK-SNARKs"]); +}); + +test("a declared pair satisfies in both directions", () => { + const eq = buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARKs"]]); + assert.deepEqual(formsFor("ZK-SNARKs", eq).sort(), ["ZK-SNARK", "ZK-SNARKs"]); + assert.deepEqual(formsFor("ZK-SNARK", eq).sort(), ["ZK-SNARK", "ZK-SNARKs"]); +}); + +test("the term itself is always tried first", () => { + const eq = buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARKs"]]); + assert.equal(formsFor("ZK-SNARKs", eq)[0], "ZK-SNARKs"); +}); + +test("an -es plural pairs correctly when declared", () => { + // The case a naive strip-one-character derivation gets wrong: + // "Unified Addresses" -> "Unified Addresse", which is nothing. + const eq = buildEquivalence(TERMS, [["Unified Address", "Unified Addresses"]]); + assert.ok(formsFor("Unified Addresses", eq).includes("Unified Address")); +}); + +test("terms that merely look related are NOT equivalent", () => { + // Argos/Argo and CSS/CS are the reason pairing is declared, not derived. + const eq = buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARKs"]]); + assert.deepEqual(formsFor("Argos", eq), ["Argos"]); + assert.deepEqual(formsFor("CSS", eq), ["CSS"]); + assert.deepEqual(formsFor("Zcash", eq), ["Zcash"]); +}); + +test("a group naming an unprotected term is rejected", () => { + assert.throws( + () => buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARKz"]]), + (e) => e instanceof TermFormsError && /not in preserveVerbatim/.test(e.message), + ); +}); + +test("a term cannot belong to two groups", () => { + assert.throws( + () => buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARKs"], ["ZK-SNARK", "Zcash"]]), + (e) => e instanceof TermFormsError && /already in group 0/.test(e.message), + ); +}); + +test("a group needs at least two members", () => { + assert.throws( + () => buildEquivalence(TERMS, [["ZK-SNARK"]]), + (e) => e instanceof TermFormsError && /at least two/.test(e.message), + ); +}); + +test("a duplicate inside a group is rejected", () => { + assert.throws( + () => buildEquivalence(TERMS, [["ZK-SNARK", "ZK-SNARK"]]), + (e) => e instanceof TermFormsError && /duplicate/.test(e.message), + ); +}); + +test("a blank or padded entry is rejected", () => { + // The dangerous shape: a term of " " matches nearly every page, so pairing + // it with a real term would silently satisfy that term everywhere. + for (const junk of ["", " ", "\t", " ZK-SNARK"]) { + assert.throws( + () => buildEquivalence([...TERMS, junk], [["ZK-SNARK", junk]]), + (e) => e instanceof TermFormsError && /non-empty strings/.test(e.message), + `expected "${junk}" to be rejected`, + ); + } +}); + +test("a non-string entry is rejected", () => { + for (const junk of [null, 42, {}, []]) { + assert.throws( + () => buildEquivalence(TERMS, [["ZK-SNARK", junk]]), + (e) => e instanceof TermFormsError, + ); + } +}); + +test("equivalentForms must be an array", () => { + assert.throws( + () => buildEquivalence(TERMS, { "ZK-SNARKs": ["ZK-SNARK"] }), + (e) => e instanceof TermFormsError && /must be an array/.test(e.message), + ); +}); + +test("groups of three or more are supported", () => { + const terms = [...TERMS, "zk-SNARK"]; + const eq = buildEquivalence(terms, [["ZK-SNARK", "ZK-SNARKs", "zk-SNARK"]]); + assert.equal(formsFor("zk-SNARK", eq).length, 3); + assert.ok(formsFor("ZK-SNARK", eq).includes("zk-SNARK")); +}); diff --git a/translation/protected-terms.json b/translation/protected-terms.json index 5c926a439..c12684e89 100644 --- a/translation/protected-terms.json +++ b/translation/protected-terms.json @@ -90,6 +90,9 @@ "Zgo", "ZK-SNARK" ], + "equivalentForms": [ + ["ZK-SNARK", "ZK-SNARKs"] + ], "glossaryOnly": [ "token", "chain",