Visual regression coverage gaps: threshold tolerance and missing RHS TOC snapshots

[mmcky]

Context

Surfaced while reviewing #388 (which addresses #387). The PR changed the LHS sidebar background colour and added a fade mask + hover-scroll to the RHS "On this page" TOC, yet all 45 Playwright visual regression tests passed without any baseline updates. That's not because nothing changed — the suite genuinely cannot see the changes it should.

Gap 1 — Threshold tolerance is too loose for colour shifts

playwright.config.ts currently sets:

```ts
toHaveScreenshot: {
maxDiffPixelRatio: 0.01, // 1% of pixels can differ
threshold: 0.2, // per-pixel: diff <20% RGB doesn't register
}
```

A pixel only counts as "different" if its RGB distance exceeds ~20%. That means:

• #efefef → #ffffff ≈ 6% per pixel — invisible to the comparator
• #efefef → rgba(239,239,239,0.4) over white ≈ ~2% per pixel — also invisible

So the entire sidebar surface changed colour and zero pixels registered as different.

The threshold: 0.2 setting is presumably tuned for cross-platform font/anti-aliasing tolerance (CI uses Ubuntu, local dev often macOS). Tightening it globally would produce noise on font-rendering deltas, but the current value lets through any colour shift below ~20% RGB distance — which is most realistic palette tweaks.

Possible directions

• Lower threshold selectively (e.g. 0.05 by default, override to 0.2 only for tests that include font-heavy regions). Would catch most palette shifts but stays tolerant of antialiasing where it matters.
• Add explicit colour assertions (e.g. await expect(sidebar).toHaveCSS('background-color', 'rgba(239, 239, 239, 0.4)')) for known-stable theme regions, complementing rather than replacing screenshot diffs.
• Both — pixel-exact assertions where the value is known; screenshot diffs for everything else with a tighter threshold.

Gap 2 — RHS "On this page" TOC has no snapshot coverage

tests/visual/theme.spec.ts snapshots .qe-page__header, .qe-sidebar, full pages, code blocks, dark mode, math, toolbar, bold/italic typography — but nothing for .qe-page__toc. The main subject of PR #388 (fade mask, hover-to-scroll behaviour) is completely uncovered.

Additional sub-gap

None of the fixture pages exercise overflow scenarios:

```ts
const testPages = [
{ name: "homepage", path: "/index.html", hasPlots: false },
{ name: "intro", path: "/intro.html", hasPlots: false },
{ name: "getting-started", ... },
{ name: "python-by-example", ... },
{ name: "numpy", ... },
{ name: "matplotlib", ... },
];
```

All are short enough that the RHS TOC fits the viewport. The actual #387 bug (TOC clipping on long pages with many sections, e.g. prob_matrix.html) is not exercised by any test.

Possible directions

• Add a .qe-page__toc snapshot per existing page (mirrors -sidebar.png / -header.png convention).
• Add a long-page fixture to the test set — either pin a deep lecture (prob_matrix.html or similar) or build a synthetic page with many headings.
• Add a hover-state snapshot to verify the mask removal + scrollbar appearance on hover.

Out of scope

This issue tracks the coverage gaps. The actual #387 fix lands in #388 as-is — no snapshot regeneration needed there since the comparator wouldn't flag a diff either way.

Acceptance

• Decision on threshold strategy (selective, explicit colour assertions, or both)
• RHS TOC snapshots added across existing fixture pages
• At least one long-page fixture covering RHS TOC overflow behaviour