Add a server-parameterized scenario framework (Phase 0)

[sibson]

What changed

This is PR 3a of the Phase 0 stack, stacked on PR 2 (#340). It extracts the abstraction the plan's Phase 0 bullet asks for ("extract the common scenarios... into a server-parameterized base class") — framework only; the scenarios beyond connect/capture land in a follow-up PR 3b.

The problem with plain test methods

tests/functional/vncservers.py had one shared test body, test_connect_key_and_capture, generated per server by register_server_tests(). Adding test_mouse, test_expect, … as sibling methods is the obvious next step, but the plan needs the same scenario list to serve three consumers, only one of which is unittest:

1. unittest — the servers × scenarios grid CI reports.
2. the recorder — the plan's vncdo record wrapper over loggingproxy drives a scenario against a server and writes a fixture directory; it needs a scenario callable outside a TestCase.
3. the Tier 3 checklist — the "short scripted scenario checklist" a community contributor runs by hand against a server we can't host; it needs to be enumerable data, not logic buried in test bodies.

So tests/functional/scenarios.py makes a Scenario a NamedTuple (name, description, required capabilities) wrapping a plain run(client, ctx) -> None function, and SCENARIOS is the ordered registry both unittest and the future recorder consume.

Capability model

VNCServer (in vncservers.py) grows a capabilities property — the one obvious place a server declares what it can do — derived from its existing fields (auth:none/auth:vncpass/auth:ard from username/password, known_size from size, renders_desktop, and an as-yet-unused input_reactive). A Scenario.requires a subset of that; a server missing a required capability skips with a reason naming the capability, never a silent pass. That distinction matters because a skip has to render differently from a pass in the compatibility matrix — otherwise macOS Screen Sharing's black framebuffer reads as green coverage instead of "not applicable here."

register_server_tests() now generates the servers × scenarios cross product: one TestCase subclass per server, one test_<scenario> method per scenario, so test ids read like TestServer_tigervnc.test_capture — a per-server, per-scenario pass/fail/skip cell.

Why PIXEL exists but is unused

scenarios.py defines three assertion-level helpers for input scenarios: PROTOCOL (session survives, always available), CHANGE (framebuffer differs at all — honest that a blinking cursor would also pass this), and PIXEL (a specific region changed — the strongest level). No Tier 1 server declares input_reactive yet: tests/servers/draw-content.sh paints static content, so nothing in the fleet reacts to input in a known region. That's deferred to its own spike (image work with real flakiness risk — font rendering, timing) rather than folded into the PR that establishes the abstraction. PIXEL is implemented now so that spike only has to flip a capability flag, not invent an assertion. I added this as an explicit Tier 1 follow-up in docs/server-compatibility-plan.md.

Scope: only connect and capture

Ported only the existing round trip, split into its natural scenarios (connect: handshake + negotiated version/security type recorded to the artifact dir; capture: the existing PNG assertions — valid PNG, size when known_size, not-flat when renders_desktop). Deliberately not adding authenticate/keyboard/mouse/expect here — that's PR 3b, once this abstraction has been reviewed with two users instead of six.

Screenshot gallery

Left tests/functional/capture_screenshots.py as-is (flat screenshots/<server>.png, one gallery). The unittest-generated tests now write into the finer-grained screenshots/<server>/<scenario>/ the spec describes, but folding the standalone gallery script into that same per-scenario layout is a bigger change (it would need to walk SCENARIOS and call scenario bodies rather than just capture_screenshot()) and is left for 3b alongside the new scenarios it would need to display.

Verification

• flake8 --count --statistics vncdotool tests — clean (0).
• make test (python -m unittest discover tests/unit) — 67 tests pass, unaffected (all changes are under tests/functional/).
• python -m unittest discover -v -s tests/functional -t . -p 'test_servers.py' with no servers running: all 6 generated tests (tigervnc/tigervnc-auth/x11vnc × connect/capture) skip with a clear "not reachable... start the servers first" message, and the process exits normally (no hang).
• Test ids confirmed via TestLoader().discover(...):

  tests.functional.test_servers.TestServer_tigervnc.test_capture
  tests.functional.test_servers.TestServer_tigervnc.test_connect
  tests.functional.test_servers.TestServer_tigervnc_auth.test_capture
  tests.functional.test_servers.TestServer_tigervnc_auth.test_connect
  tests.functional.test_servers.TestServer_x11vnc.test_capture
  tests.functional.test_servers.TestServer_x11vnc.test_connect
  

• tests/functional/capture_screenshots.py docker still runs end-to-end (skips cleanly with no servers up, writes the gallery index.html).
• No Docker daemon available in this sandbox (same limitation noted in the plan doc's spike results), so the actual Tier 1 fleet wasn't exercised live here; the skip-path verification above is the sandbox-reachable proxy for "the cross-product generation and skip logic work."

Scope note

Per instructions, docs/server-compatibility-plan.md gets only the one new Tier 1 follow-up bullet (the deferred input-reactive spike) — no other restructuring.

---

Generated by Claude Code