Spike: Docker Compose VNC test servers for compatibility testing

.github/workflows/spike-servers.yml

@@ -0,0 +1,78 @@
+name: Spike - VNC test servers
+
+on:
+  push:
+    branches:
+      - claude/spike-server-fleet
+      - claude/docker-vnc-screenshot-access-5zpj5f
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+env:
+  PIP_DISABLE_PIP_VERSION_CHECK: '1'
+  PIP_NO_PYTHON_VERSION_WARNING: '1'
+  VNCDOTOOL_SCREENSHOT_DIR: screenshots
+
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  servers:
+    name: Test server functional tests
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - name: Check out source
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
+          cache: pip
+          cache-dependency-path: requirements-dev.txt
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-dev.txt
+
+      - name: Build and start the VNC test servers
+        run: docker compose -f tests/servers/docker-compose.yml up -d --build --wait
+
+      # The steps below are diagnostics and teardown, so they carry
+      # `if: always()`: when a test fails, the container status, the
+      # screenshot of what the server was actually showing, and the server
+      # logs are exactly what's needed to debug it -- skipping them on
+      # failure would throw away the evidence. `down` likewise has to run
+      # whatever happened.
+      - name: Show test server status
+        if: always()
+        run: docker compose -f tests/servers/docker-compose.yml ps
+
+      - name: Run test server functional tests
+        run: python -m unittest discover -v -s tests/functional -t . -p 'test_servers.py'
+
+      - name: Capture screenshots and build the gallery
+        if: always()
+        timeout-minutes: 3
+        run: python tests/functional/capture_screenshots.py
+
+      - name: Dump test server container logs
+        if: always()
+        run: docker compose -f tests/servers/docker-compose.yml logs
+
+      - name: Upload screenshots
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: screenshots
+          path: screenshots/
+          if-no-files-found: ignore
+
+      - name: Stop the VNC test servers
+        if: always()
+        run: docker compose -f tests/servers/docker-compose.yml down

.gitignore

@@ -11,3 +11,4 @@ docs/_build
 
 *~
 #*
+tests/servers/screenshots/

Makefile

@@ -7,6 +7,10 @@ REQUIREMENTS_TXT?=requirements-dev.txt
 help:
 	@echo "test:		run unit tests"
 	@echo "test-func:	run functional tests"
+	@echo "servers-up:	start the docker VNC test servers"
+	@echo "servers-down:	stop the docker VNC test servers"
+	@echo "test-servers:	run functional tests against the VNC test servers"
+	@echo "screenshots:	screenshot each running VNC test server into a gallery"
 	@echo "docs:		build documentation"
 	@echo "release:	tag and push current version to trigger PyPI release"
 
@@ -41,4 +45,6 @@ include libvncserver.mk
 
 test-func: libvnc-examples test-libvnc
 
+include tests/servers/servers.mk
+
 include Makefile.venv

docs/server-compatibility-plan.md

@@ -322,8 +322,8 @@ contributing a fingerprint must be a paved road:
 `bbfd188`) holds a working proof: three in-repo Dockerfile-based services
 (`tigervnc` no-auth, `tigervnc-auth` VNC-password, `x11vnc` over Xvfb)
 defined in `tests/servers/docker-compose.yml` with `nc`-based
-healthchecks, `make servers-up`/`servers-down`/`test-fleet` wrappers, and
-a parameterized `tests/functional/test_fleet.py` that connects, types,
+healthchecks, `make servers-up`/`servers-down`/`test-servers` wrappers, and
+a parameterized `tests/functional/test_servers.py` that connects, types,
 and captures against each server. GitHub Actions run
 [31729730724](https://github.com/sibson/vncdotool/actions/runs/31729730724)
 is green end-to-end: image builds + healthcheck-gated `up --wait` in
@@ -346,6 +346,29 @@ Findings worth keeping:
 - Images stay small and layer-cached (~15–20s builds); healthchecks make
   `up --wait` a reliable barrier.
 
+Tier 1 follow-ups:
+- **Publish the screenshot gallery to GitHub Pages.** Captures currently
+  reach the web only as a zipped artifact: the job summary carries a
+  server/port/resolution table, but seeing the pixels means download →
+  unzip → open `index.html`. Actions has no inline preview for artifact
+  contents, and inlining the PNGs as `data:` URIs doesn't help — the
+  markdown sanitizer strips them. Deploying the generated gallery to
+  Pages gives a stable URL to link from the job summary (and outlives
+  artifact expiry, which matters once fixtures reference these images).
+  Needs Pages enabled on the repo, plus a decision on whether runs
+  overwrite one `latest/` gallery or are namespaced by run ID. Interim
+  option if Pages is unwanted: push captures to an orphan branch and have
+  the workflow post/update a PR comment with `raw.githubusercontent.com`
+  image links, so they render inline where review happens.
+- **Stop paying the image build tax on every run.** GitHub-hosted runners
+  start with an empty Docker cache, so layer caching only helps within a
+  run — each CI run rebuilds from scratch (~35s of the ~70s total).
+  Options: `docker/build-push-action` with `cache-from/to: type=gha`, or
+  publish the images to GHCR once and have CI pull pinned digests, which
+  folds into the digest-pinning item below.
+- Pin base images by digest and fold this workflow into the main CI one.
+- Deepen what the per-server scenario actually asserts (see Phase 0).
+
 **Tier 2 — VIABLE on both OSes**, proven on branch
 `claude/spike-os-servers` (commit `2de3252`, workflow
 `spike-os-servers.yml`); final run

ep.sh

@@ -0,0 +1,34 @@
+#!/bin/sh
+# Start a TigerVNC (Xvnc) server, with or without authentication.
+#
+# VNC_PASSWORD unset or empty -> SecurityTypes None.
+# VNC_PASSWORD set            -> classic VNC password auth, with the
+#                                password written at start-up by vncpasswd
+#                                so it is never baked into the image.
+# That is the only difference between the two TigerVNC services in
+# docker-compose.yml, so they share this entrypoint and the image.
+set -e
+
+
+
+if [ -n "$VNC_PASSWORD" ]; then
+    mkdir -p /root/.vnc
+    printf '%s' "$VNC_PASSWORD" | vncpasswd -f > /root/.vnc/passwd
+    chmod 600 /root/.vnc/passwd
+    set -- -SecurityTypes VncAuth -PasswordFile /root/.vnc/passwd
+else
+    set -- -SecurityTypes None
+fi
+
+echo Xvnc :0 \
+    "$@" \
+    -rfbport 5900 \
+    -geometry 1024x768 \
+    -depth 24 \
+    -AlwaysShared \
+    -localhost=0
+
+
+
+
+

tests/functional/capture_screenshots.py

@@ -0,0 +1,160 @@
+#!/usr/bin/env python3
+"""Capture a screenshot of every running Docker Compose VNC test server.
+
+Screenshots land in the screenshots directory (``tests/servers/screenshots``
+by default, override with ``VNCDOTOOL_SCREENSHOT_DIR``) alongside a
+self-contained ``index.html`` gallery with every capture inlined, so the
+whole set can be eyeballed by opening one file -- locally with
+``make screenshots``, or in CI by downloading the screenshots artifact.
+
+When run inside GitHub Actions a summary table is also appended to the job
+summary, so the run page says which servers were captured and how big each
+screen was without downloading anything.
+
+Servers that aren't running are skipped, and a capture failure is reported
+rather than raised: this is a diagnostic aid and must not fail a build.
+"""
+
+import base64
+import os
+import sys
+from pathlib import Path
+from typing import List, NamedTuple, Optional
+
+_HERE = Path(__file__).resolve().parent
+# This module's own directory, for test_servers, plus the repo root, so the
+# script works from a checkout without vncdotool having been pip installed.
+sys.path[:0] = [str(_HERE), str(_HERE.parents[1])]
+
+from test_servers import (  # noqa: E402
+    HOST,
+    VNC_SERVERS,
+    VNCServer,
+    port_open,
+    screenshot_dir,
+)
+
+from vncdotool import api  # noqa: E402
+
+CAPTURE_TIMEOUT = 5.0
+
+
+class Capture(NamedTuple):
+    server: VNCServer
+    path: Optional[Path]
+    status: str
+
+
+def capture(server: VNCServer, directory: Path) -> Capture:
+    if not port_open(HOST, server.port):
+        return Capture(server, None, f"skipped, nothing listening on port {server.port}")
+
+    path = directory / f"{server.name}.png"
+    try:
+        with api.connect(f"{HOST}::{server.port}", password=server.password) as client:
+            client.timeout = CAPTURE_TIMEOUT
+            client.captureScreen(str(path))
+    except Exception as exc:  # noqa: BLE001 - diagnostics must not fail the build
+        return Capture(server, None, f"failed, {exc}")
+
+    return Capture(server, path, "captured")
+
+
+def describe(path: Path) -> str:
+    """Human readable size of a captured screenshot, e.g. ``1024x768, 12.3 KiB``."""
+    kib = path.stat().st_size / 1024
+    try:
+        from PIL import Image
+
+        with Image.open(path) as image:
+            return f"{image.width}x{image.height}, {kib:.1f} KiB"
+    except Exception:  # noqa: BLE001 - fall back to the size we can always report
+        return f"{kib:.1f} KiB"
+
+
+def write_gallery(captures: List[Capture], directory: Path) -> Path:
+    """Write a single self-contained HTML page showing every screenshot."""
+    sections = []
+    for item in captures:
+        if item.path is None:
+            body = f"<p class='missing'>{item.status}</p>"
+        else:
+            encoded = base64.b64encode(item.path.read_bytes()).decode("ascii")
+            body = (
+                f"<p class='meta'>{describe(item.path)}</p>"
+                f"<img alt='{item.server.name} screenshot' "
+                f"src='data:image/png;base64,{encoded}'>"
+            )
+        sections.append(
+            f"<section><h2>{item.server.name}"
+            f" <small>port {item.server.port}</small></h2>{body}</section>"
+        )
+
+    index = directory / "index.html"
+    index.write_text(
+        "<!doctype html>\n"
+        "<html lang='en'><head><meta charset='utf-8'>"
+        "<title>vncdotool test server screenshots</title>"
+        "<style>"
+        "body{font-family:sans-serif;margin:2rem;background:#fff;color:#111}"
+        "section{margin-bottom:2.5rem}"
+        "h2{margin-bottom:.25rem}"
+        "small{font-weight:normal;color:#666}"
+        ".meta{margin:.25rem 0;color:#666}"
+        ".missing{color:#a00}"
+        "img{max-width:100%;border:1px solid #ccc}"
+        "</style></head><body>"
+        "<h1>vncdotool test server screenshots</h1>"
+        + "".join(sections)
+        + "</body></html>\n",
+        encoding="utf-8",
+    )
+    return index
+
+
+def write_job_summary(captures: List[Capture]) -> None:
+    """Append a result table to the GitHub Actions job summary, if we're in one."""
+    summary = os.environ.get("GITHUB_STEP_SUMMARY")
+    if not summary:
+        return
+
+    rows = [
+        "## VNC test server screenshots",
+        "",
+        "| server | port | result |",
+        "| --- | --- | --- |",
+    ]
+    for item in captures:
+        detail = describe(item.path) if item.path else item.status
+        rows.append(f"| {item.server.name} | {item.server.port} | {detail} |")
+    rows += [
+        "",
+        "Full size images are in the `screenshots` artifact for this run; "
+        "open `index.html` from it to see them all on one page.",
+        "",
+    ]
+    with open(summary, "a", encoding="utf-8") as handle:
+        handle.write("\n".join(rows))
+
+
+def main() -> int:
+    directory = screenshot_dir()
+    captures = [capture(server, directory) for server in VNC_SERVERS]
+
+    # api.connect() starts a background Twisted reactor thread that outlives
+    # any individual client connection -- without stopping it here this
+    # script hangs on exit instead of finishing.
+    api.shutdown()
+
+    for item in captures:
+        location = item.path if item.path else item.status
+        print(f"{item.server.name}: {location}")
+
+    index = write_gallery(captures, directory)
+    write_job_summary(captures)
+    print(f"gallery: {index}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())