Spike: UltraVNC (Windows) and Apple Screen Sharing (macOS) on hosted runners

.github/workflows/spike-os-servers.yml

@@ -0,0 +1,108 @@
+name: Spike - OS-hosted VNC servers
+
+# Viability spike: can GitHub-hosted windows-latest / macos-latest runners host
+# a live, OS-bound VNC server (UltraVNC on Windows, Apple Screen Sharing on
+# macOS) that vncdotool can connect to, drive and screenshot?
+#
+# What each server needs, and what it does and doesn't prove, is documented
+# next to the setup scripts in tests/servers/ultravnc/README.md and
+# tests/servers/screen-sharing/README.md. This file only wires those scripts
+# and tests/functional/test_os_servers.py together: the two OSes differ by
+# which script sets the server up and which shell runs it, so they are one
+# matrix job rather than two near-identical ones.
+
+on:
+  push:
+    branches:
+      - claude/spike-os-servers
+      - claude/ultravnc-apple-screen-sharing-b04ev5
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+env:
+  PIP_DISABLE_PIP_VERSION_CHECK: '1'
+  PIP_NO_PYTHON_VERSION_WARNING: '1'
+  VNCDOTOOL_SCREENSHOT_DIR: screenshots
+
+jobs:
+  os-server:
+    name: ${{ matrix.name }}
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 20
+    strategy:
+      # One OS failing must never take down the other: each is a separate
+      # data point about that OS, not a step in a shared pipeline.
+      fail-fast: false
+      matrix:
+        include:
+          - name: Windows - UltraVNC
+            runner: windows-latest
+            shell: pwsh
+            server: ultravnc
+            setup: ./tests/servers/ultravnc/setup.ps1
+            diagnostics: ./tests/servers/ultravnc/collect-diagnostics.ps1
+          - name: macOS - Screen Sharing
+            runner: macos-latest
+            shell: bash
+            server: screen-sharing
+            setup: bash tests/servers/screen-sharing/setup.sh
+            diagnostics: bash tests/servers/screen-sharing/collect-diagnostics.sh
+    defaults:
+      run:
+        shell: ${{ matrix.shell }}
+    steps:
+      - name: Check out source
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.13'
+
+      - name: Install vncdotool
+        run: |
+          python -m pip install --upgrade pip
+          pip install . pillow
+
+      - name: Set up the VNC server
+        run: ${{ matrix.setup }}
+
+      # An open port is not readiness: the first connection to macOS Screen
+      # Sharing is what starts it, and that connection can stall for good
+      # while the next one succeeds immediately.
+      - name: Wait for the server to serve a screen
+        run: python tests/functional/wait_for_servers.py os
+
+      - name: Run OS server functional tests
+        run: python -m unittest discover -v -s tests/functional -t . -p 'test_os_servers.py'
+
+      # Everything below is diagnostics and artifact collection, so it runs
+      # with if: always() -- a failing test is exactly when the screenshot
+      # and the server's own logs are worth having. The steps above are the
+      # test itself and deliberately stop the job when they fail.
+      - name: Capture screenshots and build the gallery
+        if: always()
+        timeout-minutes: 3
+        run: python tests/functional/capture_screenshots.py os
+
+      - name: Collect server diagnostics
+        if: always()
+        run: ${{ matrix.diagnostics }}
+
+      - name: Upload screenshots
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.server }}-screenshots
+          path: screenshots/
+          if-no-files-found: ignore
+
+      - name: Upload diagnostics
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.server }}-diagnostics
+          path: diagnostics/
+          if-no-files-found: ignore

Makefile

@@ -10,6 +10,7 @@ help:
 	@echo "servers-up:	start the docker VNC test servers"
 	@echo "servers-down:	stop the docker VNC test servers"
 	@echo "test-servers:	run functional tests against the VNC test servers"
+	@echo "test-os-server:	run functional tests against this OS's VNC server"
 	@echo "screenshots:	screenshot each running VNC test server into a gallery"
 	@echo "docs:		build documentation"
 	@echo "release:	tag and push current version to trigger PyPI release"

docs/server-compatibility-plan.md

@@ -373,7 +373,13 @@ Tier 1 follow-ups:
 `claude/spike-os-servers` (commit `2de3252`, workflow
 `spike-os-servers.yml`); final run
 [31730610001](https://github.com/sibson/vncdotool/actions/runs/31730610001)
-has both jobs green after four evidence-driven rounds.
+has both jobs green after four evidence-driven rounds. The recipe below
+now lives as checked-in setup/diagnostic scripts under
+`tests/servers/ultravnc/` and `tests/servers/screen-sharing/` (each with
+a README recording why it does what it does), driven by
+`tests/functional/test_os_servers.py`, which reuses the same server
+description, round trip and screenshot gallery as Tier 1 via
+`tests/functional/vncservers.py`.
 
 *Windows / UltraVNC — works, full recipe:*
 - `choco install ultravnc`; `winvnc.exe` lands at the fixed path

tests/functional/capture_screenshots.py

@@ -1,5 +1,10 @@
 #!/usr/bin/env python3
-"""Capture a screenshot of every running Docker Compose VNC test server.
+"""Capture a screenshot of every running VNC test server.
+
+Which servers those are is chosen by the group named on the command line:
+``docker`` (the default, the Docker Compose servers) or ``os`` (the
+OS-hosted server on this machine, i.e. UltraVNC or Apple Screen Sharing).
+See vncservers.py.
 
 Screenshots land in the screenshots directory (``tests/servers/screenshots``
 by default, override with ``VNCDOTOOL_SCREENSHOT_DIR``) alongside a
@@ -22,21 +27,22 @@
 from typing import List, NamedTuple, Optional
 
 _HERE = Path(__file__).resolve().parent
-# This module's own directory, for test_servers, plus the repo root, so the
+# This module's own directory, for vncservers, plus the repo root, so the
 # script works from a checkout without vncdotool having been pip installed.
 sys.path[:0] = [str(_HERE), str(_HERE.parents[1])]
 
-from test_servers import (  # noqa: E402
+from vncservers import (  # noqa: E402
     HOST,
-    VNC_SERVERS,
     VNCServer,
+    capture_screenshot,
     port_open,
     screenshot_dir,
+    select_servers,
 )
 
 from vncdotool import api  # noqa: E402
 
-CAPTURE_TIMEOUT = 5.0
+DEFAULT_GROUP = "docker"
 
 
 class Capture(NamedTuple):
@@ -51,9 +57,7 @@ def capture(server: VNCServer, directory: Path) -> Capture:
 
     path = directory / f"{server.name}.png"
     try:
-        with api.connect(f"{HOST}::{server.port}", password=server.password) as client:
-            client.timeout = CAPTURE_TIMEOUT
-            client.captureScreen(str(path))
+        capture_screenshot(server, path)
     except Exception as exc:  # noqa: BLE001 - diagnostics must not fail the build
         return Capture(server, None, f"failed, {exc}")
 
@@ -137,9 +141,10 @@ def write_job_summary(captures: List[Capture]) -> None:
         handle.write("\n".join(rows))
 
 
-def main() -> int:
+def main(argv: List[str]) -> int:
+    group = argv[1] if len(argv) > 1 else DEFAULT_GROUP
     directory = screenshot_dir()
-    captures = [capture(server, directory) for server in VNC_SERVERS]
+    captures = [capture(server, directory) for server in select_servers(group)]
 
     # api.connect() starts a background Twisted reactor thread that outlives
     # any individual client connection -- without stopping it here this
@@ -157,4 +162,4 @@ def main() -> int:
 
 
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(main(sys.argv))

tests/functional/test_os_servers.py

@@ -0,0 +1,26 @@
+"""Functional tests against the VNC server hosted by the OS we run on.
+
+Unlike the Docker servers in test_servers.py, these servers are part of the
+operating system rather than something we build: UltraVNC installed as a
+Windows service, or Apple Screen Sharing / Remote Management on macOS. The
+setup scripts live next to each server's notes in tests/servers/ultravnc
+and tests/servers/screen-sharing, and are what CI runs before this module.
+
+On a platform with no OS-hosted server described (Linux, say) there is
+simply nothing to register, and on a platform that has one but hasn't set
+it up the test skips with the command that would set it up.
+"""
+
+from vncdotool import api
+
+from .vncservers import os_servers, register_server_tests
+
+register_server_tests(os_servers(), globals())
+
+
+def tearDownModule() -> None:
+    # api.connect() starts a background Twisted reactor thread that
+    # outlives any individual client connection. Without stopping it here,
+    # the interpreter (and `unittest discover`) hangs on exit after all
+    # tests in this module have finished.
+    api.shutdown()

tests/functional/test_servers.py

@@ -6,142 +6,21 @@
 a clear message rather than failing the whole run, so this module is also
 safe to run outside of that make target.
 
+The servers themselves, and the round trip run against each of them, are
+described in vncservers.py and shared with the OS-hosted servers tested by
+test_os_servers.py.
+
 Screenshots captured here are kept rather than thrown away: each one is
 written to the screenshots directory (``tests/servers/screenshots`` by
 default, override with ``VNCDOTOOL_SCREENSHOT_DIR``) so that a failing or
 suspicious capture can be looked at directly after the run.
 """
 
-import os
-import socket
-from pathlib import Path
-from typing import NamedTuple, Optional, Tuple
-from unittest import TestCase
-
-from PIL import Image
-
 from vncdotool import api
 
-HOST = "127.0.0.1"
-CONNECT_TIMEOUT = 5.0
-PORT_PROBE_TIMEOUT = 1.0
-
-DEFAULT_SCREENSHOT_DIR = Path(__file__).resolve().parents[1] / "servers" / "screenshots"
-
-
-class VNCServer(NamedTuple):
-    name: str
-    port: int
-    password: Optional[str]
-    # Screen size the server's entrypoint configures, so a capture can be
-    # checked against what the server said it was serving.
-    size: Tuple[int, int] = (1024, 768)
-
-
-# (name, port, password) for each service in tests/servers/docker-compose.yml
-VNC_SERVERS = [
-    VNCServer("tigervnc", 5931, None),
-    VNCServer("tigervnc-auth", 5932, "vncdotool"),
-    VNCServer("x11vnc", 5933, None),
-]
-
-PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
-# getcolors() returns None above this many distinct colours, which is itself
-# proof the capture isn't a flat colour, so the cap only needs to be cheap.
-MAX_COLOURS = 256
-
-
-def screenshot_dir() -> Path:
-    """Directory screenshots are written to, created if needed."""
-    path = Path(os.environ.get("VNCDOTOOL_SCREENSHOT_DIR", DEFAULT_SCREENSHOT_DIR))
-    path.mkdir(parents=True, exist_ok=True)
-    return path
-
-
-def port_open(host: str, port: int, timeout: float = PORT_PROBE_TIMEOUT) -> bool:
-    try:
-        with socket.create_connection((host, port), timeout=timeout):
-            return True
-    except OSError:
-        return False
-
-
-class _VNCServerTestMixin:
-    """Shared test body, parameterized per-server by _make_server_test_case below.
-
-    Deliberately does NOT subclass TestCase: only the dynamically generated
-    per-server classes below should (otherwise `unittest discover` also
-    collects this shared base as its own, serverless test case).
-    """
-
-    server: VNCServer
-
-    def setUp(self) -> None:
-        if not port_open(HOST, self.server.port):
-            self.skipTest(
-                f"{self.server.name} not reachable on {HOST}:{self.server.port} -- "
-                "start the servers first with `make servers-up`"
-            )
-
-    def test_connect_key_and_capture(self) -> None:
-        """End-to-end round trip against one real server.
-
-        Passing means, in order:
-
-        * the RFB handshake completed against this server's protocol version
-          and security type (None, or VNC password auth for tigervnc-auth);
-        * a key event was accepted without the server dropping the session;
-        * a framebuffer update was received and encoded to a PNG file;
-        * that PNG is the size the server's entrypoint configured, and is not
-          a single flat colour -- i.e. we decoded real screen content
-          (the entrypoints run xlogo for exactly this reason) rather than
-          the all-black framebuffer you get when updates never arrive.
-        """
-        address = f"{HOST}::{self.server.port}"
-        png = screenshot_dir() / f"{self.server.name}.png"
-
-        with api.connect(address, password=self.server.password) as client:
-            client.timeout = CONNECT_TIMEOUT
-            client.keyPress("x")
-            client.captureScreen(str(png))
-
-        data = png.read_bytes()
-        print(f"{self.server.name}: screenshot written to {png}")
-
-        self.assertTrue(data, f"{self.server.name}: captured screenshot is empty")
-        self.assertEqual(
-            data[:8],
-            PNG_MAGIC,
-            f"{self.server.name}: captured file is not a valid PNG",
-        )
-
-        with Image.open(png) as image:
-            self.assertEqual(
-                image.size,
-                self.server.size,
-                f"{self.server.name}: capture is not the size the server serves",
-            )
-            colours = image.convert("RGB").getcolors(maxcolors=MAX_COLOURS)
-
-        self.assertNotEqual(
-            colours if colours is None else len(colours),
-            1,
-            f"{self.server.name}: capture is a single flat colour, "
-            "no screen content was decoded",
-        )
-
-
-def _make_server_test_case(server: VNCServer) -> type:
-    class_name = "TestServer_" + server.name.replace("-", "_")
-    return type(class_name, (_VNCServerTestMixin, TestCase), {"server": server})
-
+from .vncservers import DOCKER_SERVERS, register_server_tests
 
-# Register one TestCase subclass per test server so `unittest discover`
-# reports pass/fail/skip separately for each server.
-for _server in VNC_SERVERS:
-    _test_case = _make_server_test_case(_server)
-    globals()[_test_case.__name__] = _test_case
-del _server, _test_case
+register_server_tests(DOCKER_SERVERS, globals())
 
 
 def tearDownModule() -> None: