Skip to content

Commit 05feb91

Browse files
committed
Docs(fix[docs-tests]): Say what else it checks, and how
why: The module reads more than it compiles, and only the compiling half was written down. DocumentationFactsTest holds the versions in install blocks, the platform's own README and the parity citations to the build, and nothing said so — the first anyone learns of it is a failure naming a check no document mentions. The gate a contributor is told to run before opening a pull request did not route to any of it either. what: - Add a section for the claims that are not code, one row per check and where it looks, including the two documents nothing takes snippets from - Say a type-declaring block is compiled and never run, whatever its directive: a declaration has nothing to execute - Say throws: matches the exception's simple name, and that the directive comment has to sit directly above the fence - Add yamlString to the fixtures a snippet may assume, and stop counting them in prose; the count is the code's to own - Scope the document list to snippets, since the other checks read wider - Name :docs-tests:test in the checks that must pass and route to this README from there, beside the code that reads the directives
1 parent 2e15327 commit 05feb91

2 files changed

Lines changed: 60 additions & 13 deletions

File tree

.github/CONTRIBUTING.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Nothing about this is a convention you have to remember:
1818
| `integration-tests/` | no | the real-tmux suite, which spans artifacts |
1919
| `benchmarks/` | no | the carrier measurements |
2020
| `examples/` | no | whole runnable programs, run by its own suite |
21-
| `docs-tests/` | no | compiles and runs every snippet in the docs |
21+
| [`docs-tests/`](../docs-tests/) | no | compiles and runs every snippet in the docs |
2222
| `scripts/` | no | what the build does not do |
2323
| `build-logic/` | no | convention plugins, as an included build |
2424
| `docs/`, `gradle/`, `.github/` | no | everything else |
@@ -126,9 +126,20 @@ $ ./gradlew check
126126
```
127127

128128
It runs formatting, Error Prone, NullAway in JSpecify mode, and every test
129-
including the ones that start real tmux servers. It also compiles and runs every
130-
Java snippet in `README.md`, in each module's README, and in the guides — a
131-
snippet that stopped working fails the build.
129+
including the ones that start real tmux servers.
130+
131+
The Java in the documentation is compiled and run as part of that, and the
132+
claims around it — the version in every install block, what the platform says it
133+
manages — are checked with it:
134+
135+
```console
136+
$ ./gradlew :docs-tests:test
137+
```
138+
139+
How a fence says what it is — the directives, the `// →` assertions, the
140+
fixtures a snippet may assume — is in
141+
[`docs-tests/README.md`](../docs-tests/README.md), beside the code that reads
142+
them.
132143

133144
A green `check` that reported `UP-TO-DATE` for every task verified nothing.
134145
Force it when that matters:

docs-tests/README.md

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# docs-tests
22

3-
**Compiles and runs the code in the documentation. Not published.**
3+
**Compiles and runs the code in the documentation, and checks the claims around
4+
it. Not published.**
45

56
A snippet is the part of a project people copy and the part nothing compiles, so
67
it goes stale silently — and a stale snippet reads exactly as well as a working
@@ -32,10 +33,20 @@ Say otherwise with an HTML comment directly above the fence:
3233
| `<!-- snippet: compile-only: <reason> -->` | compiles; not run, for the stated reason |
3334
| `<!-- snippet: skip: <reason> -->` | not checked, for the stated reason |
3435

35-
An unrecognised directive fails the build. A snippet nobody is checking, because
36-
of a typo in the thing that says how to check it, is the state this exists to
36+
The comment has to sit directly above the fence, with nothing between them. An
37+
unrecognised directive fails the build: a snippet nobody is checking, because of
38+
a typo in the thing that says how to check it, is the state this exists to
3739
prevent.
3840

41+
`throws:` names the exception's simple name, not its package — the comparison is
42+
against `getClass().getSimpleName()`, so `IllegalArgumentException` matches and
43+
`java.lang.IllegalArgumentException` does not.
44+
45+
A block that declares a type — a `class`, `record`, `interface` or `enum` — is
46+
compiled and never run, whatever its directive says, because a declaration has
47+
nothing to execute. Statements are wrapped in a method body with the fixtures
48+
below in scope; a type is compiled as it stands.
49+
3950
`does-not-compile` earns its keep: it is what keeps
4051
`Pane_.index().startsWith("2")` an error. A README claiming the compiler rejects
4152
something would otherwise survive the day it stopped being true.
@@ -72,11 +83,11 @@ Two consequences worth knowing:
7283

7384
## What a snippet may assume
7485

75-
Documentation shows the interesting line, not the six before it that made a
76-
server. Those six are supplied: `server`, `config`, `session`, `window`, `pane`,
77-
`options`, `socket`, `directory`, `timeout`, and the common imports. A snippet
78-
declaring its own `server` shadows the supplied one, which is what a reader
79-
copying it would get anyway.
86+
Documentation shows the interesting line, not the ones before it that made a
87+
server. Those are supplied: `server`, `config`, `session`, `window`, `pane`,
88+
`options`, `socket`, `directory`, `timeout`, `yamlString`, and the common
89+
imports. A snippet declaring its own `server` shadows the supplied one, which is
90+
what a reader copying it would get anyway.
8091

8192
Consequently a fence cannot depend on a variable another fence declared — and
8293
neither can a reader who copies just that fence.
@@ -96,9 +107,34 @@ documentation, the two cannot drift.
96107
$ ./gradlew :libtmux-kotlin:test
97108
```
98109

110+
## Claims that are not code
111+
112+
A snippet is executed, so it cannot lie. A version in an install block, or a
113+
list of what the platform manages, is prose — and prose is what is still wrong
114+
six months later, in the one place every reader starts. Those are checked too:
115+
116+
| what is checked | where it looks |
117+
| --- | --- |
118+
| Every coordinate names the version this build would publish | the root README, `libtmux-bom`'s, every published module's, the Kotlin and Scala guides, and `RELEASING.md` |
119+
| `libtmux-bom`'s README lists exactly what the platform constrains | that README against `libtmux-bom/build.gradle.kts` |
120+
| Every published module's README names it first and states its coordinate | each published module's README |
121+
| A fence in a source language nothing here builds carries a directive saying so | every reader-facing document |
122+
| The contract tests the parity documents cite are unwritten or really declared | `docs/parity/python-api.md`, `docs/parity/test-map.md` |
123+
| Those documents keep saying "planned parity" while those tests are unwritten | the same two |
124+
125+
The last two are why this module reads documents it takes no snippets from.
126+
`docs/parity/` holds no Java, and `RELEASING.md` is not a place snippets come
127+
from, but a coordinate in either is a claim like any other.
128+
129+
The snippet suite also asserts a floor on how much it found. A filter or a
130+
rename can reduce a parameterised suite to nothing without failing anything, and
131+
a suite that discovers nothing passes loudly.
132+
99133
## Which documents
100134

101-
`README.md`, every package's `README.md`, and `docs/guide/*.md`.
135+
Snippets come from `README.md`, every package's `README.md`, and
136+
`docs/guide/*.md`. The checks above that are not about snippets read more than
137+
that, and each row says where it looks.
102138

103139
Not `docs/spikes`, `docs/plans` or `docs/studies`: those are dated records of what
104140
was measured or decided at the time. Holding them to today's API would either

0 commit comments

Comments
 (0)