Flaky: transient setup error in the first test of a class following another Solr test class

[reebalazs]

Status

Currently cannot be reproduced. Filed so the context is findable when (if) it resurfaces — most likely in CI, where timing differs from local runs. If it stays stale for a long time, close it.

Observed (once, 2026-07-12, local)

During the test-setup work for #81, a single occurrence in a combined run of two test directories:

pytest tests/services/suggest tests/services/response
ERROR tests/services/suggest/test_suggest.py::TestSuggestDefaultBaseSearch::test_suggest_result_path[0-expected_dict0]
6 passed, 1 error

• The error was at setup of the first test of the class that ran second (collection order: response before suggest), right after the previous directory's teardown.
• Immediate re-runs of the exact same command passed (twice), as did the reversed order and many subsequent full-suite runs (3× 449–577 tests, all green).
• Environment at the time: pre-class-scoping fixture layout (function-scoped portal/portal_with_content, i.e. per-test index clear + content creation), the session layer-keeper fixture already applied.

Hypothesis

A Solr commit-visibility race between consecutive test setups/teardowns: one test's teardown deletes its content (committed to Solr), and the next test's maintenance.clear() + content creation + query occasionally observes a stale index state — e.g. the clear/adds racing an in-flight commit from the previous teardown.

Notes:

• Solr is not transactional with the ZODB: index mutations from a previous test are only reconciled by the explicit clear/delete calls in the fixtures, so ordering/visibility between those HTTP calls is where a race can live.
• The class-scoped fixture layout introduced in Solr test suite: runtime regression since the unittest to pytest migration #81 shrinks the window considerably (far fewer clear/create cycles per run), so the flake should now be rarer — but the same pattern still exists between test classes.

If it resurfaces

• Capture the full setup traceback (the one occurrence was only recorded as a summary line).
• Check whether the failing test is the first of its class and which class ran before it.
• A likely mitigation would be an explicit hard commit (waitSearcher=true) or a short readiness check after maintenance.clear() in the portal fixture.