Benchmark: does an evidence map beat a cited memo?

[alsoleg89]

Why this matters

Doubt's deterministic contract proves structural traceability and safe rendering, but the main product hypothesis is still open: does an interactive evidence map help a reader inspect a contested decision better than a well-cited memo?

The study measures that directly instead of turning an untested preference into a product claim.

Current protocol

The preregistered study is live at:

https://alsoleg89.github.io/doubt/benchmark/

It includes:

• five contested, non-sensitive technical topics;
• one validated Doubt map and one hand-edited cited memo per topic;
• the same frozen source set in both conditions;
• shuffled topic order and balanced 2/3 condition assignment;
• 25 keyed tasks covering position, contradiction, qualification, unknown, and exact source region;
• anonymous local JSON export with no account, analytics, telemetry, or network submission;
• a fail-closed analyzer that refuses condition-level output before ten complete unique sessions.

Protocol and source artifacts:

https://github.com/alsoleg89/doubt/tree/main/benchmarks/map-vs-memo

Remaining acceptance criteria

• Protocol and hypotheses are committed before results are collected.
• Five topics use paired map and memo artifacts.
• Map and memo use the same source material.
• No participant-identifying data is collected or committed.
• Collect at least ten complete valid reader sessions.
• Publish anonymized aggregate results, including failures and neutral outcomes.
• Update the README product claim to match the observed result.

No reader-result claim should be published before the stopping rule is met.

[alsoleg89]

The preregistered reader study is now live:

https://alsoleg89.github.io/doubt/benchmark/

Implemented in 3c58002:

• five contested technical topics;
• one validated Doubt map and one hand-edited cited memo per topic;
• the same frozen source set in both conditions;
• local shuffled topic order and balanced 2/3 condition assignment;
• 25 keyed tasks covering position, contradiction, qualification, unknown, and exact source region;
• no account, personal-data fields, analytics, telemetry, or network submission;
• anonymous local JSON export;
• a fail-closed analyzer that refuses condition-level output before 10 complete unique sessions.

Verification:

• 20/20 automated tests pass;
• all five maps and memo source coverage validate;
• CI passes on Node 18, 20, and 22;
• the deployed HTML matches the committed artifact at SHA-256 22130c0b3fbccb28945f8566774b78d104df338d65a6142dcd5c6f5a25450758.

No reader result is claimed yet. Per the committed stopping rule, the first comparison will be published only after 10 complete sessions, including neutral or negative outcomes.