HPRC validation: our not_applicable vs a declared catalog value should score as mismatch, not unknown

[NoopDog]

Found while reviewing validation behavior for the project-overview deck (2026-08-18).

Current behavior

scripts/validate_against_hprc.py → validate_dimension() treats all of our sentinel outputs alike:

_SENTINELS = {"not_classified", "not_applicable", ""}
if not our_value or our_value.lower() in _SENTINELS:
    counters[f"{dim_name}_unknown"] += 1

So when the HPRC catalog declares a value (e.g. library_strategy → assay) and we assert not_applicable, the pair is scored unknown — the same as "we couldn't tell."

Why that's wrong

not_classified is a non-answer, and unknown is the right bucket for it. But not_applicable is a positive assertion — "this dimension makes no sense for this file." When the authority declared a value for that very dimension, our assertion of inapplicability is contradicted, and lumping it with non-answers under-penalizes it. It should count as a mismatch (and appear in the mismatches list with ours: not_applicable).

The other direction (report, don't score)

When the catalog omits the field and we say not_applicable, that pair is currently skipped entirely (no ground truth). Field absence is ambiguous — "doesn't apply" vs "catalog incomplete" — so it must NOT be credited as agreement. Worth reporting as a separate consistent_na count outside the accuracy figures, purely for visibility.

Definition of done

• validate_dimension splits sentinels: not_classified/empty → unknown; not_applicable vs a declared value → mismatch (with mismatch detail recorded)
• Omitted-field + our not_applicable reported as a separate consistent_na counter, excluded from accuracy
• output/hprc/hprc_validation_current.json regenerated; any accuracy changes noted
• Same policy reviewed for the other per-file validators (1000G, ENA, AnVIL comparison) — apply or note why not

🤖 Generated with Claude Code

[NoopDog]

Drift evidence from the #330 review (PR #332), for when this issue consolidates the policy: the ENA and HPRC validators already disagree on what "unknown" means.

• validate_ena_accessions.py: sentinel set {not_classified, not_applicable, conflict, ""}, keyed on the status field, case-sensitive compare (platform upcased explicitly).
• validate_against_hprc.py validate_dimension: _SENTINELS = {not_classified, not_applicable, ""} — no conflict, keyed on the value/label, and case-folds both sides.

So conflict scores unknown in one validator and could score mismatch in the other, and the two make different case-sensitivity assumptions. A shared verdict helper + sentinel set (models.py has the status constants; neither validator imports them for this) is the natural shape when this lands. The ENA validator's results file is also currently unreadable by generate_validation_report.py (flat mismatch entries vs the nested shape the report parser expects) — worth folding into the same consolidation.

🤖 Generated with Claude Code