Roadmap: study-anchored external metadata (phsid → dbGaP record + methods paper → justify metadata picks)

[NoopDog]

Idea

Every AnVIL dataset is a dbGaP study (a phs accession). The phs anchor gives us two study-level external authorities, and this roadmap covers both:

1. The structured dbGaP record — the dbGaP FHIR API (https://dbgap-api.ncbi.nlm.nih.gov/fhir/x1) exposes a ResearchStudy per phsid with fields/extensions that look directly relevant (molecular data types, study design, focus/condition, consent groups). A metadata source in its own right, alongside the consortium catalogs.
2. The study's marker/methods publication(s) — these are publicly funded datasets, so a paper describing the study is effectively mandated. Its methods section is an authoritative, citable source.

Either source can fill basic metadata (a single-species study ⇒ organism = Homo sapiens) or constrain it (study says RNA-seq only ⇒ a genomic classification for its files is suspect; a stated subset of assays/platforms narrows the search space and validates our picks).

The end goal: justify metadata picks from study-level authorities — every derived fact carries its provenance (a FHIR field, or a quote + citation from the paper), in line with the no-speculation principle.

This slots into the ADR-0001 architecture (import-primary + infer-tail): dbGaP record and paper are external authorities anchored off the phs accession, alongside consortium catalogs. See docs/adr/0001-classification-architecture.md — AnVIL is 78 phs studies, 13 open-access.

Roadmap

Three epics, filed one at a time — each phase informs whether/how the next is worth doing:

• Epic 1 — Query the phs anchor: a skill that takes a phsid and (a) pulls the structured dbGaP record via the FHIR API, surveying which metadata-relevant fields it actually populates, and (b) locates the study's publication(s), with a documented provenance chain. Also a source survey: learn where the papers actually live (dbGaP FHIR API, dbGaP study pages, NCBI links, ncpi-dataset-catalog's dbGaP studies list, …). → Epic 1: skill to query the phs anchor — dbGaP FHIR record + publication(s) for a phsid #324
• Epic 2 — Read the paper (not filed yet): given the publication(s), read the methods text and extract candidate study-level facts (organism, DNA/RNA, assay list, platform list, reference notes), each with a supporting quote + citation. Depends on Epic 1 showing papers are findable and informative.
• Epic 3 — Use the results (not filed yet): decide integration for both sources — constraints layer vs claims in the tier system vs both; interaction with the consistency linter (Self-consistency check: cross-field semantic rules over each classified record (no ground truth needed) #314) and the origin registry (Build the dataset -> origin registry: map all 13 AnVIL datasets to their source of publication and import from there #304). Explicitly deferred: no rule-engine work until Epics 1–2 show what the sources can support.

Non-goals (for now)

• No changes to rule_engine.py, unified_rules.yaml, or the claim-tier model.
• No new schema dimensions (e.g. organism) — that's an Epic 3 decision.