Builder and query compute different statistics against each other (umbrella; #122 is instance 1)

[lucapinello]

The class

Every percentile chorus prints is rank(query_statistic) against background_statistic. Those two are computed by different code in different files, and nothing compares them. When they diverge, the percentile is not a percentile — it ranks a value against a null for a different quantity.

Four confirmed instances, found independently:

#	instance	scale of the divergence
1	#122 — AlphaGenome CHIP layer misclassification. The builder passes info['description'] (no mark name) to classify_chip_layer, so 0 of 2,733 CHIP tracks classify as histone; the query reads identifier, which does carry the mark, and returns histone_marks for 1,075.	1,075 tracks (20.8% of AlphaGenome) compare a 2001 bp statistic against a 501 bp null
2	CHIP/window span parity (new). Builders slice 2*(window//(2*resolution))+1 bins about the centre; the query takes a genomic span and floor/ceil-expands it (core/result.py:157-163).	enformer 128 bp: 3 bins / 384 bp vs 4–5 bins / 512–640 bp at window=501; 15 vs 16–17 at 2001. Borzoi 32 bp likewise. All 5,313 enformer tracks + borzoi's windowed tracks + AlphaGenome's 2,733 CHIP tracks
3	RNA effect statistic (new). The builder aggregates a per-bin mean pooled over all protein-coding exons in the ~1 Mb window (build_backgrounds_alphagenome.py:661, not transcript-scoped); the query takes a mean over one named gene's exons of per-exon sums (scorers.py:272-293).	different quantities, not merely different scales
4	RNA activity denominator (new). The query divides by exon intervals (n_exons, median 8) where AlphaGenome's reference implementation divides by mask extent (gene_mask.sum(), median 3,328 bp) — alphagenome_research/model/variant_scoring/gene_mask.py:53-56.	chorus is ~358–416x under-damped and overstates, up to 170–333x on lowly-expressed tracks

Why it kept happening

resolution=1 oracles hide it. ChromBPNet's builder and query agree exactly at 1 bp, so instance 2 was invisible for months and only appears on binned oracles. And no test anywhere compares a builder statistic to its query counterpart — the two halves of the contract have never been diffed.

The structural fix

One shared statistic helper in chorus/analysis/background_sampling.py, called by both the builders and chorus/analysis/scorers.py — exon path, window slice, aggregation, pseudocount. Any future divergence then becomes a type error rather than a silent numerical bias.

Precedent to copy: chorus/oracles/cherimoya_source/scoring.py is imported by both the Cherimoya oracle and its builder, and tests/test_cherimoya.py:609 asserts the builder's source text contains the import so the sharing cannot be quietly undone.

This is the only change that prevents instance five. Fixing the four instances individually does not.

Notes

• Instance 1's classifier fix should use AlphaGenome's own CHIP_HISTONE/CHIP_TF identifier prefix, not an extension of the shared 15-mark _HISTONE_PATTERNS list: the prefix covers all 1,152 histone tracks (including 77 the patterns miss) with zero effect on other oracles, whereas widening the shared list moves ~105 enformer and ~105 borzoi tracks from 501 to 2001 bp and stales two more backgrounds.
• Instance 4's fix must divide by bins actually summed in the track's own resolution, not by exonic bases: at AlphaGenome's resolution=1 that equals gene_mask.sum() exactly, and at Borzoi's 32 bp it matches the Borzoi builder's own np.mean(pred[rna_exon_bins, idx]) units, so it does not stale Borzoi's 1,543 RNA rows. scorers.py currently reads resolution zero times, so the fix must add resolution-awareness or it silently mis-scales Borzoi by 32x.
• Instances 1–3 need a background rebuild to correct the shipped rows; instance 4 is query-side and free.
• Related: Genome/species consistency is enforced by accident — add a per-row genome field and a hard assert #124's provenance work lets the NPZ record its own window_bp/formula/pseudocount and the query prefer it, which converts this whole class from a rebuild trigger into a data-carried invariant.

🤖 Generated with Claude Code