feat(demux): fused CTC-CRF pipeline, self-describing bundles, LLR opt-in

[jayhesselberth]

Makes escpod demux in.pod5 --model <bundle> -d out/ produce barcoded POD5s with nothing else on the command line, and adds --info so a model can be interrogated before it is trusted.

Fused CRF head

The fused pipeline drove only the fingerprint heads (DTW-SVM, GBM). The CRF was reachable only as demux basecall, which takes boundaries as input, so using it meant detect → basecall → split: two intermediate files and three passes over the POD5.

--model now accepts a CRF bundle directory (sniffed by the sidecar's format key, not the extension) and runs the CRF inline — detect → prep the raw-pA window → basecall → match by edit distance → route — decoding each read once.

Verified against the 3-step path, 4,000 reads, same detector, encoder and references:

• all 3,993 shared reads get an identical call; every barcode bin matches exactly
• the fused path emits 4,000 rows to the 3-step path's 3,993. basecall drops reads with no usable window, so split never sees them; the fused path routes them unclassified like the other heads. Output now reconciles with input.

Bundles describe themselves

metadata.json gains optional barcodes, boundary, model and metrics. So this:

escpod demux in.pod5 --model crf_bundle/ --barcodes refs.csv -d out/ \
    --method cnn --cnn-model adapter.onnx

becomes:

escpod demux in.pod5 --model crf_bundle/ -d out/

Carrying references in the bundle is not only ergonomics. The CRF has state_len=4 and emits target[4:], so a hand-written CSV of full-length targets still calls the same barcode but inflates every edit distance by 4 and compresses the confidence margin that --min-margin and --recovery rank on (rnabioco/escapepod-models#36). Measured on 20,000 reads with the shipped weights: median best_dist 4 → 0, margin median 10 → 12/13, distinct margin values 12/11 → 15/14. Deriving them at bundle-build time from the encoder's own state_len removes the failure mode rather than documenting it. --barcodes remains as an override.

New --info prints identity, geometry, references with their minimum pairwise distance, the pinned detector, published metrics and caveats, then exits without touching a POD5.

LLR is opt-in, never inferred

--method had default_value = "llr". LLR boundaries cost 17.2 points of barcode recall against the same classifier (0.9928 → 0.8196) and the failure is silent — it runs and produces plausible output (rnabioco/escapepod-models#16). It now has no default:

• a bundle may pin its detector, supplying both method and weights;
• an explicit --method overrides that pin — except a bundle pinned to cnn refuses --method llr, which is the runtime guard perf: update bam-filter to use optimized filter_files() #16 asks for;
• with neither, demux and demux detect error out naming the tradeoff instead of quietly picking the worse detector.

Warning

Breaking. escpod demux and escpod demux detect now require --method when the model does not pin one. Scripts relying on the implicit llr default must add --method llr to keep current behaviour — which is the point, since that default was silently costing 17.2 points.

Interaction with #166

#166 touches demux/detect.rs and demux/mod.rs, as does this. The conflicts are textual and small (it adds --cnn-model-name; this changes --method to an Option), but they do need resolving. #166 is older and independent, so merging it first and rebasing this is the natural order.

Worth reconciling separately: both PRs use the word bundle for different things — #166's is a release archive of matched models with a BUNDLE.json, this one's is a directory the CRF loader reads via metadata.json. They are complementary, but the naming will confuse someone.

Testing

156 tests pass. Adds a unit test that --model sniffing keys off the sidecar's format field rather than the path, so a classifier JSON beside a bundle, a directory without a sidecar, and a malformed sidecar all still route to load_any_model.

Manually verified: the fused CRF short form, the long form with explicit flags, GBM regression through the fused path, and all four guard paths (llr-vs-pinned-cnn, no-method-no-pin, detect with no method, and explicit llr opt-in on an unpinned model).

🤖 Generated with Claude Code