feat(inference): opt-in llm-d EPP+Envoy router (drop-in for vllm-router)

[S1ro1]

Summary

• Adds deployment.router_backend = "llm-d" (default "vllm-router") on multi-node and disaggregated deployments. When opted in, the SLURM template launches the upstream llm-d EPP + Envoy sidecar (and pd-sidecar for P/D) instead of vllm-router. No Docker, no Kubernetes, no sudo.
• Reuses upstream-vetted EPP scoring profiles verbatim: optimized-baseline for multi-node and pd-disaggregation (disagg-headers + prefix-based-pd-decider + disagg-profile-handler + prefill/decode filters + separate prefill/decode scheduling profiles) for P/D.
• Wires the canonical llm-d pd-sidecar on each decode node: Envoy → sidecar on decode_sidecar_port → sidecar sends a max_tokens=1 prefill to a prefill worker, then forwards the actual decode (with do_remote_prefill=true + kv_transfer_params) to local vLLM. Decode then pulls KV from prefill via NIXL.
• scripts/install_llmd.sh bootstraps a local Go toolchain, go installs the EPP and pd-sidecar from github.com/llm-d/llm-d-router, and pulls Envoy 1.36 from archive.tetratelabs.io into third_party/llmd/bin/.

Known limitation — renderer / routed-experts unsupported

llm-d's EPP openai-parser understands only OpenAI-format requests (/v1/chat/completions, /v1/completions). prime-rl's renderer/TITO client and routed-experts replay both POST to /inference/v1/generate with the raw-tokens schema (prompt_token_ids), which the EPP rejects with BadRequest - invalid completions request: must have prompt field. Until upstream llm-d adds raw-tokens support, an RLConfig validator blocks the combination at config-load time with a clear error pointing users to vllm-router. Tracked for follow-up with the llm-d maintainers.

Validated on SLURM (Qwen3-30B-A3B, dp=8, 1+1 multi-node and 1+1+1 P/D)

Path	Result
router_backend = "llm-d" + multi-node + MITO	✅ all 5 steps, 257 POSTs to /v1/chat/completions, loss=0.0217
router_backend = "llm-d" + P/D + MITO (with pd-sidecar)	✅ canonical PD lifecycle: Envoy → sidecar → prefill+decode split; POST counts balance across both vLLMs (prefill ~31, decode ~36 per ~30 inference requests = 1 prefill + 1 decode each). Step 0 → Step 1 completed cleanly.
router_backend = "llm-d" + renderer / routed-experts	❌ blocked at config-validation (intentional)
router_backend = "vllm-router" (default)	unchanged

Files

• New: scripts/install_llmd.sh, scripts/write_llmd_configs.sh
• Config: packages/prime-rl-configs/src/prime_rl/configs/{inference,rl}.py (new router_backend, decode_sidecar_port, validator)
• Plumbing: src/prime_rl/entrypoints/{inference,rl}.py
• Templates: src/prime_rl/templates/{inference,multi_node_rl}.sbatch.j2 — branch on router_backend; the llm-d branch invokes write_llmd_configs.sh and launches epp/envoy (router) + pd-sidecar (each decode node, PD only).
• Docs: docs/{slurm,disaggregated-inference,logging}.md, skills/training/start-run/SKILL.md

🤖 Generated with Claude Code