Skip to content

[codex] Offload orchestrator rollout postprocess#2646

Draft
samsja wants to merge 1 commit into
mainfrom
codex/offload-orchestrator-postprocess
Draft

[codex] Offload orchestrator rollout postprocess#2646
samsja wants to merge 1 commit into
mainfrom
codex/offload-orchestrator-postprocess

Conversation

@samsja
Copy link
Copy Markdown
Member

@samsja samsja commented May 27, 2026

Summary

Moves the CPU-heavy rollout postprocess path out of the orchestrator event loop behind one worker boundary.

  • Adds build_training_batch_and_metrics(...) in prime_rl.orchestrator.postprocess for advantages, filters, accepted-rollout save, image offload, token backfill, rollout interleaving, training batch construction, and rollout metric aggregation.
  • Leaves the main coroutine responsible for async-only work: scheduler retry control, teacher logprobs, trainer send, monitor logging, heartbeat, and cleanup.
  • Preserves accepted-rollout artifact behavior by saving train_rollouts.jsonl before image offload, and preserves all-filtered retry behavior without writing rollout artifacts for rejected attempts.

Validation

  • uv run ruff check src/prime_rl/orchestrator/orchestrator.py src/prime_rl/orchestrator/postprocess.py
  • uv run pytest tests/unit/orchestrator/test_advantage.py tests/unit/orchestrator/test_filters.py tests/unit/orchestrator/test_trajectories.py tests/unit/orchestrator/test_sft_trajectories.py tests/unit/orchestrator/test_orchestrator_setup.py -q

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant