-
Notifications
You must be signed in to change notification settings - Fork 140
Description
Problem
When agents run out of time during self-improvement iterations, they get cut off mid-improvement with no chance to submit their current best answer. This can result in lost work — the agent may have a strong answer in progress that never gets submitted.
Observed in log_20260308_092606_670139 where the run ended early due to a framework error during round evaluation, but the broader pattern applies: agents should always have a chance to submit before timeout.
Proposed Solution
When X minutes remain in the orchestrator timeout, inject a message telling agents to finalize and submit their current best answer via new_answer.
- Trigger at a configurable threshold (e.g., 80% of timeout elapsed or a fixed minutes-remaining value)
- Give agents a firm but graceful wrap-up window
- The message should instruct: "Time is running low. Finalize your current work and submit your best answer via
new_answer." - Wait until all agents have submitted their answers
- Once all answers are in, the normal flow continues — agents enter the next round where they can only vote, as usual
- If an agent still hasn't submitted after the wrap-up window, force-submit current workspace state as their answer
Key Principle
Agents should NOT be told to vote during the wrap-up window. Submission and voting are separate phases. The wrap-up window only ensures every agent submits a new_answer. The subsequent round naturally moves agents into vote-only mode.
Context
This came up during analysis of stuck-agent behavior. The self-improvement → plateau → evaluator rescue cycle works well, but without a graceful timeout, agents can be cut off before submitting any answer at all. The existing soft/hard timeout hooks handle tool blocking but don't explicitly guide agents toward submission.
Related
- Round evaluator feedback quality improvements (implementation_guidance field)
- Coordination workflow documentation updates