Skip to content

[FEATURE] Graceful timeout: submit current answers when time is running low #984

@ncrispino

Description

@ncrispino

Problem

When agents run out of time during self-improvement iterations, they get cut off mid-improvement with no chance to submit their current best answer. This can result in lost work — the agent may have a strong answer in progress that never gets submitted.

Observed in log_20260308_092606_670139 where the run ended early due to a framework error during round evaluation, but the broader pattern applies: agents should always have a chance to submit before timeout.

Proposed Solution

When X minutes remain in the orchestrator timeout, inject a message telling agents to finalize and submit their current best answer via new_answer.

  • Trigger at a configurable threshold (e.g., 80% of timeout elapsed or a fixed minutes-remaining value)
  • Give agents a firm but graceful wrap-up window
  • The message should instruct: "Time is running low. Finalize your current work and submit your best answer via new_answer."
  • Wait until all agents have submitted their answers
  • Once all answers are in, the normal flow continues — agents enter the next round where they can only vote, as usual
  • If an agent still hasn't submitted after the wrap-up window, force-submit current workspace state as their answer

Key Principle

Agents should NOT be told to vote during the wrap-up window. Submission and voting are separate phases. The wrap-up window only ensures every agent submits a new_answer. The subsequent round naturally moves agents into vote-only mode.

Context

This came up during analysis of stuck-agent behavior. The self-improvement → plateau → evaluator rescue cycle works well, but without a graceful timeout, agents can be cut off before submitting any answer at all. The existing soft/hard timeout hooks handle tool blocking but don't explicitly guide agents toward submission.

Related

  • Round evaluator feedback quality improvements (implementation_guidance field)
  • Coordination workflow documentation updates

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions