feat: Make max iteration limit resumable without reminders

[utkarsh-in]

#2406

Summary

• Add MAX_ITERATIONS_REACHED status to ConversationExecutionStatus enum
• Mark MAX_ITERATIONS_REACHED as terminal state
• Add ConversationIterationLimitEvent for better error handling
• Modify send_message() to reset MAX_ITERATIONS_REACHED to IDLE on new user input
• Modify run() to allow restarting from MAX_ITERATIONS_REACHED state
• Add budget information to agent system prompt
• Add budget warning messages at 80% and 95% of max iterations
• Update tests to cover new status transitions

This change allows conversations to resume after hitting max iterations when a new user message is sent, instead of permanently stopping.

Checklist

• If the PR is changing/adding functionality, are there tests to reflect this?
• If there is an example, have you run the example to make sure that it works?
• If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
• If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
• Is the github CI passing?

[enyst]

To note, I don't believe we've ever done this: "begin wrapping up and provide... etc". The reminder never changed, it was only informative e.g. "you're at iteration m out of n".

[enyst]

I actually sent OpenHands to verify what I said, and here is its response. Oh, it wasn't even applied for tool calling, meaning it wasn't applied for ~SOTA LLMs since ~October 2024 or so.

I checked the legacy V0 CodeAct code.

Short answer: no, the live reminder did not push the agent to finish faster. The actual injected text was just the remaining-turn count plus a finish-format hint.

What I found:

• The helper that formats the reminder is in:
  • openhands/utils/prompt.py:136-150
• Its text is:
  • ENVIRONMENT REMINDER: You have {N} turns left to complete the task. When finished reply with <finish></finish>.

Historically, that reminder was actually injected from CodeAct here:

• older CodeActAgent._get_messages() had:
  • if not self.function_calling_active: self.prompt_manager.add_turns_left_reminder(messages, state)

So in the old CodeAct path, it only applied in the non-function-calling mode.

Did it ever say more than just the number?

• In commit e27a5ef75 (“In-context reminder”), add_turns_left_reminder() was temporarily expanded to also prepend a workflow reminder every 10 steps.
• But that extra text was things like:
  • follow the workflow
  • test thoroughly
  • hidden tests exist
  • quality/completeness matter more than brevity

So even there, it was not “hurry up” or “finish faster” language. If anything, it nudged toward more thoroughness, not speed.

Important

That extra workflow-reminder version appears to have been in dead code by then:

• the call site that actually injected add_turns_left_reminder() was removed earlier in commit 07f0d1ccb
• after that, the helper still existed, but I did not find a live call to it in CodeAct

Net conclusion

• Live old CodeAct injection: just “you have N turns left … when finished reply with <finish></finish>”
• No live urgency/push-to-finish-faster wording
• One later dead-code variant added extra reminder text, but it pushed for thoroughness, not speed

[utkarsh-in]

Resolved in 37e697b

[enyst]

Thank you for trying this! It's very interesting.

I do wonder about one or two things though: first, benchmarks like any non-interactive runs, really do stop at max_iterations, they have to, so that it doesn't go on forever. But interactive runs don't have to stop, they only need to pause: so the user via an UI can say something, and it will continue. That might mean that the user experience could be affected if the LLM is told that it'd better hurry, instead of running normally.

So idk, I think maybe it's worth asking: do we want this applied only to non-interactive runs? do we know we are in a non-interactive run?

Secondly, I'm a bit curious. I'd like to know what in-context reminders tell the LLM in other agentic tools. I seem to recall off-hand that OpenAI had a reminder since a long time ago at least.

Reminding it of fragments of the system prompt or instructions, like telling it again how the process works, is fine IMHO; telling the state of its runtime, like number of iterations remaining is fine; idk, WDYT, is hurrying it or nudging it to completion fine?

[utkarsh-in]

Hi @enyst I agree that nudging it to completion can be removed.
I think we should have number of iterations for both interactive and non-interactive runs to reach a logical conclusion in the current run, even if we continue further.

[enyst]

@VascoSch92 I'd love your thoughts on this discussion.

[enyst]

I think the agent did this? I don't think it's the right fix, only release PRs should bump versions. I think updating from main branch will resolve the issue that apparently needed this

[VascoSch92]

After the discussion in the relevant issue, I think we don't want to have a reminder for the iteration limit. At least, not now.

However, I think it would be interesting (at least for me, for subagents) to have a new error type, i.e., MAX_ITERATIONS_REACHED, because right now this type of error falls under the generic ERROR category, which makes it indistinguishable.

Can we just make this change in the PR?

Reminding it of fragments of the system prompt or instructions — like telling it again how the process works — is fine IMHO; telling it the state of its runtime, like the number of remaining iterations, is fine too. I don't know — is nudging it or hurrying it toward completion fine? WDYT?

I have already used context budget pressure on agents, but it is difficult to say whether it was effective or not. I think it is effective when you have a very limited number of iterations, but that is actually not our case, at least so far, I have never seen a task fail due to an iteration overflow in a benchmark using the OpenHands agent.

[utkarsh-in]

@VascoSch92 @enyst just to clarify

• Keep ConversationIterationLimitEvent
• Keep it as resumable
• Keep Budget Info in agent
• remove 85% and 95% reminder

Is that correct?