fix(event_loop): raise dedicated exception when encountering max toke…

[dbschmigelski]

Description

This is the first step to address #541. We indicated our plan

Strands is going to move forward with the following solution:

We will be consistent with other failure types and fail hard by default with a MaxTokensReachedException.

To allow agents to recover without terminating the event loop a new Hook event will be added. This event will be triggered when max tokens is reached where the result of the hook will determine if we proceed or of if the exception is re-thrown.

Finally, to aid customers, an implementation of the HookProvider will be added to the SDK to solve this use case in a manner which we believe will work for the majority of use cases. The implementation will do the following:

Purge the invalid ToolUse message from the messages array
Insert a new message indicating that the previous call failed with max_tokens
The idea here is that on the next iteration of the cycle, the max_tokens message will bias the agent to use or not avoid certain problematic tools.

The followup to this will be the HookEvent and HookProvider implementation. The trickiness in this CR is the question of where this, and the followup hook, should go. Some of the interesting points are

• Should we emit the trace with the old message? Should we invoke the hook for the new EventLoopException/MaxTokensReachedException before emitting this trace? Meaning if the customer recovers the message is lost?
• Should we execute the hook before adding the message to the messages array?
• Do we add a MaxTokensReachedEvent or a general EventLoopExceptionEvent, where EventLoopExceptionEvent is called when any request ending exception is encountered.

My inclination is to execute the hook before MessageAddedEvent and before the trace. Meaning the customer modifies in place then we proceed as normal. This would look like the following but is going to be raised in the followup. Docs will be updated there as well

       if elif stop_reason == "max_tokens":
            agent.hooks.invoke_callbacks(MaxTokensReachedEvent(stop_reason, message, selected_tool))
       # Add message in trace and mark the end of the stream messages trace
        stream_trace.add_message(message)
        stream_trace.end()

        # Add the response message to the conversation
        agent.messages.append(message)
        agent.hooks.invoke_callbacks(MessageAddedEvent(agent=agent, message=message))
        yield {"callback": {"message": message}}

        # Update metrics
        agent.event_loop_metrics.update_usage(usage)
        agent.event_loop_metrics.update_metrics(metrics)

        # If the model is requesting to use tools
        if stop_reason == "tool_use":
            # Handle tool execution
            events = _handle_tool_execution(
                stop_reason,
                message,
                agent=agent,
                cycle_trace=cycle_trace,
                cycle_span=cycle_span,
                cycle_start_time=cycle_start_time,
                invocation_state=invocation_state,
            )
            async for event in events:
                yield event

            return
        elif stop_reason == "max_tokens":
            raise EventLoopMaxTokensReachedException(
                (
                    "Agent has reached an unrecoverable state due to max_tokens limit. "
                    "For more information see: "
                    "https://strandsagents.com/latest/user-guide/concepts/agents/agent-loop/#maxtokensreachedexception"
                )
            )

Related Issues

#541

Documentation PR

strands-agents/docs#195

Type of Change

Bug fix

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

• I ran hatch run prepare

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.