Skip to content

Conversation

@raphael
Copy link
Member

@raphael raphael commented Dec 28, 2025

Summary

This PR introduces three related improvements to goa-ai:

1. Cursor-Based Pagination DSL

New DSL functions allow tools to declare cursor-based pagination fields:

Tool("search", "Search docs", func() {
    Args(SearchArgs)
    Return(SearchResult)
    BoundedResult(func() {
        Cursor("cursor")        // payload field
        NextCursor("next_cursor") // result field
    })
})

Contract:

  • Cursors are opaque; callers must not parse or synthesize them
  • When paging, keep all other arguments unchanged; only set the cursor field
  • Both fields must be optional Strings in their respective schemas

2. Bounded Result Shape Enforcement

BoundedResult() now enforces canonical bounds fields on tool results:

Field Type Required
returned Int
truncated Boolean
total Int optional
refinement_hint String optional

Authoring rule: Either declare none of these fields (let BoundedResult() add them) or declare all of them. Hybrid declarations are rejected at DSL eval time.

3. Provider Concurrency & Health Responsiveness

The toolregistry/provider.Serve function now uses a worker pool architecture:

  • Non-blocking subscription loop: drains the toolset stream without blocking on tool execution
  • Health ping responsiveness: pings are handled promptly even when all workers are busy
  • Configurable concurrency: MaxConcurrentToolCalls and MaxQueuedToolCalls options
  • Ack grace period: SinkAckGracePeriod for deployments with long-running tools

Additional Changes

  • Reminder injection fix: Respects Bedrock constraint that tool_use must be immediately followed by tool_result (no injecting system reminders between them)
  • Agent child result collection: Uses Await + IsReady to process child completions as they arrive rather than blocking sequentially
  • Tool result reminders: New templates for retry hints and bounds reminders (injected backstage to guide the planner)

Testing

  • Added provider tests verifying ping responsiveness while tools are in-flight
  • Added runtime tests for agent child ordering and mixed batch execution
  • All existing tests pass

Documentation

  • Updated docs/dsl.md with pagination DSL and bounds field contracts
  • Updated docs/overview.md and docs/runtime.md with bounded result details

…and provider concurrency

DSL & Codegen
- Add Cursor() and NextCursor() DSL functions inside BoundedResult() to declare
  cursor-based pagination fields
- BoundedResult() now auto-adds canonical bounds fields (returned, total,
  truncated, refinement_hint) when not already present; hybrid declarations
  are rejected at DSL eval time
- Codegen emits PagingSpec on tool specs when cursor fields are declared
- Generated ResultBounds() method now extracts next_cursor from result

Runtime
- Add NextCursor field to agent.Bounds for cursor-based pagination
- Add tool_result_reminders.go with templates for retry hints and bounds
  reminders (injected backstage to guide the planner on truncation/paging)
- Fix reminder injection: respect Bedrock constraint that tool_use must be
  immediately followed by tool_result (user message), don't inject between
- collectAgentChildResults now uses Await + IsReady to process completions
  as they arrive rather than blocking sequentially on each child

Provider
- Refactor toolregistry/provider to use a worker pool with configurable
  concurrency (MaxConcurrentToolCalls) and queue depth (MaxQueuedToolCalls)
- Provider subscription loop drains the toolset stream without blocking on
  tool execution, ensuring health pings are handled promptly
- Add SinkAckGracePeriod option for deployments with long-running tools
- Add tests verifying ping responsiveness while tools are in-flight

Documentation
- Update docs/dsl.md with pagination DSL and bounds field contracts
- Update docs/overview.md and docs/runtime.md with bounded result details
@raphael raphael enabled auto-merge (squash) December 28, 2025 20:16
@raphael raphael merged commit b77f649 into main Dec 28, 2025
2 checks passed
@raphael raphael deleted the feat/bounded-result-pagination-and-provider-concurrency branch December 28, 2025 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants