Add async dataset runner for concurrent task execution #308
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds run_dataset_async to fan out run_single_task calls with an asyncio concurrency cap.
Supports group_size runs per task and returns a flat list of Trace results.
Includes focused tests for the async runner’s behavior and grouping.
Note
Medium Risk
Introduces new concurrency and error-handling behavior around task execution; failures are now captured per-task and returned as error
Traces, so ordering/throughput and partial-failure semantics should be validated.Overview
Adds
run_dataset_asyncas a new public API to execute datasets by concurrently fanning outrun_single_taskcalls with anasynciosemaphore cap, optionalgroup_sizerepeats, and deterministic result ordering.Updates the datasets package exports to include
run_dataset_async, and adds tests covering empty input validation andgroup_sizerepetition/trace naming.Written by Cursor Bugbot for commit f6ff476. This will update automatically on new commits. Configure here.