Skip to content

Enhance task execution with cycle control and retry mechanisms#42

Merged
myui merged 15 commits into
mainfrom
cycle_controller_fix
Apr 4, 2026
Merged

Enhance task execution with cycle control and retry mechanisms#42
myui merged 15 commits into
mainfrom
cycle_controller_fix

Conversation

@myui

@myui myui commented Apr 3, 2026

Copy link
Copy Markdown
Collaborator

This pull request introduces two new example scripts—task_iterations.py and task_retries.py—with accompanying documentation, and refactors the core cycle and retry control logic in the workflow engine to centralize and clarify iteration and retry management. The changes make the cycle and retry mechanisms more robust and easier to use, and update the default retry behavior to be opt-in rather than opt-out.

Key changes:

New Examples and Documentation

  • Added task_iterations.py and task_retries.py example scripts, each with detailed docstrings and usage instructions, to demonstrate iterative task execution and automatic retries, respectively. The README.md was updated to include these examples, their concepts, and recommended order of study. [1] [2] [3] [4]

Cycle (Iteration) Control Refactor

  • Centralized all cycle counting and limit logic in CycleController, making it the single source of truth. Task execution context now delegates cycle-related properties and checks (like cycle_count, max_cycles, can_iterate()) to the controller. [1] [2] [3] [4] [5]
  • The cycle increment and limit check now occur at more precise points in the workflow, ensuring correct behavior for iteration tasks and preventing off-by-one errors or race conditions. [1] [2]

Retry Control Refactor

  • Introduced a RetryController (imported and instantiated in ExecutionContext) to manage retry counts and limits, mirroring the pattern used for cycles. Task execution context delegates retry-related properties to this controller. [1] [2] [3] [4]
  • Per-task max_retries are now set on the controller at execution time, ensuring correct retry behavior for each task.

Default Behavior and API Changes

  • Changed the default default_max_retries from 3 to 0, making retries opt-in rather than opt-out, which is safer and more predictable for most workflows. [1] [2]
  • Updated the API for iteration and retry to ensure that per-task settings are respected and that the correct context is always used, especially for dynamically created iteration tasks. [1] [2]

These changes improve the reliability, clarity, and usability of dynamic task iteration and retry features in the workflow engine.

@myui

myui commented Apr 3, 2026

Copy link
Copy Markdown
Collaborator Author

@claude plz review this PR.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class examples, tests, and refactors to centralize cycle (iteration) and retry control in the Graflow workflow engine via dedicated controllers, with retries now opt-in by default.

Changes:

  • Introduces CycleController-backed iteration behavior (next_iteration, cycle_count, max_cycles, can_iterate) with expanded tests.
  • Introduces RetryController and @task(max_retries=...) with expanded unit/integration tests; updates default retry behavior to default_max_retries=0.
  • Adds new example scripts and documentation for iteration and retry workflows.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
graflow/core/context.py Delegates cycle/retry state to controllers; refactors next_iteration() and per-task controller configuration.
graflow/core/engine.py Adds engine-level retry loop that re-enqueues failed tasks.
graflow/core/cycle.py Refactors cycle logic into CycleController with explicit limit checking.
graflow/core/retry.py Adds RetryController for retry budgets, counts, and last-error tracking.
graflow/core/task.py Extends TaskWrapper to carry max_cycles/max_retries through construction/pickling.
graflow/core/decorators.py Extends @task(...) API to accept max_cycles and max_retries.
graflow/exceptions.py Adds RetryLimitExceededError.
examples/07_dynamic_tasks/task_iterations.py New iteration-focused example script.
examples/07_dynamic_tasks/task_retries.py New retry-focused example script.
examples/07_dynamic_tasks/README.md Documents new examples and recommended learning order.
tests/core/test_cycle_controller.py New unit tests for CycleController.
tests/core/test_next_iteration.py New integration tests for next_iteration() behavior through the engine.
tests/core/test_cycle_controller_integration.py Converts prior script-like file into pytest assertions for integration coverage.
tests/core/test_retry_controller.py New unit tests for RetryController.
tests/core/test_retry_integration.py New integration tests validating engine retry lifecycle and context visibility.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread graflow/core/engine.py Outdated
# Exception already stored by handler, just re-raise
# Retry logic: re-enqueue task if retries remain
retry_ctrl = context.retry_controller
base_task_id = task_id # retry uses the original task_id

This comment was marked as resolved.

Comment thread examples/07_dynamic_tasks/task_iterations.py Outdated
Comment thread graflow/core/engine.py Outdated
Comment on lines 282 to 283
# No retries left — re-raise
raise exceptions.as_runtime_error(e)

This comment was marked as resolved.

@myui
myui force-pushed the cycle_controller_fix branch from 5cbdfb3 to 0c4f691 Compare April 3, 2026 08:25
@myui

myui commented Apr 4, 2026

Copy link
Copy Markdown
Collaborator Author

@claude review this PR.

@myui
myui merged commit 91c5136 into main Apr 4, 2026
3 checks passed
@myui
myui deleted the cycle_controller_fix branch April 4, 2026 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants