Skip to content

Non-Retryable Error and Bulk Resume/Restart docs #474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 4, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion mint.json
Original file line number Diff line number Diff line change
@@ -1353,7 +1353,9 @@
"group": "DLQ",
"pages": [
"workflow/rest/dlq/resume",
"workflow/rest/dlq/restart"
"workflow/rest/dlq/restart",
"workflow/rest/dlq/bulk-restart",
"workflow/rest/dlq/bulk-resume"
]
}
]
9 changes: 9 additions & 0 deletions qstash/features/retry.mdx
Original file line number Diff line number Diff line change
@@ -104,3 +104,12 @@ Upstash-Retried: 0 // This is the first attempt
Upstash-Retried: 1 // This request has been sent once before and now is the second attempt
Upstash-Retried: 2 // This request has been sent twice before and now is the third attempt
```

## Non-Retryable Error

By default, QStash retries requests for any response that does not return a successful 2XX status code.
To explicitly disable retries for a given message, respond with a 489 status code and include the header `Upstash-NonRetryable-Error: true`.

When this header is present, QStash will immediately mark the message as failed and skip any further retry attempts. The message will then be forwarded to the Dead Letter Queue (DLQ) for manual review and resolution.

This mechanism is particularly useful in scenarios where retries are generally enabled but should be bypassed for specific known errors—such as invalid payloads or non-recoverable conditions.
95 changes: 95 additions & 0 deletions workflow/rest/dlq/bulk-restart.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: "Bulk Restart Workflow Runs"
description: "Restart multiple failed workflow runs in a single request."
api: "POST https://qstash.upstash.io/v2/workflows/dlq/restart"
authMethod: "bearer"
---

The bulk restart feature allows you to restart multiple failed workflow runs from the Dead Letter Queue (DLQ), using their original payloads and configurations.

You can specify individual DLQ IDs or apply filters to identify the workflow runs you want to restart.

A maximum of 50 workflow runs can be restarted per request. If more runs are available, a cursor is returned, which can be used in subsequent requests to continue the operation. When no cursor is returned, all entries have been processed.

Each restarted workflow run is assigned a new random Run ID.

## Request Parameters

<ParamField query="dlqIds" type="array">
A list of DLQ IDs corresponding to the failed workflow runs you want to restart.
</ParamField>

<ParamField query="fromDate" type="integer">
Optional. Restart workflow runs that failed on or after this unix millisecond timestamp.
</ParamField>

<ParamField query="toDate" type="integer">
Optional. Restart workflow runs that failed on or before this unix millisecond timestamp.
</ParamField>

<ParamField query="workflowUrl" type="string">
Optional. Restart workflow runs where the workflow URL matches this value.
</ParamField>

<ParamField query="workflowRunId" type="string">
Optional. Restart workflow runs matching this specific Run ID or ID prefix.
</ParamField>

<ParamField query="workflowCreatedAt" type="integer">
Optional. Restart workflow runs created at the specified unix millisecond timestamp.
</ParamField>

<ParamField header="Upstash-Flow-Control-Key" type="string">
Optional. Override the flow control key for the restarted workflows. If not provided, the original key is reused.
</ParamField>

<ParamField header="Upstash-Flow-Control-Value" type="string">
Optional. Override the flow control value for the restarted workflows. If not provided, the original value is reused.
</ParamField>

<ParamField header="Upstash-Retries" type="integer">
Optional. Override the retry configuration for the steps in the restarted workflows.
</ParamField>

## Response

<ResponseField name="cursor" type="string">
A cursor to paginate through additional matching DLQ entries. If not present, there are no more entries to process.
</ResponseField>

<ResponseField name="workflowRuns" type="array">
A list of resumed workflow runs, each containing a new run ID and creation timestamp.
</ResponseField>


## Request Example

<RequestExample>
```sh
curl -X POST https://qstash.upstash.io/v2/workflows/dlq/restart \
-H "Authorization: Bearer <token>" \
-H "Upstash-Flow-Control-Key: custom-key" \
-H "Upstash-Flow-Control-Value: parallelism=1" \
-H "Upstash-Retries: 3" \
```
</RequestExample>


<ResponseExample>
```json
{
"cursor": "",
"workflowRuns": [
{
"workflowRunId": "wfr_resumed_A",
"workflowCreatedAt": 1748527971000
},
{
"workflowRunId": "wfr_resumed_B",
"workflowCreatedAt": 1748527971000
}
]
}
```
</ResponseExample>

103 changes: 103 additions & 0 deletions workflow/rest/dlq/bulk-resume.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "Bulk Resume Workflow Runs"
description: "Resume multiple workflow runs at once"
api: "POST https://qstash.upstash.io/v2/workflows/dlq/resume"
authMethod: "bearer"
---

The bulk resume feature allows you to resume multiple failed workflow runs from the Dead Letter Queue (DLQ) in a single request, continuing each run from the point of failure rather than starting over.

This is useful when you want to preserve the progress of long-running workflows that partially succeeded before failing, and resume them all efficiently without losing successful step results.

Each resumed workflow is created as a new run. All successfully completed steps from the original runs are preserved, and only the failed or pending steps are executed again.

A maximum of 50 workflow runs can be resumed per request. If more runs are available, a cursor is returned, which can be used in subsequent requests to continue the operation. When no cursor is returned, all entries have been processed.

You can specify exact DLQ IDs or apply filters to select which workflows to resume.

<Note>
You may modify the workflow code **after the point of failure**, but changes **before the failed step** are not supported and may cause the resume to fail.

For more information, see [Handle workflow route code changes](/workflow/howto/changes).
</Note>


## Request Parameters

<ParamField query="dlqIds" type="array">
A list of DLQ IDs corresponding to the failed workflow runs you want to resume.
</ParamField>

<ParamField query="fromDate" type="integer">
Optional. Resume workflow runs that failed on or after this unix millisecond timestamp.
</ParamField>

<ParamField query="toDate" type="integer">
Optional. Resume workflow runs that failed on or before this unix millisecond timestamp.
</ParamField>

<ParamField query="workflowUrl" type="string">
Optional. Resume workflow runs where the workflow URL matches this value.
</ParamField>

<ParamField query="workflowRunId" type="string">
Optional. Resume workflow runs matching this specific Run ID or ID prefix.
</ParamField>

<ParamField query="workflowCreatedAt" type="integer">
Optional. Resume workflow runs created at the specified unix millisecond timestamp.
</ParamField>

<ParamField header="Upstash-Flow-Control-Key" type="string">
Optional. Override the flow control key for the resumed workflows. If not provided, the original key is reused.
</ParamField>

<ParamField header="Upstash-Flow-Control-Value" type="string">
Optional. Override the flow control value for the resumed workflows. If not provided, the original value is reused.
</ParamField>

<ParamField header="Upstash-Retries" type="integer">
Optional. Override the retry configuration for the steps in the resumed workflows.
</ParamField>

## Response

<ResponseField name="cursor" type="string">
A cursor to paginate through additional matching DLQ entries. If not present, all matching entries have been processed.
</ResponseField>

<ResponseField name="workflowRuns" type="array">
A list of resumed workflow runs, each containing a new run ID and creation timestamp.
</ResponseField>

## Request Example

<RequestExample>
```sh
curl -X POST https://qstash.upstash.io/v2/workflows/dlq/resume \
-H "Authorization: Bearer <token>" \
-H "Upstash-Flow-Control-Key: custom-key" \
-H "Upstash-Flow-Control-Value: parallelism=1" \
-H "Upstash-Retries: 3"
```

</RequestExample>

<ResponseExample>
```json
{
"cursor": "",
"workflowRuns": [
{
"workflowRunId": "wfr_resumed_A",
"workflowCreatedAt": 1748527971000
},
{
"workflowRunId": "wfr_resumed_B",
"workflowCreatedAt": 1748527971000
}
]
}
```
</ResponseExample>

9 changes: 3 additions & 6 deletions workflow/rest/dlq/restart.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Restart Workflow Run"
description: "Restart a failed workflow run from the beginning"
api: "POST https://qstash.upstash.io/v2/dlq/restart/{dlqId}"
api: "POST https://qstash.upstash.io/v2/workflows/dlq/restart/{dlqId}"
authMethod: "bearer"
---

@@ -11,18 +11,15 @@ The restart feature allows you to start a failed workflow completely over from t
This is useful when you want to ensure a clean execution or when the workflow failure might have been caused by corrupted state that requires a fresh start.

When you restart a workflow, completely new workflow run is created using the original workflow's initial configuration and payload.
All previous step results are discarded and the workflow executes as if it's running for the first time.a
All previous step results are discarded and the workflow executes as if it's running for the first time.

You can overwrite the original workflow's run ID, retries and flow control settings by passing the respective headers in the restart request.
You can overwrite the retries and flow control settings by passing the respective headers in the restart request.

## Request

<ParamField path="dlqId" type="string" required>
The ID of the DLQ message containing the failed workflow run
</ParamField>
<ParamField header="Upstash-Workflow-RunId" type="string">
Optional. Overwrite the workflow run ID for the restarted workflow. If not provided, a new workflow run ID will be generated.
</ParamField>
<ParamField header="Upstash-Flow-Control-Key" type="string">
Optional. Overwrite the flow control key for the restarted workflow. If not provided, the original workflow run configuration will be reused.
</ParamField>
7 changes: 2 additions & 5 deletions workflow/rest/dlq/resume.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Resume Workflow Run"
description: "Resume a failed workflow run from where its left off"
api: "GET https://qstash.upstash.io/v2/dlq/resume/{dlqId}"
api: "POST https://qstash.upstash.io/v2/workflows/dlq/resume/{dlqId}"
authMethod: "bearer"
---

@@ -10,7 +10,7 @@ The resume feature allows you to continue a failed workflow run from exactly whe

This is particularly useful for long-running workflows where you don't want to lose progress from successful steps when a single step fails.

When you resume a workflow, a fresh workflow run is created to avoid mixing failed and successful run states.
When you resume a workflow, a fresh workflow run is created.
All data from successfully executed steps is maintained.

You can overwrite the workflow's run ID, retries and flow control settings by passing the respective headers in the resume request.
@@ -27,9 +27,6 @@ You can overwrite the workflow's run ID, retries and flow control settings by pa
<ParamField path="dlqId" type="string" required>
The ID of the DLQ message containing the failed workflow run
</ParamField>
<ParamField header="Upstash-Workflow-RunId" type="string">
Optional. Overwrite the workflow run ID for the resumed workflow. If not provided, the original workflow run ID will be reused with a new timestamp.
</ParamField>
<ParamField header="Upstash-Flow-Control-Key" type="string">
Optional. Overwrite the flow control key for the resumed workflow. If not provided, the original workflow run configuration will be reused.
</ParamField>