fix: configurable timeout for AI requests (#6497) #7267
Merged
+68
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add AI_REQUEST_TIMEOUT_SECONDS environment variable (default 3600s) to fix timeout issues with slow AI models like self-hosted Ollama. Previously hardcoded at 300 seconds, causing legitimate long-running requests to fail. Fixes windmill-labs#6497
devdattatalele
requested review from
alpetric,
hugocasa and
rubenfiszel
as code owners
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA |
Add comprehensive documentation about reverse proxy timeout requirements. Without proper NGINX/proxy configuration, connections will still timeout at the proxy layer regardless of backend timeout settings. Enhanced documentation includes: - CRITICAL warning about proxy configuration requirement - Example NGINX configuration snippet - Explanation of proxy vs backend timeout interaction This addresses the root cause in issue windmill-labs#6497 where logs showed "upstream prematurely closed connection" indicating proxy-level timeout. Part of windmill-labs#6497
Contributor
|
Thanks! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #6497
The AI chat was timing out after 5 minutes for slow models like self-hosted Ollama. This was caused by a hardcoded 300-second timeout in the HTTP client.
This PR makes the timeout configurable via AI_REQUEST_TIMEOUT_SECONDS environment variable with a default of 1 hour. Values outside the valid range (1-86400 seconds) fall back to the default with a warning.
Changes:
Added configurable timeout in backend/windmill-api/src/ai.rs
Registered env var in backend/windmill-common/src/global_settings.rs
Replaced magic numbers with named constants
Improved error handling and logging
Note for deployment: If using NGINX or similar proxy, ensure proxy_read_timeout is set to at least the same value as the AI timeout to avoid premature disconnections.
Testing: Validated timeout parsing logic for valid/invalid inputs and confirmed backward compatibility.