fix: surface real cause when outfit suggestion AI response is truncated (#139)#142
Open
AnayGarodia wants to merge 1 commit into
Open
fix: surface real cause when outfit suggestion AI response is truncated (#139)#142AnayGarodia wants to merge 1 commit into
AnayGarodia wants to merge 1 commit into
Conversation
…ncated Fixes Anyesh#139. The user's attached backend/LM Studio logs show the actual failure: the model (a reasoning-capable Qwen3 build) put its whole chain-of-thought in `reasoning_content`, exhausted the completion token budget doing so, and returned `content: ""` with `finish_reason: "length"`. `_parse_ai_response` then failed to parse the empty string as JSON, and that failure was swallowed by a broad `except Exception` into the generic "AI service is not available. Please check your AI endpoint configuration in Settings." — which is misleading, since the endpoint responded successfully and is correctly configured; it just never emitted any output content. `AIService.generate_text()` now detects this empty-content/finish_reason case right after extracting the response and raises a new `AIResponseTruncatedError` with the specific cause (reasoning consumed the budget vs. some other truncation) and an actionable hint (raise AI_MAX_TOKENS or disable extended thinking for the model). It retries across the existing retry loop before giving up, since the failure is non-deterministic (depends on how long the model reasons). In `RecommendationService.generate_recommendation`, this error type is now caught explicitly and re-raised with its original, specific message instead of being caught by the generic fallback handler. Reproduced with a unit test built from the exact response payload shape in the issue's attached logs (empty content + reasoning_content + finish_reason "length"); also confirmed the app's error text is not a literal "An error has occured" string anywhere in the codebase, so the issue title is the reporter's paraphrase rather than a literal typo to fix. pairing_service.py has an analogous parsing/error-swallowing pattern but is intentionally left untouched here — it's a separate feature (see Anyesh#140) and out of scope for this fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #139.
The issue's attached backend + LM Studio logs pin down the actual failure
mechanism (I did not have an equivalent AI endpoint to reproduce the full
end-to-end flow, but the logs contain the exact API response shape, which
is enough to reproduce the failure precisely in a unit test — see below):
models return their chain-of-thought in a separate
reasoning_contentfield, distinct from
content.reasoning_contentand never got to writingcontent. The responsecame back with
content: "",finish_reason: "length".RecommendationService._parse_ai_responsethen fails to parse""asJSON (
Could not parse AI response as JSON:— matches the log lineexactly, note the empty string after the colon).
ValueErroris caught by a broadexcept Exceptioningenerate_recommendationand replaced with a generic, misleadingmessage: "AI service is not available. Please check your AI endpoint
configuration in Settings." The AI service was available and was
correctly configured — it just didn't produce any output content.
I also grepped the whole codebase for the literal string "An error has
occured"/"occurred" — it does not exist anywhere in the frontend. The
issue title/body is the reporter's own paraphrase of what they saw, not a
literal string bug, so there's nothing to fix there.
Fix
AIService.generate_text()now checks for empty/blankcontentrightafter extracting the API response. If found, it inspects
finish_reasonand
reasoning_contentto build a specific, actionable error(
AIResponseTruncatedError), e.g. "returned an empty response: itsreasoning/thinking output consumed the entire completion token budget
before it produced a response... Try raising AI_MAX_TOKENS (currently
8000) or disabling extended thinking/reasoning mode for this model."
loop — it retries (up to
AI_MAX_RETRIES) before giving up, sincewhether the model finishes reasoning in time is non-deterministic.
RecommendationService.generate_recommendationnow catchesAIResponseTruncatedErrorexplicitly and re-raises it with its ownspecific message, instead of letting the generic
except Exceptionhandler stomp it with the "AI service is not available" text.
Scope
This is deliberately scoped to the outfit-suggestion path
(
ai_service.py/recommendation_service.py), which is what #139reports.
pairing_service.pyhas the same_parse_ai_response/generic-error pattern and likely has the same latent issue, but that's a
separate code path (see #140, which is about pairings UI copy, not this
bug) — left out of this PR to keep it minimal and reviewable. Happy to
follow up there if maintainers want it.
I was not able to fully reproduce the end-to-end flow (would need a local
reasoning-model endpoint like the reporter's LM Studio + Qwen3 setup), but
I reproduced the exact failure mechanism with a unit test built from the
literal response payload shape captured in the issue's attached logs, and
verified the fix resolves it while leaving normal responses unaffected.
Testing
TestGenerateTextTruncatedResponseinbackend/tests/test_ai_service.py:content+reasoning_content+finish_reason: "length"→raises
AIResponseTruncatedErrormentioning the reasoning cause andAI_MAX_TOKENScontentwithoutreasoning_content→ still raisesAIResponseTruncatedError(generic truncation)contentresponse → unaffected, still returnscontent
docker compose up postgres redis):386 passed.ruff checkandruff format --checkpass on all changed files.Checklist