-
-
Notifications
You must be signed in to change notification settings - Fork 683
fix(ai): keep response content when a stray </think> follows a real think block #836
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
altic-dev
merged 1 commit into
altic-dev:main
from
YuriNachos:fix/strip-thinking-tags-stray-close
Aug 13, 2026
+81
−2
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
Tests/FluidDictationIntegrationTests/StripThinkingTagsTests.swift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| @testable import FluidVoice_Debug | ||
| import XCTest | ||
|
|
||
| // Regression tests for `LLMClient.stripThinkingTags`. | ||
| // | ||
| // Models that emit a proper <think>…</think> block can still leak a stray </think> | ||
| // later in their answer. The orphan pass (meant for Nemotron's no-opening-tag | ||
| // "thoughts</think>response" output) used to reclassify every character between the | ||
| // real close and the stray close as thinking, dropping it from the visible response. | ||
|
|
||
| @MainActor | ||
| final class StripThinkingTagsTests: XCTestCase { | ||
| private let client = LLMClient.shared | ||
|
|
||
| /// A stray closing tag after a real <think>…</think> block must not eat the | ||
| /// answer text that precedes it. Previously "The literal " was moved into | ||
| /// thinking and removed from the response. | ||
| func testStrayCloseAfterRealThinkBlockKeepsContent() { | ||
| let result = self.client.stripThinkingTags( | ||
| "<think>reasoning here</think>The literal </think> tag is stray." | ||
| ) | ||
|
|
||
| XCTAssertEqual(result.thinking, "reasoning here", "only the real think block is thinking") | ||
| XCTAssertTrue(result.content.contains("The literal"), "content before the stray close must stay visible") | ||
| XCTAssertTrue(result.content.contains("tag is stray"), "content after the stray close must stay visible") | ||
| } | ||
|
|
||
| /// Nemotron-style output has no opening tag and uses </think> as the separator: | ||
| /// everything before it is thinking, everything after is content. This must keep | ||
| /// working (the orphan pass is still applied when no opening tag was present). | ||
| func testOrphanCloseWithoutOpeningTagStillSplits() { | ||
| let result = self.client.stripThinkingTags("reasoning here</think>Hello world.") | ||
|
|
||
| XCTAssertEqual(result.thinking, "reasoning here") | ||
| XCTAssertEqual(result.content, "Hello world.") | ||
| } | ||
|
|
||
| /// The `<thinking>` variant of the Nemotron separator is handled the same way. | ||
| func testOrphanCloseWithoutOpeningTagSupportsLongFormTag() { | ||
| let result = self.client.stripThinkingTags("thoughts</thinking>response") | ||
|
|
||
| XCTAssertEqual(result.thinking, "thoughts") | ||
| XCTAssertEqual(result.content, "response") | ||
| } | ||
|
|
||
| /// A normal, well-formed <think>…</think> block is unchanged. | ||
| func testProperThinkBlockIsUnchanged() { | ||
| let result = self.client.stripThinkingTags("<think>reasoning here</think>Hello world.") | ||
|
|
||
| XCTAssertEqual(result.thinking, "reasoning here") | ||
| XCTAssertEqual(result.content, "Hello world.") | ||
| } | ||
|
|
||
| /// Plain content with no thinking tags at all is returned verbatim. | ||
| func testPlainContentWithNoTagsIsUnchanged() { | ||
| let result = self.client.stripThinkingTags("Just a normal response with no tags at all.") | ||
|
|
||
| XCTAssertEqual(result.thinking, "") | ||
| XCTAssertEqual(result.content, "Just a normal response with no tags at all.") | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When Nemotron-style non-streaming output uses
reasoning</think>answer, this flag now disables the orphan-separator pass if the visible answer happens to contain the literal text<think>or<thinking>anywhere. For example,reasoning</think>Explain the <think> tagis returned as visible content with the reasoning leaked at the front instead of extractingreasoning, because there was no real opening tag before the separator—only tag text in the answer. Consider basing this guard on whether the proper-pair regex actually removed a real block before skipping orphan parsing.Useful? React with 👍 / 👎.