Skip to content

Upgrade query-generator SQL step to Claude Sonnet 5 #49187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
allenhouchins wants to merge 2 commits into
base: main
Choose a base branch
from
+15 −3
Open

Upgrade query-generator SQL step to Claude Sonnet 5 #49187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d275d54
Upgrade query-generator SQL step to Claude Sonnet 5
allenhouchins Jul 13, 2026
f404a7b
Fix systemPrompt not being passed to the SQL-generation call
allenhouchins Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion website/api/controllers/query-generator/get-llm-generated-sql.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,9 @@ module.exports = {
"couldNotGenerateQueries": true
}`;

let sqlReport = await sails.helpers.ai.prompt.with({prompt:sqlPrompt, baseModel:'claude-sonnet-4-6', expectJson: true, systemPromptForQueryGeneration})
// Effort is set to "low" because the schema was already narrowed down to relevant tables by the
// schema-filtration step above -- the model doesn't need to spend much effort re-deriving that context.
let sqlReport = await sails.helpers.ai.prompt.with({prompt:sqlPrompt, baseModel:'claude-sonnet-5', expectJson: true, systemPrompt: systemPromptForQueryGeneration, effort: 'low'})
.intercept((err)=>{
Comment on lines +176 to 179
if(this.req.isSocket){
// If this request was from a socket and an error occurs, broadcast an 'error' event and unsubscribe the socket from this room.
Expand Down
14 changes: 12 additions & 2 deletions website/api/helpers/ai/prompt.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ module.exports = {
},
expectJson: { type: 'boolean', defaultsTo: false },
systemPrompt: { type: 'string', example: 'Here is data about each computer, as JSON: ```[ … ]```' },
effort: {
type: 'string',
description: 'Optional effort level for adaptive thinking (controls thinking depth vs. token/latency cost). Only supported on Anthropic models with output_config.effort support (e.g. Claude Sonnet 5, Claude Opus 4.6+). Ignored for models that don\'t support it (e.g. Claude Haiku 4.5, OpenAI models).',
example: 'low'
},
},


Expand All @@ -54,7 +59,7 @@ module.exports = {
},


fn: async function ({prompt, baseModel, expectJson, systemPrompt}) {
fn: async function ({prompt, baseModel, expectJson, systemPrompt, effort}) {

// TODO: Write a comprehensive test suite that prompts hundreds of times in parallel to see which combo
// of JSON prompt suffix + base model works the best, through actual experimentation. Then document
Expand All @@ -79,14 +84,19 @@ Please do not add any text outside of the JSON or wrap it in a code fence. Neve

let requestData = {
model: baseModel,
max_tokens: 4096,// eslint-disable-line camelcase
// Bumped from 4096 so that models with adaptive thinking on by default (e.g. Claude Sonnet 5)
// have enough headroom for thinking tokens without truncating the actual response.
max_tokens: 8192,// eslint-disable-line camelcase
messages: [
{ role: 'user', content: prompt+(expectJson? JSON_PROMPT_SUFFIX : '') }
]
};
if (systemPrompt) {
requestData.system = systemPrompt;
}
if (effort) {
requestData.output_config = { effort };// eslint-disable-line camelcase
}
Comment on lines +87 to +99

@coderabbitai coderabbitai Bot Jul 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Map the target file and nearby Anthropic helpers
ast-grep outline website/api/helpers/ai/prompt.js --view expanded || true
rg -n "anthropicResponse\.content|content\[0\]\.text|output_config|effort|thinking|type === 'text'|text block" website/api/helpers/ai -S || true

# Read the target file around the cited lines
wc -l website/api/helpers/ai/prompt.js
sed -n '1,220p' website/api/helpers/ai/prompt.js

# Find any other Anthropic response parsing in the repo
rg -n "content\[0\]\.text|content\.find\(\(block\) => block\.type === 'text'\)|content\.find\(|block\.type === 'text'" -S .

Repository: fleetdm/fleet

Length of output: 9216

🌐 Web query:

Anthropic Messages API response content blocks thinking text block adaptive thinking output_config.effort Sonnet 5

💡 Result:

In the Anthropic Messages API, adaptive thinking is a mechanism for Claude models to dynamically determine the depth of their reasoning based on query complexity [1][2]. For models supporting this, including Claude Sonnet 5, it replaces the legacy budget_tokens configuration [3][2]. To configure adaptive thinking, you use the thinking object in the request body with type: "adaptive" [1][2]. The output_config.effort parameter is used to provide soft guidance on the desired thinking depth [4][1]. Key details: 1. Configuration Structure: The effort parameter must be placed within a dedicated output_config object in the request body, not inside the thinking object [4]. 2. Effort Levels: The effort parameter supports values such as low, medium (default), high, and xhigh (availability varies by model) [1]. This allows you to influence how much reasoning Claude applies to a task [1]. 3. Model Support: On models like Claude Sonnet 5, adaptive thinking is enabled by default [2]. You can explicitly disable it by passing thinking: {type: "disabled"} if preferred [2]. Attempting to use the legacy manual thinking: {type: "enabled", budget_tokens: N} configuration on these models will typically result in a 400 error [3][2]. 4. Displaying Thinking: On newer models, the thinking block content defaults to "omitted" [3][5]. If you wish to see the reasoning text, you must explicitly set thinking: {type: "adaptive", display: "summarized"} [5]. 5. Usage: Use max_tokens to set a hard limit on the total output (thinking plus response text) [1]. Adaptive thinking, paired with effort, is the recommended approach for managing reasoning depth on supported models [3][2].

Citations:

🌐 Web query:

Anthropic Messages API response content array text thinking blocks default omitted content[0].text

💡 Result:

In the Anthropic Messages API, when extended thinking is enabled, the API response contains an array of content blocks [1]. You should not assume that the first block (response.content) is a text block, as thinking blocks are distinct objects that appear in the content array [1]. Regarding the behavior of thinking content in the response: 1. Thinking Content Types: When extended thinking is enabled, the model returns specialized content blocks of type thinking or redacted_thinking, which are distinct from text blocks [1][2]. 2. Omission and Display: Whether the actual thinking text is returned depends on the configuration provided in the request's thinking parameter [3]. - If configured to return full reasoning, the thinking content is provided in the thinking block [4][5]. - If the configuration (often via an adaptive thinking or display setting) results in the content being omitted or summarized, the thinking block may be present but contain no text, or it may be omitted entirely depending on the specific API version and model behavior [3][6][1]. 3. Robust Handling: Because the structure of the content array varies based on model output and configuration, you must iterate over the response.content array and check the type of each block (e.g., if block.type == "text" or if block.type == "thinking") rather than directly accessing response.content.text [1][5][2]. Attempting to access.text on a thinking block or an unexpected block type will cause an error [1].

Citations:

Extract the text block from Anthropic responses

anthropicResponse.content[0] is not guaranteed to be a text block here; with adaptive thinking enabled, the first block can be thinking/redacted_thinking, which makes rawPromptResponse undefined and breaks JSON parsing. Scan content for the first type === 'text' block and throw if none is present.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@website/api/helpers/ai/prompt.js` around lines 87 - 99, Update the Anthropic
response handling that derives rawPromptResponse from anthropicResponse.content
to scan for the first block whose type is "text" rather than assuming content[0]
is text. Throw an explicit error when no text block exists, preserving the
existing downstream parsing flow for valid text responses.

Comment on lines +97 to +99

@coderabbitai coderabbitai Bot Jul 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the target file around the reported lines
sed -n '1,220p' website/api/helpers/ai/prompt.js

# Search for effort/output_config/model gating logic
rg -n "output_config|effort|claude-|Haiku|supports.*effort|unsupported" website/api -g '!**/node_modules/**'

# Find any model capability allowlists or validators
rg -n "model.*(allow|support|validate|capab|feature)|effort.*(allow|support|validate)|claude" website -g '!**/node_modules/**'

Repository: fleetdm/fleet

Length of output: 50373

🌐 Web query:

Anthropic output_config effort unsupported models ignored or error Claude Haiku 4.5

💡 Result:

The error "output_config: Extra inputs are not permitted" occurs when an Anthropic API request includes the output_config parameter (used for structured outputs or reasoning effort) while using an integration or provider that does not support it [1][2][3]. Key reasons for this error include: 1. Provider Restrictions: Cloud providers like Amazon Bedrock, Google Vertex AI, and Azure AI Foundry often have specific request-body requirements [1][2][4][3]. If these services do not recognize the output_config field, they reject the entire request with an "Extra inputs are not permitted" or similar validation error [2][3]. 2. Model Capability Mismatches: While Claude Haiku 4.5 supports structured outputs in many environments, it may not support all features within output_config (such as effort) depending on the specific model endpoint or deployment (e.g., via GitHub Copilot or specific proxy services) [4][5][6][7]. 3. Integration Errors: Many tools and SDKs automatically inject output_config into requests (e.g., for automated session title generation or structured output enforcement) [2][3]. If the underlying service (like Vertex AI or Bedrock) has not been updated to accept this parameter, the integration will fail [2][3]. To resolve this issue: - Strip the output_config parameter from the request body before sending it to the provider if the provider's documentation confirms it is unsupported [1][4][6]. - If you are using an integration tool (like Claude Code or a proxy), ensure you are on the latest version, as many of these issues are resolved by adding request transformers that selectively strip unsupported fields [2][3][7]. - For structured outputs on restricted providers, rely on system prompt instructions to enforce the output format instead of using the output_config API parameter [2][3]. For detailed configuration, refer to the provider-specific documentation for Amazon Bedrock or Google Vertex AI, as their support for these parameters can differ from the standard Anthropic API [8][9][10].

Citations:

🌐 Web query:

site:platform.claude.com effort Claude Haiku 4.5 output_config ignored unsupported models

💡 Result:

Based on current Claude Platform documentation, Claude Haiku 4.5 does not support the output_config.effort parameter [1]. In the Claude API, unsupported fields or parameters sent to a model are generally handled in one of two ways: they are either silently ignored [2] or they trigger a validation error [3][4]. When specific parameters like output_config.effort (which is typically reserved for models that support adaptive thinking, such as Claude Opus 4.8 or Claude Fable 5 [4][5]) are passed to a model that does not support them, the API may reject the request with a 400-series error [4]. If you are encountering issues, ensure your request parameters align with the capabilities of Claude Haiku 4.5: 1. Adaptive thinking and the output_config.effort parameter are not supported on Claude Haiku 4.5 [1]. 2. For structured outputs, the API uses output_config.format [6][4]. Ensure you are not conflating output_config.effort with other output_config sub-parameters. 3. If you are using a library (such as an OpenAI-compatible SDK), note that some libraries may attempt to map parameters automatically, which can lead to unexpected errors if the underlying model does not support them [2]. You should check your API request payload to ensure that the output_config object only contains fields valid for the model you are using.

Citations:

Gate effort by model. Claude Haiku 4.5 does not accept output_config.effort, so this helper should only attach it for supported Anthropic models (or reject unsupported baseModel/effort combinations). Otherwise a claude-haiku-* caller can get a 4xx from Anthropic.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@website/api/helpers/ai/prompt.js` around lines 97 - 99, Update the effort
handling in the prompt helper so output_config.effort is attached only when
baseModel identifies a supported Anthropic model; ensure claude-haiku-* requests
omit it or reject the unsupported baseModel/effort combination before sending
the request.


let anthropicResponse = await sails.helpers.http.post('https://api.anthropic.com/v1/messages', requestData, {
'x-api-key': sails.config.custom.anthropicSecret,
Expand Down
Loading