Required optimization on local review with claude cli

[selvarasu-g]

We have noticed that running Sashiko’s local review with the Claude CLI takes considerably longer than expected. The delay hurts productivity, especially when we only need to review minimal code changes. Are there any techniques for reducing the overall review time when using the Claude CLI?.

[derekbarbosa]

I've heard of similar complaints from others, unfortunately.

Based on your comments in #183 - I wonder if this is just a consequence of model availability/speed and other variables (interactions, "thinking time", turns in the conversation, tasks in each review stage, etc). It all could certainly compound and effect the speed.

Could you experiment with decreasing the size of max_interactions in Settings.toml?
The default value is max_interactions = 50.

on a related note:
@rgushchin maybe with #188 , we could define a "light" set of prompts for faster reviews?

[rgushchin]

We can in theory just squash 1-7 into a single stage if the user wants something fast. It will 100% affect the quality of reviews, so I’m not sure how realistically practical it is. I guess users want fast and good, not fast and bad. For the latter a use can also use a faster (and usually cheaper) model. Actually it might be a better idea: at least with Gemini Flash is almost as good in finding real bugs, but it’s also much better at hallucinating non-existing problems. But it also has a cost, just different one.

[rgushchin]

@selvarasu-g I’m not sure what your expectations are based on and I can’t speak for Claude, but with Gemini a single “big” review is between a 1/2 hour and an hour. I’d suggest to look into full logs and checks if something is off, e.g. if there are failed stages, retries, loops, etc.

[selvarasu-g]

@selvarasu-g I’m not sure what your expectations are based on and I can’t speak for Claude, but with Gemini a single “big” review is between a 1/2 hour and an hour. I’d suggest to look into full logs and checks if something is off, e.g. if there are failed stages, retries, loops, etc.

Hi @rgushchin , Thank you for your feedback. Our goal is to accelerate or at least reduce the overall LLM review effort for a local repository while preserving review quality. I concur that a single bit review typically takes between half an hour and an hour, which matches the experience with claude‑cli. However, the process is currently unstable; it often fails because of the JSON‑format issue (see #183 ) and LLM time‑out errors.

[anders-heimer]

Hi @selvarasu-g,

I share the same observations as you, I have a created a patch set that improved my review times when running kiro-cli and OLLAMA locally: #176 . It might help you as well.

Based on you log in #183, I am seeing the same issue. When running with a less capable model, it consistently fails to produce the required {"concerns": [...]} JSON structure, instead emitting tiny 11-90 token fragments that aren't valid JSON. Sashiko relies solely on prompt instructions to elicit structured output from a model too weak to follow them reliably. I am working on an RFC for this.

[selvarasu-g]

Hi @selvarasu-g,

I share the same observations as you, I have a created a patch set that improved my review times when running kiro-cli and OLLAMA locally: #176 . It might help you as well.

Based on you log in #183, I am seeing the same issue. When running with a less capable model, it consistently fails to produce the required {"concerns": [...]} JSON structure, instead emitting tiny 11-90 token fragments that aren't valid JSON. Sashiko relies solely on prompt instructions to elicit structured output from a model too weak to follow them reliably. I am working on an RFC for this.

Hi @anders-heimer , Thanks for your update. could you please share the RFC once its ready? I will do verification .

[selvarasu-g]

Hi @selvarasu-g,
I share the same observations as you, I have a created a patch set that improved my review times when running kiro-cli and OLLAMA locally: #176 . It might help you as well.
Based on you log in #183, I am seeing the same issue. When running with a less capable model, it consistently fails to produce the required {"concerns": [...]} JSON structure, instead emitting tiny 11-90 token fragments that aren't valid JSON. Sashiko relies solely on prompt instructions to elicit structured output from a model too weak to follow them reliably. I am working on an RFC for this.

Hi @anders-heimer , Thanks for your update. could you please share the RFC once its ready? I will do verification .

Hi @anders-heimer , Do you get chance to work on mentioned RFC?

[anders-heimer]

Hi @selvarasu-g ,

Please take a look at #235.

The Ollama/OpenAI-compatible backend works as such, but I ran into issues when using a less capable local model compared to frontier models. For my setup I had to tune max_input_tokens and context_window_size to fit both the model and GPU.

The RFC branch adds bounded local-model behavior: prompt preflight, schema-only finalization, typed budget failures, capped exploration, and minimal fallback when the full staged review does not fit.

Recommendation: if possible, test first with the same model/configuration before comparing other local models. That gives us a stable baseline before tuning for different model behavior.

I tested with qwen3-coder-30b-a3b-128k-t0 on a L40S card.

Br, Anders

[selvarasu-g]

Hi @selvarasu-g ,

Please take a look at #235.

The Ollama/OpenAI-compatible backend works as such, but I ran into issues when using a less capable local model compared to frontier models. For my setup I had to tune max_input_tokens and context_window_size to fit both the model and GPU.

The RFC branch adds bounded local-model behavior: prompt preflight, schema-only finalization, typed budget failures, capped exploration, and minimal fallback when the full staged review does not fit.

Recommendation: if possible, test first with the same model/configuration before comparing other local models. That gives us a stable baseline before tuning for different model behavior.

I tested with qwen3-coder-30b-a3b-128k-t0 on a L40S card.

Br, Anders

Hi @anders-heimer , Thanks for sharing the RFC. it’s an improvement over the previous version. However, I’m still encountering a JSON problem, and the Claude CLI repeatedly times out. The logs display:

026-06-04T09:31:50.177481Z INFO sashiko::reviewer: [review-bin] stage 8 deduplication failed on retry too: JSON parse error: trailing characters at line 1 column 27823
2026-06-04T09:31:50.177507Z INFO sashiko::reviewer: [review-bin] Stage 8 did not produce valid schema output; preserving all 35 concern(s) without deduplication
2026-06-04T09:31:50.187523Z INFO sashiko::reviewer: [review-bin] Running Stage 9 (Verification)
2026-06-04T09:32:42.594355Z INFO sashiko::reviewer: [review-bin] Stage 4 AI execution failed (inner attempt 1/3): Remote AI Error: claude CLI timed out after 10 minutes
2026-06-04T09:32:42.597285Z INFO sashiko::reviewer: [review-bin] Stage 1 AI execution failed (inner attempt 1/3): Remote AI Error: claude CLI timed out after 10 minutes
2026-06-04T09:32:42.663923Z INFO sashiko::reviewer: [review-bin] Stage 3 AI execution failed (inner attempt 1/3): Remote AI Error: claude CLI timed out after 10 minutes
2026-06-04T09:32:42.702565Z INFO sashiko::reviewer: [review-bin] Stage 2 AI execution failed (inner attempt 1/3): Remote AI Error: claude CLI timed out after 10 minutes

Are the tuning flags max_input_tokens and context_window_size applicable when using claude_cli? . The reason for aksing this adjusting these flags doesn’t seem to make a difference with claude cli.
We expecting that lowering max_input_tokens and context_window_size would shorten the model’s thinking time or depth of analysis, at least enough for a high‑level review. Is this correct?

[anders-heimer]

Hi Selvaru,

Are the tuning flags max_input_tokens and context_window_size applicable when using claude_cli? . The reason for aksing this adjusting these flags doesn’t seem to make a difference with claude cli.
We expecting that lowering max_input_tokens and context_window_size would shorten the model’s thinking time or depth of analysis, at least enough for a high‑level review. Is this correct?

You are right: max_input_tokens / context_window_size affect whether Sashiko sends the full prompt or takes a fallback path, but they do not directly make Claude CLI emit cleaner JSON. My earlier comment was missing that context.

For Claude CLI, #235 may help more with failure containment than producing valid JSON more reliably. The
relevant provider-independent pieces are:

• schema-only finalization, if the Claude path reaches that shared finalization flow
• central truncation detection, if claude_cli maps truncation into AiResponse.truncated
• prompt preflight and typed budget failures
• minimal fallback when the full staged review does not fit
• Stage 8 bad-JSON fallback: preserve all concerns and continue
• provider-failure / recitation retry where wired through the shared worker path
• duplicate tool-call guard / capped exploration where bounded mode is enabled

The structured-output work is intentionally OpenAI-compatible specific to keep the RFC less intrusive. That was also where I saw the strongest JSON-format improvement in my Ollama/Qwen testing.

It seem that the testing does not fully carry over between providers: the shared worker-side behavior may help, but JSON reliability and timeout behavior might need to be validated per backend.

Br, Anders