feat(llm): add first-class llmtr/ provider prefix for the LLMTR gateway - #1148
feat(llm): add first-class llmtr/ provider prefix for the LLMTR gateway#1148knowhycodata wants to merge 2 commits into
Conversation
Make LLMTR (https://llmtr.com) a first-class option alongside OpenRouter: users set STRIX_LLM="llmtr/<model>" + LLM_API_KEY (llmtr-...) and nothing else. LLMTR is a Turkey-hosted OpenAI-compatible gateway, so StrixProvider resolves llmtr/<model> to LiteLLM's openai/<model> route, and _configure_llmtr_routing supplies the gateway base URL (an explicit LLM_API_BASE still wins) plus Strix attribution headers. Attribution is also sent per-request via _request_headers, mirroring OpenRouter. Adds unit tests for prefix detection, routing resolution, base-URL/header configuration, explicit-base override, the non-LLMTR no-op path, and frontier-model detection through the llmtr/ prefix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ag2XdAsvrR7XKMmQsAjqSN
Greptile SummaryThis PR adds
Confidence Score: 3/5The PR should not merge until mixed main/deduplication model routing and reasoning capability detection correctly account for the new LLMTR prefix. Process-global endpoint selection is derived from the main model even when deduplication uses another model, and LLMTR-prefixed reasoning models silently lose configured reasoning effort because capability detection never removes the new prefix. Files Needing Attention: strix/config/models.py and the deduplication routing coverage in tests/test_models.py Important Files Changed
Prompt To Fix All With AI### Issue 1
strix/config/models.py:568
**Global routing ignores dedupe model**
When the main and deduplication models use different providers and exactly one uses `llmtr/`, `_configure_llmtr_routing` configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.
### Issue 2
strix/config/models.py:469-473
**LLMTR prefix bypasses reasoning detection**
When an LLMTR-hosted reasoning model is used with `STRIX_REASONING_EFFORT`, capability detection examines the original `llmtr/<provider>/<model>` name without removing the new prefix, so `make_model_settings` silently omits the configured reasoning effort.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "feat(llm): add first-class llmtr/ provid..." | Re-trigger Greptile |
| return | ||
| _configure_litellm_compatibility() | ||
| _configure_openrouter_attribution(llm.model) | ||
| _configure_llmtr_routing(llm.model, llm.api_base) |
There was a problem hiding this comment.
Global routing ignores dedupe model
When the main and deduplication models use different providers and exactly one uses llmtr/, _configure_llmtr_routing configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.
Knowledge Base Used: Configuration and Telemetry
Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/config/models.py
Line: 568
Comment:
**Global routing ignores dedupe model**
When the main and deduplication models use different providers and exactly one uses `llmtr/`, `_configure_llmtr_routing` configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.
**Knowledge Base Used:** [Configuration and Telemetry](https://app.greptile.com/strix-org-3/-/custom-context/knowledge-base/usestrix/strix/-/docs/telemetry-and-config.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| if prefix == "llmtr" and stripped_model_name: | ||
| # LLMTR is an OpenAI-compatible gateway, so route llmtr/<model> | ||
| # through LiteLLM's openai/ provider. The gateway base URL and | ||
| # attribution headers are configured in _configure_llmtr_routing. | ||
| return self._get_fallback_provider("litellm"), f"openai/{stripped_model_name}" |
There was a problem hiding this comment.
LLMTR prefix bypasses reasoning detection
When an LLMTR-hosted reasoning model is used with STRIX_REASONING_EFFORT, capability detection examines the original llmtr/<provider>/<model> name without removing the new prefix, so make_model_settings silently omits the configured reasoning effort.
Knowledge Base Used: Configuration and Telemetry
Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/config/models.py
Line: 469-473
Comment:
**LLMTR prefix bypasses reasoning detection**
When an LLMTR-hosted reasoning model is used with `STRIX_REASONING_EFFORT`, capability detection examines the original `llmtr/<provider>/<model>` name without removing the new prefix, so `make_model_settings` silently omits the configured reasoning effort.
**Knowledge Base Used:** [Configuration and Telemetry](https://app.greptile.com/strix-org-3/-/custom-context/knowledge-base/usestrix/strix/-/docs/telemetry-and-config.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.When the dedupe model uses llmtr/ but the main model is a different provider, the global litellm.api_base (derived from the main model) is not the LLMTR gateway, so the dedupe request would resolve to LiteLLM's default openai/ endpoint and fail. Mirror the existing per-call dedupe credential pattern: inject the LLMTR gateway base URL into the dedupe request's extra_args when no explicit DEDUPE_LLM_API_BASE is set. An explicit DEDUPE_LLM_API_BASE still wins. Addresses Greptile review feedback on the mixed main/dedupe routing case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ag2XdAsvrR7XKMmQsAjqSN
|
Thanks @greptile — went through both points empirically. Issue 1 (mixed main/dedupe routing) — valid in one direction, fixed. Confirmed the real failure: when the dedupe model is Fixed in The symmetric case (main Issue 2 (reasoning-effort detection) — not a defect. |
|
The fix in Both points are well-resolved:
LGTM. Tip: You can customize Greptile's behavior for this repo with |
What
Makes LLMTR a first-class provider prefix, so it works like OpenRouter with zero extra configuration:
No
LLM_API_BASE, noopenai/double-prefix. Any model in the LLMTR catalog works by prefixing its id withllmtr/(e.g.llmtr/openai/gpt-5.5,llmtr/google/gemini-3.5-flash, or the Turkey-hostedllmtr/llmtr/trendyol-asure-12b).Why
LLMTR is a Turkey-hosted, OpenAI-compatible gateway giving single-key access to 265+ models (OpenAI, Anthropic, Google, Qwen) plus Turkey-hosted open models with in-region data residency. Today it can be reached only via the manual OpenAI-compatible recipe (
openai/<model>+LLM_API_BASE); this promotes it to a named prefix so setup is as simple as OpenRouter's.How
LLMTR is OpenAI-compatible, so this reuses the existing LiteLLM route rather than adding provider-specific request code:
StrixProvider._resolve_prefixed_modelresolvesllmtr/<model>to LiteLLM'sopenai/<model>route._configure_llmtr_routingsets the gateway base URL (https://llmtr.com/v1) via LiteLLM's module default — an explicitLLM_API_BASEstill wins, for a proxy in front of LLMTR — and adds Strix attribution headers (HTTP-Referer,X-Title) so scans are identifiable in the LLMTR dashboard._request_headersalso attaches the attribution headers per request, mirroring the existing OpenRouter path.The API key flows through the same
litellm.api_keyglobal that OpenRouter already relies on; frontier-model detection, reasoning-effort selection, and the chat-completions tool schema all already handle a two-segment model id, so no other call sites needed changes.Tests
tests/test_models.pyadds coverage for prefix detection, routing resolution (llmtr/... → openai/...), base-URL + attribution configuration, the explicit-LLM_API_BASEoverride, the non-LLMTR no-op path, and frontier detection through thellmtr/prefix.python -m pytest tests/test_models.py→ 90 passedruff checkclean on changed files;mypyclean on changed files (pre-existing win32-onlyfcntlnotes incodex.pyare unrelated)Relation to #1147
#1147 documents the manual OpenAI-compatible recipe (works today, no code change). This PR is the code that makes the shorter
llmtr/prefix possible; docs can be updated to prefer it once this lands.🤖 Generated with Claude Code