Skip to content

feat(llm): add first-class llmtr/ provider prefix for the LLMTR gateway - #1148

Open
knowhycodata wants to merge 2 commits into
usestrix:mainfrom
knowhycodata:feat/llmtr-native-provider
Open

feat(llm): add first-class llmtr/ provider prefix for the LLMTR gateway#1148
knowhycodata wants to merge 2 commits into
usestrix:mainfrom
knowhycodata:feat/llmtr-native-provider

Conversation

@knowhycodata

Copy link
Copy Markdown

What

Makes LLMTR a first-class provider prefix, so it works like OpenRouter with zero extra configuration:

export STRIX_LLM="llmtr/anthropic/claude-sonnet-5"
export LLM_API_KEY="llmtr-..."

No LLM_API_BASE, no openai/ double-prefix. Any model in the LLMTR catalog works by prefixing its id with llmtr/ (e.g. llmtr/openai/gpt-5.5, llmtr/google/gemini-3.5-flash, or the Turkey-hosted llmtr/llmtr/trendyol-asure-12b).

Why

LLMTR is a Turkey-hosted, OpenAI-compatible gateway giving single-key access to 265+ models (OpenAI, Anthropic, Google, Qwen) plus Turkey-hosted open models with in-region data residency. Today it can be reached only via the manual OpenAI-compatible recipe (openai/<model> + LLM_API_BASE); this promotes it to a named prefix so setup is as simple as OpenRouter's.

How

LLMTR is OpenAI-compatible, so this reuses the existing LiteLLM route rather than adding provider-specific request code:

  • StrixProvider._resolve_prefixed_model resolves llmtr/<model> to LiteLLM's openai/<model> route.
  • _configure_llmtr_routing sets the gateway base URL (https://llmtr.com/v1) via LiteLLM's module default — an explicit LLM_API_BASE still wins, for a proxy in front of LLMTR — and adds Strix attribution headers (HTTP-Referer, X-Title) so scans are identifiable in the LLMTR dashboard.
  • _request_headers also attaches the attribution headers per request, mirroring the existing OpenRouter path.

The API key flows through the same litellm.api_key global that OpenRouter already relies on; frontier-model detection, reasoning-effort selection, and the chat-completions tool schema all already handle a two-segment model id, so no other call sites needed changes.

Tests

tests/test_models.py adds coverage for prefix detection, routing resolution (llmtr/... → openai/...), base-URL + attribution configuration, the explicit-LLM_API_BASE override, the non-LLMTR no-op path, and frontier detection through the llmtr/ prefix.

  • python -m pytest tests/test_models.py → 90 passed
  • ruff check clean on changed files; mypy clean on changed files (pre-existing win32-only fcntl notes in codex.py are unrelated)

Relation to #1147

#1147 documents the manual OpenAI-compatible recipe (works today, no code change). This PR is the code that makes the shorter llmtr/ prefix possible; docs can be updated to prefer it once this lands.

🤖 Generated with Claude Code

Make LLMTR (https://llmtr.com) a first-class option alongside OpenRouter:
users set STRIX_LLM="llmtr/<model>" + LLM_API_KEY (llmtr-...) and nothing
else. LLMTR is a Turkey-hosted OpenAI-compatible gateway, so StrixProvider
resolves llmtr/<model> to LiteLLM's openai/<model> route, and
_configure_llmtr_routing supplies the gateway base URL (an explicit
LLM_API_BASE still wins) plus Strix attribution headers. Attribution is
also sent per-request via _request_headers, mirroring OpenRouter.

Adds unit tests for prefix detection, routing resolution, base-URL/header
configuration, explicit-base override, the non-LLMTR no-op path, and
frontier-model detection through the llmtr/ prefix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ag2XdAsvrR7XKMmQsAjqSN
@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds llmtr/ as a first-class OpenAI-compatible gateway prefix and injects the LLMTR endpoint and attribution headers into model requests.

  • Rewrites llmtr/<model> to LiteLLM's openai/<model> route.
  • Configures the default LLMTR API endpoint while preserving explicit API-base overrides.
  • Adds global and per-request attribution headers.
  • Adds routing, classification, override, and frontier-model tests.

Confidence Score: 3/5

The PR should not merge until mixed main/deduplication model routing and reasoning capability detection correctly account for the new LLMTR prefix.

Process-global endpoint selection is derived from the main model even when deduplication uses another model, and LLMTR-prefixed reasoning models silently lose configured reasoning effort because capability detection never removes the new prefix.

Files Needing Attention: strix/config/models.py and the deduplication routing coverage in tests/test_models.py

Important Files Changed

Filename Overview
strix/config/models.py Adds LLMTR provider resolution and global routing configuration, but routing is keyed only to the main model and capability detection does not normalize the new prefix.
strix/core/inputs.py Adds per-request LLMTR attribution headers while retaining caller-supplied header precedence.
tests/test_models.py Covers direct prefix routing and global configuration but omits mixed main/deduplication models and reasoning-effort propagation.
Prompt To Fix All With AI
### Issue 1
strix/config/models.py:568
**Global routing ignores dedupe model**

When the main and deduplication models use different providers and exactly one uses `llmtr/`, `_configure_llmtr_routing` configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.

### Issue 2
strix/config/models.py:469-473
**LLMTR prefix bypasses reasoning detection**

When an LLMTR-hosted reasoning model is used with `STRIX_REASONING_EFFORT`, capability detection examines the original `llmtr/<provider>/<model>` name without removing the new prefix, so `make_model_settings` silently omits the configured reasoning effort.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat(llm): add first-class llmtr/ provid..." | Re-trigger Greptile

Comment thread strix/config/models.py
return
_configure_litellm_compatibility()
_configure_openrouter_attribution(llm.model)
_configure_llmtr_routing(llm.model, llm.api_base)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Global routing ignores dedupe model

When the main and deduplication models use different providers and exactly one uses llmtr/, _configure_llmtr_routing configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.

Knowledge Base Used: Configuration and Telemetry

Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/config/models.py
Line: 568

Comment:
**Global routing ignores dedupe model**

When the main and deduplication models use different providers and exactly one uses `llmtr/`, `_configure_llmtr_routing` configures LiteLLM's process-global endpoint from only the main model before the independently selected deduplication model is resolved, causing deduplication requests to reach the wrong gateway and fail authentication or model lookup.

**Knowledge Base Used:** [Configuration and Telemetry](https://app.greptile.com/strix-org-3/-/custom-context/knowledge-base/usestrix/strix/-/docs/telemetry-and-config.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment thread strix/config/models.py
Comment on lines +469 to +473
if prefix == "llmtr" and stripped_model_name:
# LLMTR is an OpenAI-compatible gateway, so route llmtr/<model>
# through LiteLLM's openai/ provider. The gateway base URL and
# attribution headers are configured in _configure_llmtr_routing.
return self._get_fallback_provider("litellm"), f"openai/{stripped_model_name}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 LLMTR prefix bypasses reasoning detection

When an LLMTR-hosted reasoning model is used with STRIX_REASONING_EFFORT, capability detection examines the original llmtr/<provider>/<model> name without removing the new prefix, so make_model_settings silently omits the configured reasoning effort.

Knowledge Base Used: Configuration and Telemetry

Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/config/models.py
Line: 469-473

Comment:
**LLMTR prefix bypasses reasoning detection**

When an LLMTR-hosted reasoning model is used with `STRIX_REASONING_EFFORT`, capability detection examines the original `llmtr/<provider>/<model>` name without removing the new prefix, so `make_model_settings` silently omits the configured reasoning effort.

**Knowledge Base Used:** [Configuration and Telemetry](https://app.greptile.com/strix-org-3/-/custom-context/knowledge-base/usestrix/strix/-/docs/telemetry-and-config.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

When the dedupe model uses llmtr/ but the main model is a different
provider, the global litellm.api_base (derived from the main model) is
not the LLMTR gateway, so the dedupe request would resolve to LiteLLM's
default openai/ endpoint and fail. Mirror the existing per-call dedupe
credential pattern: inject the LLMTR gateway base URL into the dedupe
request's extra_args when no explicit DEDUPE_LLM_API_BASE is set. An
explicit DEDUPE_LLM_API_BASE still wins.

Addresses Greptile review feedback on the mixed main/dedupe routing case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ag2XdAsvrR7XKMmQsAjqSN
@knowhycodata

Copy link
Copy Markdown
Author

Thanks @greptile — went through both points empirically.

Issue 1 (mixed main/dedupe routing) — valid in one direction, fixed. Confirmed the real failure: when the dedupe model is llmtr/... but the main model is a different provider, the global litellm.api_base is derived from the main model, so the dedupe request resolved to LiteLLM's default openai/ endpoint (api.openai.com) instead of the LLMTR gateway.

Fixed in bcfa922 by mirroring the existing per-call dedupe-credential pattern (_dedupe_extra_args): a dedicated llmtr/ dedupe model now pins the gateway base URL on the request itself. An explicit DEDUPE_LLM_API_BASE still wins. New tests in tests/test_dedupe_model.py cover the pin, the override, and the non-LLMTR no-op.

The symmetric case (main llmtr/, dedupe a different native provider, no DEDUPE_LLM_API_BASE) behaves exactly like an explicit LLM_API_BASE on the main model would today — the shared base URL is process-wide, which is the documented reason _dedupe_extra_args exists — so it's an existing, documented limitation rather than a new regression.

Issue 2 (reasoning-effort detection) — not a defect. model_supports_reasoning doesn't strip llmtr/, but its last-segment fallback (model_cost.get(name.rsplit("/", 1)[1])) resolves the bare model id, so detection is identical to the openrouter/ prefix (which also isn't in the strip list). Verified across deepseek-reasoner, o3, and gpt-5.4 that llmtr/..., openrouter/..., and the bare id all return True — reasoning effort is applied correctly, so no change was needed.

@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

The fix in bcfa922 looks correct and clean. The elif is_llmtr_model(dedupe.model) guard correctly falls through only when no explicit DEDUPE_LLM_API_BASE is set (since api_base is checked first), and it applies the constant rather than a string literal — exactly mirroring the existing pattern.

Both points are well-resolved:

  • Issue 1: Per-call api_base pinning with the explicit override still taking priority. The symmetric case (main llmtr/, dedupe a different native provider) being a pre-existing process-wide limitation is a fair characterization — it's the documented reason _dedupe_extra_args exists in the first place.

  • Issue 2: The last-segment fallback (name.rsplit("/", 1)[1]) in model_supports_reasoning is the same path openrouter/ takes, so parity holds. No change needed.

LGTM.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant