Skip to content

fix(llmobs): skip downstream OpenAI span check for litellm_proxy/ models#17237

Draft
ZStriker19 wants to merge 1 commit intomainfrom
worktree-mlob-6958-litellm-proxy-fix
Draft

fix(llmobs): skip downstream OpenAI span check for litellm_proxy/ models#17237
ZStriker19 wants to merge 1 commit intomainfrom
worktree-mlob-6958-litellm-proxy-fix

Conversation

@ZStriker19
Copy link
Copy Markdown
Collaborator

Problem

The ddtrace litellm integration produces empty LLMObs spans when using a litellm_proxy/ model (e.g. litellm_proxy/azure-gpt-5-nano). The request routes through a proxy, so the OpenAI integration never fires — but the span was being suppressed as if it would.

Root Cause

_has_downstream_openai_span() does a substring check for "azure", "gpt", "openai" in the model name. A model like litellm_proxy/azure-gpt-5-nano matches "azure", so submit_to_llmobs=False is set and the span is dropped entirely.

Fix

Short-circuit in _has_downstream_openai_span() when the model name starts with litellm_proxy/ — these requests always go through a proxy, never directly to OpenAI/Azure.

Testing

Added 9 parametrized unit tests for _has_downstream_openai_span() covering: proxy suppression (the fix), normal OpenAI/Azure passthrough, streaming, disabled OpenAI integration, and non-OpenAI models. No VCR cassette needed — the method is pure logic.

Closes MLOB-6958

🤖 Generated with Claude Code

Models with the litellm_proxy/ prefix route through a proxy and never
invoke the OpenAI integration directly, so the downstream span check
was incorrectly suppressing LLMObs span submission for these requests.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Codeowners resolved as

ddtrace/llmobs/_integrations/litellm.py                                 @DataDog/ml-observability
releasenotes/notes/litellm-proxy-model-llmobs-fix-a1b2c3d4e5f6a7b8.yaml  @DataDog/apm-python
tests/contrib/litellm/test_litellm_llmobs.py                            @DataDog/ml-observability

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant