Add anthropic retrieve batches and retreive file content support #17700
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ) | ||
|
|
||
| verbose_proxy_logger.info( | ||
| f"Stored Anthropic batch managed object with unified_object_id={unified_object_id}, batch_id={model_object_id}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
To address this issue, we should avoid logging the raw unified_object_id and batch_id values if they could incorporate or be derived from sensitive user data (such as API keys, passwords, identifiers, etc.). The best fix is to redact or mask the sensitive portions before logging, or simply omit logging these fields altogether if not strictly necessary. If logging is required for troubleshooting or audit, log only non-sensitive, high-level information (e.g., log that a batch managed object was stored, but do not log any identifiers).
The problematic code is in litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py, lines 513–514.
The fix involves updating the logging statement to either:
- Redact sensitive information;
- Omit the identifiers;
- Or hash the identifiers (e.g., use a one-way hash and log only the hash).
Given best practices, it’s preferable to simply remove the logging of the identifier or replace it with a generic statement that does not include potentially sensitive IDs.
| @@ -510,7 +510,7 @@ | ||
| ) | ||
|
|
||
| verbose_proxy_logger.info( | ||
| f"Stored Anthropic batch managed object with unified_object_id={unified_object_id}, batch_id={model_object_id}" | ||
| "Stored Anthropic batch managed object for cost tracking." | ||
| ) | ||
| else: | ||
| verbose_proxy_logger.warning("Managed files hook not available, cannot store batch object for cost tracking") |
| else: | ||
| # Fallback to model name | ||
| actual_model_id = model_name | ||
| verbose_proxy_logger.warning(f"Model not found in router, using model name: {actual_model_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
Sensitive data should never be logged, especially if it originates from untrusted input. Here, the problematic log statement is in AnthropicPassthroughLoggingHandler.get_actual_model_id_from_router. To fix the issue:
- Avoid logging the actual, raw model name if it is user-supplied and not guaranteed to be safe (sanitize, redact, or hash).
- Instead of printing the full model name, log only that the fallback occurred, or log a redacted version (e.g., truncating, masking, or indicating its class/type).
- Optionally, use a utility to sanitize strings before logging, e.g., only print the model provider, or a fixed prefix, or a hash.
- Apply this fix specifically to line 537 (and optionally 541 for consistency), in litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py.
- No additional dependencies are needed, as the fix can be implemented in plain Python.
| @@ -534,9 +534,9 @@ | ||
| else: | ||
| # Fallback to model name | ||
| actual_model_id = model_name | ||
| verbose_proxy_logger.warning(f"Model not found in router, using model name: {actual_model_id}") | ||
| verbose_proxy_logger.warning("Model not found in router, using fallback model identifier.") | ||
| return actual_model_id | ||
| else: | ||
| # Fallback if router is not available | ||
| verbose_proxy_logger.warning(f"Router not available, using model name: {model_name}") | ||
| verbose_proxy_logger.warning("Router not available, using fallback model identifier.") | ||
| return model_name |
| return actual_model_id | ||
| else: | ||
| # Fallback if router is not available | ||
| verbose_proxy_logger.warning(f"Router not available, using model name: {model_name}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
To fix this issue, we want to ensure that potentially sensitive user-supplied values, such as model_name, are never included in clear-text logs. The best approach is to avoid including the real value of model_name in the log message and instead log a generic placeholder, such as [REDACTED], or simply do not mention the specific model name at all. This preserves logging context (e.g., that a fallback occurred) without risking data leakage. We will edit the log statement at line 541 in litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py, replacing the message string so the potentially sensitive data is not logged. No other changes are required.
| @@ -538,5 +538,5 @@ | ||
| return actual_model_id | ||
| else: | ||
| # Fallback if router is not available | ||
| verbose_proxy_logger.warning(f"Router not available, using model name: {model_name}") | ||
| verbose_proxy_logger.warning("Router not available, using model name: [REDACTED]") | ||
| return model_name |
Title
Add anthropic retrieve batches and retreive file content support
Relevant issues
Fixes LIT-1457
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
Changes