What happened?

We're using LiteLLM with Anthropic (Sonnet 4) in the background and noticed that LiteLLM seems to convert any image URLs we pass to base64 content before it's dispatched to the Anthropic API, even though the Anthropic API supports raw image URLs natively.

According this the LiteLLM documentation on this topic (https://docs.litellm.ai/docs/proxy/image_handling), I'd expect this to happen only for providers who don't support this.

The impact is that due to the base64 conversion, images are much bigger than they need to be and the Anthropic limit of 5MB per image is hit earlier (i.e already at ~4MB).

I'm assuming the Base64 conversion is the root cause, using the exact same image directly with the Anthropic API works just fine.

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.74.3-stable.patch.4

Twitter / LinkedIn details

No response