Skip to content

Tests and examples for handlers.llm should be fully parametric in model name #589

@eb8680

Description

@eb8680

The basic litellm.completion API doesn't distinguish providers per se, it just takes a model name. We should likewise attempt to scrub all hard-coded provider dependence from our handlers.llm library code, tests and examples, including logic that checks for provider-specific environment variables like OPENAI_API_KEY that are only used internally in litellm, in favor of implicit provider dispatch through model names.

There are a number of benefits to doing this beyond simplifications to test code. For example, by making the test suite parametric in a model name it would be easy to run parallel CI build stages with different models or providers by simply varying an environment variable, which might help with #587.

It could also help shift the burden of validating model configuration further onto litellm and out of our code, e.g. by deferring to utilities like litellm.supports_vision to check that a model is suitable for a given task.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions