Skip to content

Conversation

@QiuZiXian
Copy link
Contributor

llm添加openai风格服务端支持(自定义代理地址和模型名称)

使用mindsearch的时候,发现无法使用自定义代理地址、模型名称的openai风格服务端。
GPTAPI有进行模型校验等原因,秉承只add的原则。基于GPTAPI扩展GPTStyleAPI:

支持openai风格:

已测试通过:

  • xinference 1.2.0
  • ollama 0.5.5
  • one-api v0.6.10-alpha.6
  • baichuan 直连
  • lmdeploy 0.7.0

使用方法:

设置api_base、 model_name、key(如果服务端不需要,可忽略)

    model_name = "deepseek-r1:1.5b"
    gpttool = GPTStyleAPI(
        model_type=model_name,
        api_base=api_base,
        key="sk-IXgCTwuoEwxL1CiBE4744688D8094521B70f4aDeE6830c5e",
    )
    res = gpttool.chat(inputs=[
        {
            "role": "user",
            "content": "世界第一高峰是"
        }])

具体参见tests/test_llms/test_gptstyleapi.py

@flingjie
Copy link

是不是用litellm来统一调用各个provider更加简洁和容易维护?

@QiuZiXian QiuZiXian mentioned this pull request Feb 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants