Skip to content

Add support for configurable base_url in autoembedding #3771

@SumJest

Description

@SumJest

Proposal:

Currently, Manticore Search supports auto embeddings for several providers (OpenAI, Voyage, Jina, Sentence Transformers). For cloud-based providers (like OpenAI, Voyage, Jina), the endpoint is hardcoded to the official API base URL.

Problem:
In some environments, users may need to use:

  • Custom endpoints (e.g., OpenAI-compatible APIs like Azure OpenAI, local proxy servers, or self-hosted gateways).
  • Private/self-hosted inference endpoints that expose OpenAI-compatible or HuggingFace-compatible APIs.

Right now, there is no way to override the default base_url, which makes it impossible to use Manticore’s autoembedding feature with these setups.

Proposal:
Add a new optional parameter BASE_URL='<custom_url>' for auto embedding configuration.
For example:

CREATE TABLE products_openai (
    title TEXT,
    description TEXT, 
    embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='openai/text-embedding-ada-002' FROM='title,description' API_KEY='...' BASE_URL='https://custom-endpoint.example.com/v1'
);

Use cases:

  • Support for Azure OpenAI Service, which uses endpoints like https://<resource>.openai.azure.com/ instead of https://api.openai.com/.
  • Allowing users to connect to self-hosted API gateways or custom providers that expose OpenAI-compatible or HuggingFace-compatible APIs.
  • Increased flexibility and easier integration in enterprise environments with strict networking/security requirements.

This change would make auto embedding much more flexible while remaining backward compatible (if BASE_URL is not provided, defaults remain unchanged).

Checklist:

To be completed by the assignee. Check off tasks that have been completed or are not applicable.

  • Implementation completed
  • Tests developed
  • Documentation updated
  • Documentation reviewed
  • OpenAPI YAML updated and issue created to rebuild clients

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions