-
-
Notifications
You must be signed in to change notification settings - Fork 617
Description
Proposal:
Currently, Manticore Search supports auto embeddings for several providers (OpenAI, Voyage, Jina, Sentence Transformers). For cloud-based providers (like OpenAI, Voyage, Jina), the endpoint is hardcoded to the official API base URL.
Problem:
In some environments, users may need to use:
- Custom endpoints (e.g., OpenAI-compatible APIs like Azure OpenAI, local proxy servers, or self-hosted gateways).
- Private/self-hosted inference endpoints that expose OpenAI-compatible or HuggingFace-compatible APIs.
Right now, there is no way to override the default base_url
, which makes it impossible to use Manticore’s autoembedding feature with these setups.
Proposal:
Add a new optional parameter BASE_URL='<custom_url>'
for auto embedding configuration.
For example:
CREATE TABLE products_openai (
title TEXT,
description TEXT,
embedding_vector FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
MODEL_NAME='openai/text-embedding-ada-002' FROM='title,description' API_KEY='...' BASE_URL='https://custom-endpoint.example.com/v1'
);
Use cases:
- Support for Azure OpenAI Service, which uses endpoints like
https://<resource>.openai.azure.com/
instead ofhttps://api.openai.com/
. - Allowing users to connect to self-hosted API gateways or custom providers that expose OpenAI-compatible or HuggingFace-compatible APIs.
- Increased flexibility and easier integration in enterprise environments with strict networking/security requirements.
This change would make auto embedding much more flexible while remaining backward compatible (if BASE_URL
is not provided, defaults remain unchanged).
Checklist:
To be completed by the assignee. Check off tasks that have been completed or are not applicable.
- Implementation completed
- Tests developed
- Documentation updated
- Documentation reviewed
- OpenAPI YAML updated and issue created to rebuild clients