Unable to use Responses API with Azure OpenAI when using a custom base URL value

[jcbartle]

@danny-avila - following on from the discussion in #10153.

I was able to test a PDF upload to the Responses API in Azure OpenAI and got it working via a standard REST call, but NOT through LibreChat.

The problem seems to stem from two items:

• Using a custom baseURL in librechat.yaml for Azure OpenAI endpoints.
• LibreChat does not honor the api-version query string parameter when selecting Responses API

We're using an Azure API Management instance as our "AI Gateway", but this is similar conceptually to your custom CloudFlare example you have documented.

Here's a snippet from my librechat.yaml file for reference:

azureOpenAI:
    # Endpoint-level configuration
    plugins: false
    assistants: false
    summarize: false
    groups:
    # Group-level configuration
    - group: "Standard Models"
      apiKey: "${AZURE_OPENAI_SUBSCRIPTION_KEY}"
      instanceName: "N/A"
      version: "2025-04-01-preview"
      baseURL: "https://<AZURE_APIM_URL>/openai/<API_NAME>/inference/deployments/${DEPLOYMENT_NAME}"

When I turn on the Responses API option in the LibreChat UI, I see this URL getting called in our APIM logs:

https://<AZURE_APIM_URL>/openai/<API_NAME>/inference/deployments/<DEPLOYMENT_ID>/responses

This results in a 404 for two reasons. The correct URL which should be getting hit it:

https://<AZURE_APIM_URL>/openai/<API_NAME>/inference/responses?api-version=2025-04-01-preview

In LibreChat, the /deployments/<DEPLOYMENT_ID> section is still present and the api-version query string variable has been dropped.

If I update my librechat.yaml file and change the baseURL value to:

https://<AZURE_APIM_URL>/openai/<API_NAME>/inference

The Responses API URL is now correct, but the api-version query variable is still missing, so the call still fails. The only way I can get this fixed is by overwriting this code in config.ts and hard-coding the api-version:

configOptions.defaultQuery = {
	...configOptions.defaultQuery,
	//'api-version': configOptions.defaultQuery?.['api-version'] ?? 'preview',
	'api-version': '2025-04-01-preview'
};

I, so far, have been able to find no way to have the api-version be retained through configuration.

HOWEVER, this causes a big problem in that all non-Responses API calls will now fail due to the baseURL value being incorrect. The only two formats which work for non-Responses API calls are:

https://<AZURE_APIM_URL>/openai/<API_NAME>/inference/deployments/${DEPLOYMENT_NAME}
https://<AZURE_APIM_URL>/openai/<API_NAME>/inference/deployments

For most implementations, such as the openai Python module, the baseURL value for Azure OpenAI is specified like this (i.e., not including /deployments or the deployment ID):

https://<AZURE_APIM_URL>/openai/<API_NAME>/inference

If LibreChat accepted this format and still sent over the api-version query string parameter, it would fix both the "standard" inference calls and the Responses API calls.

We are using the...I don't know..."classic" Responses API and not the new "v1" API, which - as far as I can tell - just went GA on September 1st.

Is there a way to utilize the Responses API when also utilizing a custom baseURL for Azure OpenAI endpoints?

[georgenmatthew]

Hi @jcbartle , I have experienced a similar issue using an Azure PrivateLink endpoint baseURL. Toggling "Use Responses API" breaks things. If I modify the baseURL, I can get the Responses API working but then non-Responses breaks.

Interested in any resolution you may find.

[jcbartle]

@danny-avila - did you have any thoughts on this one? I am leaning towards this being a bug or at least design flaw with the custom base URL functionality for Azure OpenAI. I think there are a couple of different ways to solve it. Not sure of the best approach. Item 3 seems easiest.

1 - modify base URL config to not require /deployments or the deployment ID. This would match the handling for other common implementations.
2 - introduce new config field for responses API baseURL.
3 - adjust responses API URL construction handling when using custom baseURL.

The code also needs to still send over the api-version query string parameter value, as this is required for "classic" API versions.

There also likely needs to be some configuration setting for using the "classic" API versions or the new v1 API version, though this item is less important in my mind.

[jainrahulsethi]

Hi Guys,

Even after turning on the responses API in the settings, I see these 3 options

Which option will let the file directly go to Azure OpenAI & then let it do the RAG instead of the file_search doing the heavy lifting?

[stam-lab]

@jcbartle can you submit this as a bug? We are facing the exact same issue and it is big problem for us. Thank you!

[danny-avila]

It should be fixed already on latest version: #10289

[jcbartle]

Good morning, @danny-avila. I've made some progress on sorting out this issue. The recent PR #10289 by @peeeteeer and PR #10336 by @maxesse have helped! However, things are not perfect, and I think some decisions will need to be made on the best path forward.

(apologies this is a little long, but hopefully all the context can help others)

For azureOpenAI configuration, LibreChat doesn't send over the api-version query string parameter when selecting Use Responses API. This means the "classic" Responses API (via API version 2024-04-01-preview) doesn't work, and you must instead use the new v1 API. Cool. I've implemented that internally within our AI Gateway. However...

In order to use the new v1 API, the URL format changes. Whereas the "classic" Azure OpenAI URLs look like this:

https://resource.openai.azure.com/openai/deployments/deployment_id/chat/completions?api-version=api_version
https://resource.openai.azure.com/openai/responses?api-version=2025-04-01-preview

The new v1 API URLs look like this:

https://resource.cognitiveservices.azure.com/openai/v1/chat/completions
https://resource.cognitiveservices.azure.com/openai/v1/responses

Issue Number 1 - v1 API in Azure OpenAI Endpoint

When I tried using the azureOpenAI endpoint in the librechat.yaml file, you cannot use the v1 API, because, when you set a baseURL value of, say, https://resource.cognitiveservices.azure.com/openai/v1, the final URL will end up looking like this when not using the Responses API:

https://resource.cognitiveservices.azure.com/openai/v1/deployment_id/deployments/deployment_id/chat/completions?api-version=2025-04-01-preview

This results in a 404.

When using the Responses API, it will work properly after PR #10289.

One potential solution would be to have two separate endpoint configuration sections, one for azureOpenAI and one for azureOpenAIv1. Right now, the azureOpenAI endpoint is splitting things between two completely different API schemas, and I don't believe that makes sense. It should either be using the "classic" Azure OpenAI API or the new v1 API, not flipping back and forth depending on the state of the Responses API toggle switch.

Additionally, the azureOpenAI endpoint doesn't support the customParams configuration feature, so you can't force Responses API to be on by default.

The way I finally got the v1 API working with or without the Responses API toggle enabled was via a custom endpoint configuration. This works almost perfectly.

Issue Number 2 - Upload to Provider versus Upload Image

A minor issue with custom configuration is that the file upload menu reads "Upload to Provider" all the time, even though file uploads are only supported when using the Responses API and not when using the chat/completions endpoint. When the Responses API toggle is off, the menu should read "Upload Image". When it's on, it should read "Upload to Provider".

Issue Number 3 - Ordering of Endpoints in ModelSelect Dialog

The ModelSelect dialog supports ordering of endpoints via the ENDPOINTS environment variable value (documented in this README file). Unfortunately, you can't order custom endpoints individually - anything under custom is treated as one entity for ordering purposes. Pretend we have three endpoints, two custom (one of which is Azure OpenAI using the v1 API) and Bedrock. The order in the ModelSelect dialog will be one of these two:

Custom 1
Custom 2
Bedrock

or

Bedrock
Custom 1
Custom 2

There's no way to have:

Custom 1
Bedrock
Custom 2

Recap and Workaround

The current answer / workaround to this Q&A is:

1. Deploy the new v1 API
2. Create a custom endpoint in librechat.yaml

For anyone who needs it, here is an anonymized and stripped-down custom config which works:

custom:
- name: "Azure AI Foundry"
  apiKey: "your_api_key"
  baseURL: "https://ai-gateway.yourorg.com/azure-openai-v1"
  customParams:
    defaultParamsEndpoint: azureOpenAI
  models:
    default:
	- "gpt-5-chat-2025-10-03"
	- "gpt-4.1-nano-2025-04-14"
    fetch: false
  summarize: false
  titleConvo: true
  titleModel: "gpt-4.1-nano-2025-04-14"

Thoughts on what a long-term solution would be? Given that the point of the Azure OpenAI v1 API is compatibility and not requiring all the custom configuration which the "classic" Azure OpenAI does, I'm actually okay with just having it as a custom endpoint, as this seems to work just fine. The only real problem I have with this is the lack of ordering of custom endpoints in the ModelSelect dialog - if that can be addressed, I think we have a solution. I still don't think the behavior of the azureOpenAI endpoint makes sense, but it seems that is an issue which is likely to go away if Microsoft's long-term focus is on the new v1 API.

[riosengineer]

@jcbartle i am interested in this topic also. I am using apim as the proxy and want to be able to add custom endpoints with subscription keys so i can track usage and limit tokens per team. In your latest post, can you confirm that custom endpoint to foundry VIA APIM base url does work with just a sub key as the header? (In my case I will make the api key user provided for the GUI)

Thanks!

[jcbartle]

Hi, @riosengineer. That's not a super-quick answer, and it's outside the scoped discussion of this thread. Can you post a new discussion in Q&A, tag me, and I'll be happy to provide our experience? I want to keep this thread clean.

Update: new thread ID if anyone's interested: #10445