🧠 feat: Azure AI Search vector search support with hybrid search #10441

benverhees · 2025-11-10T22:10:36Z

Summary

This PR adds Azure AI Search vector search support, enabling semantic search capabilities in the Azure AI Search tool. The implementation includes:

Primary Feature - Azure AI Search Vector Search:

Added support for vector search queries in Azure AI Search tool
Introduced optional AZURE_AI_SEARCH_VECTOR_FIELDS configuration field to specify the vector field names in the Azure AI Search index
Updated Azure Search API version from 2023-11-01 to 2025-09-01 to support vector search features
Enhanced tool schema to support:
- vectorQueryText: Text to be vectorized for vector search queries
- filter: OData $filter expression for search queries
- query: Text for keyword search (existing functionality)
Supports hybrid search (combining keyword and vector search) when both query and vectorQueryText are provided

Supporting Features:

Agent Tool Configuration UI: Added ability to edit tool configurations directly from the agent tools list with a gear icon, allowing users to update authentication credentials without removing and re-adding tools.
Plugin Auth Values API: New endpoint GET /agents/tools/:pluginKey/auth-values that returns plugin authentication values with sensitive fields masked using a placeholder value (<redacted>).
Sensitive Field Handling: Added sensitive: true flag to all API key fields in plugin manifest configurations. Sensitive fields are masked in the API response and can be updated without requiring the user to re-enter existing values.
Optional Field Support: Added support for optional flag in plugin auth configurations, allowing fields to be marked as optional in the UI.

Dependencies:

Updated @azure/search-documents from ^12.0.0 to ^12.2.0 to support vector search features

Change Type

Please delete any irrelevant options.

New feature (non-breaking change which adds functionality)

Testing

Test Configuration:

Primary Testing - Azure AI Search Vector Search:

Vector Search Configuration:
- Configure Azure AI Search tool with AZURE_AI_SEARCH_VECTOR_FIELDS set to the names of your vector fields
- Verify the vector field appears as optional in the configuration UI
- Test configuration without vector field (should work with keyword search only)
Vector Search Queries:
- Test vector search query with vectorQueryText parameter only
- Verify results are returned based on semantic similarity
Hybrid Search (Keyword + Vector):
- Test hybrid search with both query (keyword) and vectorQueryText (vector) parameters
- Verify results combine both keyword and semantic search
- Test with various combinations of query parameters
Filter Support:
- Test filter parameter with OData $filter expressions
- Verify filters work with both keyword and vector searches
- Test complex filter expressions

Supporting Features Testing:

Sensitive Field Handling:
- Configure a plugin with sensitive fields
- Call GET /agents/tools/{pluginKey}/auth-values
- Verify sensitive fields return <redacted> placeholder
- Verify non-sensitive fields return actual values
- Update plugin with existing sensitive field values (empty should preserve, new value should update)
Agent Tool Configuration UI:
- Navigate to agent tools list
- Verify gear icon appears for tools with auth config (including Azure AI Search)
- Click gear icon to open edit dialog
- Verify existing values are loaded (sensitive fields masked)
- Test updating Azure AI Search configuration including vector field

Checklist

Please delete any irrelevant options.

My code adheres to this project's style guidelines
I have performed a self-review of my own code
I have commented in any complex areas of my code
My changes do not introduce new warnings
I have written tests demonstrating that my changes are effective or that my feature works
Local unit tests pass with my changes
Any changes dependent on mine have been merged and published in downstream modules.

client/src/components/Plugins/Store/PluginAuthForm.tsx

danny-avila · 2025-11-11T14:41:12Z

api/server/controllers/UserController.js


    if (action === 'install') {
+      // Get tool config to check for sensitive fields
+      const { manifestToolMap } = require('~/app/clients/tools');


no dynamic imports

danny-avila · 2025-11-11T14:41:26Z

api/server/controllers/tools.js

+  try {
+    const { pluginKey } = req.params;
+    const { manifestToolMap } = require('~/app/clients/tools');
+    const { getAuthFields } = require('~/app/clients/tools/util/handleTools');


refrain from dynamic imports

danny-avila · 2025-11-11T14:42:43Z

thanks, mostly looks good to me. possibly as a followup, I can see people asking for multiple vector search fields. possibly handle with 1 input, comma-delimited? can be in a followup PR after I test this a bit

danny-avila · 2025-11-11T14:43:10Z

and just missing documentation PR

benverhees · 2025-11-11T15:28:21Z

and just missing documentation PR

I will create one later today!

benverhees · 2025-11-11T20:59:35Z

thanks, mostly looks good to me. possibly as a followup, I can see people asking for multiple vector search fields. possibly handle with 1 input, comma-delimited? can be in a followup PR after I test this a bit

I'll add support for multiple vector fields as part of this PR, I also found a bug when loading pre-exisiting auth configs that don't have the newly added field yet. I'll fix that too.

benverhees · 2025-11-11T22:34:34Z

@danny-avila docs are in https://github.com/LibreChat-AI/librechat.ai/pull/441/files

benverhees · 2025-11-17T19:53:58Z

@danny-avila I just fixed the merge conflict that was in package-lock.json. Is there something I can do to get this merged, or do you need to test this as you mentioned?

The force push was because I needed to rebase. In a merge commit, some eslint errors emerged that I think were missed by contributers that don't have pre-commit running. After running npm run lint I noticed there are much more errors in the codebase. I created a PR for the auto-fixable errors in #10549. But there are still some errors left, so that might need some attention in the future.

github-advanced-security bot found potential problems Nov 10, 2025

View reviewed changes

client/src/components/Plugins/Store/PluginAuthForm.tsx Fixed Show fixed Hide fixed

benverhees force-pushed the feat/azure-ai-vector-search branch 2 times, most recently from 1c20d19 to 6658e37 Compare November 10, 2025 22:30

benverhees marked this pull request as ready for review November 11, 2025 10:16

benverhees changed the title ~~feat: Azure AI Search vector search support with hybrid search~~ 🧠 feat: Azure AI Search vector search support with hybrid search Nov 11, 2025

danny-avila requested changes Nov 11, 2025

View reviewed changes

benverhees requested a review from danny-avila November 11, 2025 22:40

benverhees added 6 commits November 17, 2025 18:22

Add vector search support to Azure AI Search plugin

f6d038d

feat: add ability to edit plugin auth values from agent tools interface

e5d1edd

test: add tests for tools auth-values endpoint

e60c5c5

refactor: replace dynamic imports with static imports

fd8540b

feat: support multiple vector fields for vector search

83b8ee7

fix: prevent errors when optional authentication fields are missing

187f1b6

benverhees force-pushed the feat/azure-ai-vector-search branch from d351f73 to 187f1b6 Compare November 17, 2025 17:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

🧠 feat: Azure AI Search vector search support with hybrid search #10441

🧠 feat: Azure AI Search vector search support with hybrid search #10441

benverhees commented Nov 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

danny-avila Nov 11, 2025

Uh oh!

benverhees Nov 11, 2025

Uh oh!

danny-avila Nov 11, 2025

Uh oh!

benverhees Nov 11, 2025

Uh oh!

danny-avila commented Nov 11, 2025

Uh oh!

danny-avila commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 17, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

🧠 feat: Azure AI Search vector search support with hybrid search #10441

Are you sure you want to change the base?

🧠 feat: Azure AI Search vector search support with hybrid search #10441

Conversation

benverhees commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change Type

Testing

Test Configuration:

Checklist

Uh oh!

Uh oh!

danny-avila Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

benverhees Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

danny-avila Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

benverhees Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

danny-avila commented Nov 11, 2025

Uh oh!

danny-avila commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 11, 2025

Uh oh!

benverhees commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

benverhees commented Nov 10, 2025 •

edited

Loading

benverhees commented Nov 17, 2025 •

edited

Loading