Dynamic MCP Server and Tool Discovery

[nhtruong]

Motivation

Currently, MCP servers are determined by a static list within librechat.yaml, this works fine for small scale deployments, but a much larger audience, we need a way to dynamically discover MCP servers without having to update the librechat.yaml file. There are 3 major use cases that I can think of:

1. Allowing admins to add MCP servers without having to update the librechat.yaml file

On top of having a static list of MCP servers, we can allow admins to add MCP servers via the admin interface. This way, admins can add MCP servers as needed without having to update the librechat.yaml file and redeploy the application. The MCP server metadata can be stored in the database and can be retrieved by the application at runtime.

Expanding this further, MCP server entries stored in the database with the same names as those in the librechat.yaml file can override the static entries, allowing admins to modify the MCP server metadata without having to update the librechat.yaml file. This will allow for a smooth transition from a static list to a dynamic list of MCP servers.

2. Allowing each user to specify their own MCP servers

Users can be given the option to specify their own MCP servers in the application settings. This way, users can connect to any MCP server they trust or prefer without having to ask their LibreChat admins to add it to the static list. The application can provide a UI for users to add, remove, or modify their MCP server metadata. The user-specified MCP servers can be stored in the database associated with the user's account.

This can also be expanded to allow the owner of a user group to specify the MCP servers for that group, allowing for more control over the MCP servers used by a specific group of users.

3. Local MCP Server Discovery

A team at Shopify is working on a local MCP server discovery mechanism that allows LibreChat, which runs on the cloud, to discover MCP servers running on the user's local machine. This feature creates a tunnel running on the user's machine and dynamically registering MCP servers on LibreChat. This allows users to connect to their local MCP servers without having to expose them to the internet or ask their LibreChat admins to add them to the static list.

Top-level Design

This is a complex problem, and we need a strong foundation to be able to seamlessly support different discovery methods. The previous work on refactoring the MCPManager set the first building blocks for this vision. This is how I envision how the components of MCPManagers should behave to support Dynamic MCP Server Discovery:

I. MCP Servers Registry

This will be the authoritative source of truth for all MCP servers provided by LibreChat. No matter how a server is discovered, it must be registered in this registry. Other than having the ability to ingest new MCP servers, this registry will also provide a way to query the list of available MCP servers and answer questions like "What are the available MCP servers for a specific user?" or "What form of authentication does a server require?".

Since the list of servers and their details can change dynamically, this registry should hold the server configs in a cache so that all LibreChat backend servers can see the same list of MCP servers. This cache must be updated anytime a new MCP server is added or an existing one is modified.

At startup, this registry loads up the MCP servers from the librechat.yaml file, from the database, and/or from a local MCP tunnel. It then builds a set of caches to hold different tiers of MCP servers from scratch (e.g. app-level, group-level, user-level). This process, if run by all backend servers will lead to a race condition, so it should be run by a single backend server called the leader. The leader also periodically refreshes the cache by re-querying the MCP Server Discovery API.

II. MCP ToolBox

Each user (or user group) should have their own set of MCP tools that are available to them. The tools should be dynamically loaded based on the MCP servers available to the users, and the tools each server provides for that user.

To find out which tools are available to a user, the application must first query the MCP Servers Registry to get the list of available MCP servers for that user, then query each server to get the list of tools it provides for that user. This process is expensive, so it should be cached for a reasonable amount of time.

The list of tools per user will be populated on-demand, instead of at startup.

III. MCP Connections Repository

While modern Streamable HTTP MCP servers are perfect for ephemeral connections, LibreChat also supports HTTP+SSE and STDIO MCP servers, which are either stateful or expensive to create and destroy. Therefore, we need a way to manage these connections in a way that allows us to reuse them across multiple requests or shared between multiple users.

There should be a connections repo for all app-level connections (shared between all users), a connections repo for each user group, and a connections repo for each user. There can be different life-cycle rules for each of these repositories (e.g., app-level connections can be long-lived, while user-level connections can be short-lived or idle for a shorter period of time).

Each connection should be lazy-loaded and cached for a reasonable amount of time (Note that STDIO connections can be quite a memory hog, so we need to be careful about how we manage them). These connections should always try to reconnect if they are disconnected, and the repository must gracefully garbage collect any connections that are no longer needed or have been idle for too long to avoid memory leaks.

Implementation

Here's a laundry list of tasks that need to be done to implement the above:

1. Move the Cache logic from the backend api into its own package so the MCP package can import it.
2. Come up with a way to elect a leader backend server that will be responsible for expensive operations or those that are susceptible to race conditions.
3. Changes to the MCPServersRegistry:
   • save the MCP servers in a cache that can be shared between all backend servers.
   • add the ability to query the list of available MCP servers for a specific user group or user.
   • add the ability to dynamically/periodically discover new MCP servers and update the cache accordingly.
4. Changes to the MCPConnectionsRepository:
   • add more options for life-cycle rules (ephemeral, TTL, idle timeout, etc.).
   • add the ability to gracefully garbage collect idle connections.
5. Add MCPToolBox to the MCP package with the ability to query the list of available tools for a specific user:
   • with caching per user per server, and with TTL.
   • must be aware of the changes in the MCPServersRegistry to add or remove tools as needed.
     6 Consolidate all MCP logic outside the MCP package into the MCP package.

[leblancfg]

Related: the "official" MCP registry was launched a couple weeks ago.

I feel like it is still very early days for that project. But: having dynamic MCP server registration in LibreChat would be the first step in aligning with that direction, and eventually being able to support it.

In the image above, LibreChat of course would be one of the clients, e.g. Cursor

[danny-avila]

Excellent write-up @nhtruong!

This aligns with a lot of discussions I've had on this topic across all the LibreChat channels.

1. Allowing admins to add MCP servers without having to update the librechat.yaml file

This is the main item already in the works. It will be possible to add/edit "pre-defined" MCP servers, first on a role basis and eventually also include user groups, to determine what MCP servers are available to what users from an admin perspective.

What I'm working on applies to all configuration (librechat.yaml) options, which includes mcpServers.

2. Allowing each user to specify their own MCP servers

I think this is a great opportunity to apply the new ACL/permissions system when allowing users to add MCP servers from the UI, so they can share who else gets access in a granular way.

3. Local MCP Server Discovery

This is awesome and looking forward to that!

[mescaja]

Automatic discovery of MCP from public registries into ACME registry as new MCPs are added and obsolete ones get decommissioned will be save significant amount of time in operations and maintenance.

Another question comes to mind is versioning control in ACME registry as those MCP evolve and new version of the same MCP becomes available in the registry, the version that will be available and release to end users to pick and use will be important. I guess an active flag can be used to manage that or even better ACLs to make multiple version for instance available to admin and test users only during certain time before it gets released to wider audience.

I see the ACL/RBAC/Permisioning policies better placed in the ACME registry rather than inside LibreChat. Security implementation in the ACME MCP registry protected by OIDC for instance so when the person logs into LibreChat and access the registry, will only be able to access the MCPs from the ACME registry that he is entitle to.

ACME registry will have corporate MCPs as well as public ones, all managed and protected centrally with LibreChat siting on top consuming them

This way as others in this chat already pointed out, instead of having all the MCPs added to the libechat.yaml file, you'll have just one url pointing to ACME registry. Having a Redis Cache to store temporary MCPs details pulled from ACME one will most certainly make a significant performance gain particularly when high number of concurrent users.

[nhtruong]

Thanks @danny-avila !

The dynamic MCP servers lists will have to rely on Redis a lot to keep things consistent across sessions and across multiple LC backend servers. As you and I have discussed, moving the the cache logic into packages/api is needed to streamline the usage of Redis. I have a PR up to do just that and then some (That PR also converts the Redis JS code to TS, and adds integration tests for the Redis feature).

I'm going to work on expanding the MCPServersRegistry to add

• ability to add/update/delete MCP servers
• ability to query available servers by user.
• caching
  You mentioned that there's already some work in this area, I wonder if we'd step on each other's toe then?

[atefbellaaj-at-slalom]

Excellent work guys!
As some of you know we rolled out Librechat worldwide on a scalable infrastructure supporting a large user base, and we need this feature as well. We’re ready to contribute to its development and help move it forward.

From a use-case perspective, here are the main items we can deliver progressively as releasable packages.

Package 1 (High priority)

• Enable users to add an MCP server via the UI introduced in PR #7800.

• I’m aligned with @nhtruong ’s backend approach: store MCP server configs in a new mcpServers collection and update the cache accordingly.

• Suggestion: protect each mcpServer entry with the ACL collection from the start, mirroring the agent permission pattern.

  • Phase 1: assign the creator as owner automatically.
  • Implement permission checks immediately for update, delete, and query (single/all); defer sharing UX to later phases.

• Tool invocation for an MCP server should not require permissions. Permission filters apply only to the chat badge or when adding available MCP servers to an agent => this reduces complexity when sharing an agent with MCP capabilities.

• Manage MCP servers in a dedicated right-panel section. Provide a shortcut to add an MCP server from the same place where an MCP can be attached to an agent.

Package 2 (Medium priority)

• Add a “share with everyone - Read only Permission Level” toggle. This lets admins make MCP servers available organization-wide.
  • Consider making sharing role-dependent. I’ll open a feature request to support assigning application roles to groups (not only users).
• Support shared MCP servers with sensitive config (e.g., API keys) to make them ready-to-use.
• Add an optional whitelist/restriction list for MCP server domains in librechat.yml.
• Lightweight admin approval workflow (configurable) when users add MCP servers, with an optional webhook to trigger external processes.
  • Background: our infrastructure will need updates to allow outbound traffic to new MCP servers.

Future packages (early ideas)

• Full sharing integration, exactly like agent sharing.
• MCP single sign ON in case Librechat is protected with the same Auth server and Reuse Oauth Token Feature is enabled.
• Explore the local discovery concepts mentioned by @nhtruong.

---

Open questions: Stateless vs. stateful MCP servers

I’m still wrapping my head around stateless vs. stateful MCP server designs. A few questions for the group:

• What are the plans for stateful MCP servers when running multiple containers?
• Are sticky sessions at the load balancer required, or is there a recommended approach (e.g., externalized session/state store)?
  I am not yet very familiar with the protocol, so any pointers are much appreciated.

@danny & @nhtruong any updates on this? Maybe we can sync on a technical approach and split the work to move it forward together. I’m also happy to hop on a quick chat whenever works for you. (I’m based in Munich, though I can be flexible with meeting time.)

Atef Bellaaj [email protected] on behalf of Daimler Truck AG.
Provider & Legal Notice | Daimler Truck
Copyright (c) {2025} Daimler Truck AG

[nhtruong]

Oh nice catch! I initially wanted privateUserServers: Map<string | undefined, ServerConfigsCache> = new Map(); to be its own class but the size of the PR was already getting out of hand so I resorted to using Map and have a follow up to turn that into a first-class object later (the logic is a bit complicated). Totally overlooked the fact that this in-mem Map would cause this issue.

For now the quick fix is when a userId doesn't exist in the map when we wants to access its servers, we will simply create an entry for that user (just like how we created it when we add the first server configs for that user).

[atefbellaaj-at-slalom]

@nhtruong & @danny-avila
I’m following up from this PR.
Some of the comments there made me step back and look at the broader picture. I’d like to get your thoughts on a few points.

From a purely feature perspective, here’s my current understanding of the intended end goal:
1. A user or admin (depending on roles) can create a private MCP server and share it with others.
2. A user can connect a private MCP server to an agent.
3. If that’s the case : what happens when the agent is shared with other users, but the associated MCP server remains private? => Our assumption, is: A user is able to consume the mcp server since they have access to its parent agent. (Similar to files attached to agents)

This make me also think about the new lifecycle of the MCP registry:
1. At App start: publicly shared servers and YAML-defined servers are loaded into the sharedServerCache.
2. At runtime: When a private server is shared or unshared with (EVERYONE), it should move between the shared cache and the private cache accordingly.
Correct?

However, a few things remain unclear:

• How are private servers initially populated? On every page reload, or just once?
  • If they only contain unshared mcp servers, maybe we can load them once (if missing) with a TTL.
• How should we handle MCP servers that are shared between users (directly or through sharing agents that consume them)?
  • Should these be loaded into a cache, or retrieved directly from the database?
  • How do we ensure that MCP servers accessible through shared agents are also properly loaded?
  • If they are cached, how should the cache stay updated given frequent change events.. e.g. agent permission updates, MCP metadata changes, or MCP permission changes?

I will update the PR depending on what you guys thing about the above.

[danny-avila]

What happens now: In order to add an MCP server to an agent, it must be initialized at the time of adding in the agent builder form. For another user, if an MCP server is added to an agent, at the time of chatting with agent, the MCP server is initialized if not already initialized.

If you can get it to the point of the MCP server being added to the Agent, it should work at this point.

1. A user or admin (depending on roles) can create a private MCP server and share it with others.

Yes, and sharing wouldn't necessarily mean initializing the server, until use or adding to agent

2. A user can connect a private MCP server to an agent.

At this point in time, a server is initialized when adding to an agent, if not already initialized, for that user adding to the agent.

3. If that’s the case : what happens when the agent is shared with other users, but the associated MCP server remains private? => Our assumption, is: A user is able to consume the mcp server since they have access to its parent agent. (Similar to files attached to agents)

This is my assumption too.

Hopefully this clears up some. I'm more fuzzy on the caching behavior as I would need to review the current codebase to give a good answer on that.

[atefbellaaj-at-slalom]

Thank you, @danny-avila, for confirming our assumption. I am glad to see we’re aligned on the feature direction.

Proposed Solution

After reviewing the edge cases and trade-offs, here’s a comprehensive approach that should address all concerns:

Public Server Lifecycle

• Sources:
  • YAML configuration
  • Database entries shared with PUBLIC
  • MCP servers added to agents shared with PUBLIC (deduplicated; consider chatMenu: false to hide from the Badge in the chat)
• Load behavior:
  • Loaded into both AppShared and UserShared caches on application startup.

Private Server Lifecycle

• On each refresh, we need to check whether private servers are already populated (INITIALIZED::Status::userId flag).

  • If FALSE:
    • Compute the private server configurations and populate the registry (possibly through the initializer).
    • We may need to lock the cache before adding private servers, or alternatively, make the addition process idempotent.
    • @nhtruong — thoughts on this approach?
  • If TRUE:
    • Simply serve the cached data.

• The following server configurations should be passed to the private server initializer or setter (since we don’t currently inspect the config, it might make sense to introduce a new method, e.g., addPrivateServers(...), in the registry). The list excludes globally shared servers:

  • MCP configurations owned by or directly shared with the user.
  • MCP servers from agents that are directly shared with the user.

• With this approach: MCP servers directly shared between users are duplicated across user caches (each user maintains their own copy), but they remain unique across the three cache scopes: AppSharedServers, UserSharedServers, and PrivateServers[userId].

Migration Between Private and Shared

Cache entries should be migrated when:

• An MCP server is shared or unshared with PUBLIC
• A parent agent containing MCP servers is shared or unshared with PUBLIC

Cache Invalidation / update Strategy

We can leverage Redis SCAN-based pattern deletion to efficiently invalidate / update entries across all users.

• Pattern: MCP::ServersRegistry::Servers::*:serverName
• Removes the target server from all user caches in a single operation
  Or we simply delete all INITIALIZED::Status::userId of the users involved with that mcp server / Agent.

Invalidation (or update) Triggers:
Cache invalidation occurs whenever relevant MCP configuration or permissions change:
1. MCP metadata update (user edits server configuration)
2. MCP permission change (share/unshare with users)
3. Agent permission change (affects transitive MCP access)
4. Agent MCP configuration update (adding or removing MCP servers)
Maybe this is too much to track?

Implementation Details

• Introduce a new ServerConfigsCacheOperations class (follows SRP and remains separate from the factory).

• Add static methods to both ServerConfigsCacheRedis and ServerConfigsCacheInMemory for invalidating a server name across all users.
  (or we create a new class for privateUserServers: Map<string | undefined, ServerConfigsCache>)

  • Redis: use SCAN + DEL pattern-based invalidation
  • In-memory: iterate through privateUserServers map and remove matching entries
  • Update a single server across owners?

• these methods can be used on registry privateServers update / delete.

While this approach increases write load when adding or updating MCP servers, it’s a reasonable compromise given that read operations occur far more frequently.

@nhtruong - I would really appreciate your thoughts on this, especially regarding the caching patterns and Redis considerations.

[atefbellaaj-at-slalom]

@nhtruong @danny-avila I’ve made some revisions to the proposal above.

[MemmoB]

This feature sounds great! Thank you for planning it.

Maybe this is not the right place to raise this topic, I'm sorry if it's misplaced.
But I would like to ask if it would be possible to in this feature also add support for adding a MCP server source - i.e. a URL to load a list of MCP servers from to include in LibreChat, and not only by adding servers manually through the UI or in librechat.yaml, or servers running locally on a user machie, but a list of servers available on an internal network.

In my organization we use Backstage, https://github.com/backstage/backstage, as a service catalog and we would like to going forward turn internal API:s into MCP tools. As an experiment we extended Backstage with MCP server support, so that developers in their individual repositories can define MCP servers in their catalog-info.yaml file. Something along the lines of:

---
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: exp-mcp-server
  title: Experimental MCP Service
  namespace: exp
  description: MCP service
spec:
  type: mcp
  lifecycle: experimental
  owner: group:teams/team-a
  definition: |
    transport: streamable-http
    url: https://exp-mcp.dev.kn.somedomain.se/
    description: Experimental MCP service

This information is periodically indexed by Backstage.

What we then did was to fetch this information from our internal Backstage and included the results in LibreChat.
We extended librechat.yaml with the following entry:

dynamicMCPServers:
  enabled: true
  apiUrl: "https://mcp-lookup-service.somedomain.se/api/mcp-servers"
  format: "registry"
  forceRefreshOnStartup: false
  overrideStatic: true
  authorization:

The lookup-service is a proxy in front of Backstage to fetch all MCP types.
Then we added code to handle fetching the list and showing it in UI.

This is just an example of how a registry could live outside of LibreChat. What would be interesting is to have some kind of plugin system where you can add code to load your list of MCPs in whatever way you want, and then convert it to a format LibreChat understands so it includes it in it's internal cache and in the UI.

Thank you.

[nhtruong]

@MemmoB I've thought of such an use case, too. That will be simple to add once we have all of those puzzle pieces in place.