Current selected LLM not persistent after clicking "New Chat"

[JordiHigueraDT]

What happened?

Before: Without using Model Specs and selecting an LLM from the endpoints list, whenever the user clicked on "New Chat" a new chat within the same endpoint was starting which is very useful for the users.

Now after Model Specs update: While using Model Specs and selecting an LLM from the endpoints list, whenever the user clicks on "New Chat" a new chat appears but the endpoint is not persistent and the UI shows "select a new Model"

I think it is due to a UI error. If you text with "select a new Model" the previously selected endpoint keeps answering you

This does not happen with Agents if prioritize: false

Here the shortened .yaml config that we use:

modelSpecs:
 enforce: false
 prioritize: false
 addedEndpoints:
   - agents
 list:
   - name: "bedrock-model-router"
     label: "bedrock auto mode"
     default: false
     description: "Routes requests to best Bedrock model"
     iconURL: "bedrock"
     group: "bedrock"
     preset:
       endpoint: "bedrock"
       model: "arn:aws:bedrock:eu-central-1:034362058238:default-prompt-router/amazon.nova:1"
       promptPrefix: "Current Date & Time: {{current_datetime}}. My name is {{current_user}}."
   - name: "eu.amazon.nova-pro-v1:0"
     label: "amazon.nova-pro"
     default: false
     description: "High-end Bedrock model for advanced tasks"
     iconURL: "bedrock"
     group: "bedrock"
     preset:
       endpoint: "bedrock"
       model: "eu.amazon.nova-pro-v1:0"
       promptPrefix: "Current Date & Time: {{current_datetime}}. My name is {{current_user}}."
   - name: "eu.amazon.nova-lite-v1:0"
     label: "amazon.nova-lite"
     default: false
     description: "Balanced performance Bedrock model variant"
     iconURL: "bedrock"
     group: "bedrock"
     preset:
       endpoint: "bedrock"
       model: "eu.amazon.nova-lite-v1:0"
       promptPrefix: "Current Date & Time: {{current_datetime}}. My name is {{current_user}}."
   - name: "eu.amazon.nova-micro-v1:0"
     label: "amazon.nova-micro"
     default: false
     description: "Lightweight, cost-efficient Bedrock model"
     iconURL: "bedrock"
     group: "bedrock"
     preset:
       endpoint: "bedrock"
       model: "eu.amazon.nova-micro-v1:0"
       promptPrefix: "Current Date & Time: {{current_datetime}}. My name is {{current_user}}."
   - name: "anthropic-model-router"
     label: "anthropic auto mode"
     default: false
     description: "Routes to optimal Anthropic Claude model"
     iconURL: "anthropic"
     group: "anthropic"
     preset:
       endpoint: "bedrock"
       model: "arn:aws:bedrock:eu-central-1:034362058238:default-prompt-router/anthropic.claude:1"
       promptPrefix: "Current Date & Time: {{current_datetime}}. My name is {{current_user}}."

Version Information

3d1cedb

Steps to Reproduce

1. Select an Endpoint
2. Click on "New Chat"
3. The Endpoint is not visible anymore in the UI

What browsers are you seeing the problem on?

Safari

Relevant log output

.

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Prioritize must be set to true

[JordiHigueraDT]

Thanks @danny-avila. When we had it set to “true” the issue was then happening with the Agents. Selecting an Agent and clicking “new chat” was sending you back to the Endpoint so there was no way to start a new chat with the Agent

[danny-avila]

I think this might be more of an issue with agents then, will take a look soon

[JordiHigueraDT]

Hey @danny-avila, any update here? How can we support you? We cannot release the Model Prompt Routing as we are blocked by this bug.

Thank you!

[danny-avila]

This is addressed here: #10258

The issue more specifically was that model specs could not be selected as the "last conversation setup" used to apply to a new chat, when a model spec was last selected/used.

Previously, this behavior would only be observed with prioritize: true. This would seem to create an issue with agents by always prioritizing a model spec instead of the last selected option, the agents in this case.

Now, prioritize: false can remain as such and allow using last selected setup, whether spec or not, to be used.

[JordiHigueraDT]

Thank you Danny! That worked wonders :)