webui: add server info to chat message

[am17an]

Fixes #13570. Tested this change out locally, this a model card similar to the timing card, also added a similar checkbox to toggle this behavior.

Also checked if it has getting added to the IndexedDB

I purposefully did not store the whole serverProps object because it was quite large

[ngxson]

As said in the last comment, I think the web UI code reached a state where we should put more thinking about edge cases and the UX, rather than "just make it works"

[ngxson]

use <ul> <li> instead

[am17an]

I did it this way to maintain homogeneity with the other dropdown. Let me if we should still use <ul> <li> here

<div className="dropdown-content bg-base-100 z-10 w-64 p-2 shadow mt-4">
                    <b>Prompt</b>
                    <br />- Tokens: {timings.prompt_n}
                    <br />- Time: {timings.prompt_ms} ms
                    <br />- Speed: {timings.prompt_per_second.toFixed(1)} t/s
                    <br />
                    <b>Generation</b>
                    <br />- Tokens: {timings.predicted_n}
                    <br />- Time: {timings.predicted_ms} ms
                    <br />- Speed: {timings.predicted_per_second.toFixed(1)} t/s
                    <br />
                  </div>
                  

[ngxson]

This is quite fragile for 2 reasons:

• If user changes model but forget to reload the frontend, this serverProps object will be out of sync
• This design is not future-proof. If we decide to add a new key into serverProps in the future, it will be quite messy. The serverProps object is not supposed to be saved an reuse later

[am17an]

@ngxson thanks for your review!
getServerProps is only called once on mount, so at the moment there is no way to get these in sync in case the backend changes but the frontend has not reloaded. One imprecise solution is to periodically poll the backend /health API and reload once the server comes back online. Let me know if that works
In case of future proofing the design, do you wish to de-couple these things? I think it might still be good to use the serverProps object to read these values but store them as separate types in the message.

[ngxson]

One imprecise solution is to periodically poll the backend /health API and reload once the server comes back online

We can simply call /props each N seconds to get the up-to-date server props. But this requires some thinking about the UX

[am17an]

How about a toast when we receive new server props which are not the same from before? However, we will again have to inspect object properties to really get the LlamaCppServerProps out of there, we can't do old_object_json == new_object_json. There is really no nice way to say right now that we just want LlamaCPPServerProps from this bigger json object without listing out all the properties