Feat: Configure LLM parameters (temp, top_p) per model/agent profile

[lefrog-dont-code]

Description

Zed's language model configuration currently does not support specificying model parameters like temperature and top_p for inference providers. This limits users from fine-tuning model behavior for specific tasks (like code generation where lower temp is preferred vs documentation vs one purpose agents for like git mcp tool, etc...)

Having the ability to compose our own agent capabilities (tools) in Zed is amazing, but model parameters are paramount to this paradigm.

Use case

Configuring model parameters per task or agent profile significantly enhances the quality and relevance of AI-generated content within Zed. Different scenarios benefit from distinct generation strategies:

1. Creative Content, Documentation: For drafting documentation, brainstorming, or generating explanatory text, users need to set higher temperature and suitable top_p values to foster more diverse and engaging outputs.

2. Specialized Single-Purpose Agents: When configuring custom AI agents for specific tasks (e.g., a "Git Commit Message Pro," "Code Linter Rule Explainer"), fine-grained control over parameters like temperature, max_tokens, repetition_penalty and top_k for that agent's profile would ensure the agent consistently produces tailored, concise output adhering to specific constraints, making it a reliable and efficient specialized tool.

3. Controlling Verbosity and Focus: For quick queries or summaries, users benefit from adjusting max_tokens to control output length or repetition_penalty to discourage rambling.

4. Experimentation and Provider-Specific Optimizations: Advanced users or developers integrating new models need to specify provider-specific parameters. This allows full leverage of the chosen model/provider capabilities.

Without this granular control, users are subject to default parameters, leading to sub-optimal results for many common development, writing, and specialized agent workflows that Zed aims to support.

Expected Behavior:
Zed should probably:

1. Forward all configured model parameters to the API, allowing per-model customization of parameters like:
   temperature repetition_penalty top_k top_p etc...

2. Allow for provider specific configuration objects, like:

"available_models": [
{
          "name": "qwen3-235b",
          "display_name": "Qwen 3 235b",
          "max_tokens": 32768,
          "venice_parameters": {
            "include_venice_system_prompt": false  <-- this is specific to the inference provider I'm using
          },
          "temperature": 0.1,

Proposed Solution:

1. Expose all available OpenAI API parameters in Zed's model configuration json
2. Ensure all configured parameters are preserved in API requests
3. Allow per-agents custom profile parameter overrides for task-specific tuning (including system prompt?)

[a3lem]

Yes please! Inference settings make a huge difference, especially with smaller local LLMs.

[captainvera]

👍
some openai-compatible providers set their own defaults on some of the inference parameters like max_output_tokens making them virtually unusable with Zed's agentic mode.

It would be great if we could simply pass our own configuration of those parameters.
Zed doesn't need to do any type checking, just allow forwarding of parameters to the API request.

[cyb3rko]

We really need this! Otherwise for example the thinking process or reasoning models if included in the output without any formatting.