llama-swap (llama.cpp) provider error: API request failed: tools param requires --jinja flag

[rosmur]

I setup a llama-swap (OpenAI API compatible) endpoint as follows:

"openai_compatible": {
      "Mac Studio llama-swap": {
        "api_url": "http://<url>/v1",
        "available_models": [
          {
            "name": "qwen3-coder:a3b",
            "display_name": null,
            "max_tokens": 128000,
            "max_output_tokens": 32000,
            "max_completion_tokens": 128000,
            "capabilities": {
              "tools": true,
              "images": false,
              "parallel_tool_calls": false,
              "prompt_cache_key": false
            }
          }
        ]
      }
    }

I am getting the error:

API request to http://192.168.86.215:9292/v1 failed: tools param requires --jinja flag

It occurs even if i toggle the tool param to be false.

[coleleavitt]

hi you can either start your server with jinja to have tools set

./server --jinja ...

or

{
  "language_models": {
    "openai_compatible": {
      "Mac Studio llama-swap": {
        "api_url": "http://<url>/v1",
        "available_models": [
          {
            "name": "qwen3-coder:a3b",
            "display_name": null,
            "max_tokens": 128000,
            "max_output_tokens": 32000,
            "max_completion_tokens": 128000,
            "capabilities": {
              "tools": false,               // <--- Disable tools here
              "images": false,
              "parallel_tool_calls": false,
              "prompt_cache_key": false
            }
          }
        ]
      }
    }
  }
}

or can you provide the setup for your server as well thanks!

[rosmur]

Hi,

I did toggle the tools param to false, but I still got the same error.

(I reloaded the window as well.) If this should take care of the issue then let me try a restart, reboot and circle back

[coleleavitt]

you're right there is a bug, I am working on making a patch right now, but let me know if it resolves and resolve this as solved if so

[coleleavitt]

can you test this fork and see if this gives you the same issue?

https://github.com/coleleavitt/zed @rosmur

[Hexorg]

Hey, @coleleavitt I'm not OP, but I had the same issue - that's how I found this thread. Your provided repo - works. I am able to do completions that way.

[coleleavitt]

Hey, @coleleavitt I'm not OP, but I had the same issue - that's how I found this thread. Your provided repo - works. I am able to do completions that way.

Awesome! I got a draft pull request which I will activate for merge after review. Can you create another discussion post and reference this and mark as resolved since OP isn't active

[Hexorg]

@coleleavitt New discussion is at #39126

[coleleavitt]

@coleleavitt New discussion is at #39126

commented, mark answered thanks

[rosmur]

Thanks @coleleavitt do you still need me to test out the fork or is it already in a PR?

[coleleavitt]

Thanks @coleleavitt do you still need me to test out the fork or is it already in a PR?

It's already a PR, you can use the fork in the meantime: #39084

Mark as answered :)