How do I set up the parameters for f5-tts for OpenAI API?

[nomonkeynodeal]

If this even works, I guess I would have to set the preset. But I don't understand what the parameters would be. How would I specify a sound sample?

[rsxdalv]

You can do it directly via extra params or via the global presets. What are you using to call it?

I tried to generate an explanation with chatgpt...

You just pass your audio sample into ref_audio_orig. It can be a path to a .wav file (like "sample.wav") or the waveform itself if you’ve already loaded it. That’s the clip the model uses to pick up the voice/style you want.

ref_text should be the exact text that was spoken in that reference clip. So if your sample says “Hello there,” set ref_text = "Hello there". The model needs that to match the sounds in the audio to the right phonemes.

Reference - the function signature:

def infer(
    ref_audio_orig,
    ref_text,
    gen_text,
    model,
    remove_silence,
    cross_fade_duration=0.15,
    nfe_step=32,
    speed=1,
    show_info=gr.Info,
):

[nomonkeynodeal]

I spent a while trying to make this happen. Literally even the AI begged for documentation.

NB. I stuck the voice sample I want to use in a readily accessible folder and checked 1000 times that it is indeed accessible.

I asked Claude to summarize what we tried (probably better to skip the the "suspected problem" part):

F5-TTS OpenAI-Compatible API Troubleshooting Summary

Environment

• Docker: ttsweb-ui running F5-TTS
• OpenAI-compatible endpoint: http://ttswebui.lan:7778/v1/audio/speech
• Gradio UI endpoint: http://ttswebui.lan:7770/gradio_api/ (working)
• Reference audio file: /app/tts-webui/checkpoints/voices/paget.mp3 (existence confirmed)

Goal

Perform voice cloning with F5-TTS using the OpenAI-compatible REST API endpoint with custom reference audio.

What We Tried

Attempt 1: Standard parameters at root level

{
  "model": "f5-tts",
  "text": "Hello, I am a synthetic voice!",
  "ref_text": "...",
  "ref_audio": "/app/tts-webui/checkpoints/voices/paget.mp3",
  "remove_silence": false,
  "voice": "default",
  "device": "auto"
}

Error: Field required: input

Attempt 2: Changed text to input

{
  "model": "f5-tts",
  "input": "Hello, I am a synthetic voice!",
  "ref_text": "...",
  "ref_audio": "/app/tts-webui/checkpoints/voices/paget.mp3",
  ...
}

Error: TypeError: infer_decorated() missing 4 required positional arguments: 'ref_text', 'text', 'model', and 'remove_silence'

Attempt 3: Minimal OpenAI-standard request

{
  "model": "f5-tts",
  "input": "Hello, I am a synthetic voice!",
  "voice": "default"
}

Error: Same TypeError: infer_decorated() missing 4 required positional arguments

Attempt 4: Using params object for F5-TTS parameters

{
  "model": "f5-tts",
  "input": "Hello, I am a synthetic voice!",
  "voice": "paget",
  "params": {
    "ref_text": "...",
    "ref_audio": "/app/tts-webui/checkpoints/voices/paget.mp3",
    "remove_silence": false,
    "text": "Hello, I am a synthetic voice!"
  }
}

Error: TypeError: infer_decorated() missing 1 required positional argument: 'model'

Attempt 5: Added model to params

{
  "model": "f5-tts",
  "input": "Hello, I am a synthetic voice!",
  "voice": "paget",
  "params": {
    "model": "f5-tts",
    "ref_text": "...",
    "ref_audio": "/app/tts-webui/checkpoints/voices/paget.mp3",
    "remove_silence": false,
    "text": "Hello, I am a synthetic voice!"
  }
}

Error: [Errno 2] No such file or directory: 'Hello, I am a synthetic voice!'

Attempt 6: Removed text from params

{
  "model": "f5-tts",
  "input": "Hello, I am a synthetic voice!",
  "voice": "paget",
  "params": {
    "model": "f5-tts",
    "ref_text": "...",
    "ref_audio": "/app/tts-webui/checkpoints/voices/paget.mp3",
    "remove_silence": false
  }
}

Error: TypeError: infer_decorated() missing 1 required positional argument: 'text'

Suspected Problem

The OpenAI-compatible /v1/audio/speech endpoint appears to have an incomplete or buggy implementation for F5-TTS voice cloning. Specifically:

1. Parameter mapping issue: The endpoint expects OpenAI-standard parameters (model, input, voice) but the underlying F5-TTS infer_decorated() function requires different parameters (model, text, ref_text, ref_audio, remove_silence)

2. Inconsistent parameter passing: When parameters are provided in the params object, they appear to be passed incorrectly to the underlying function, sometimes in the wrong order or with incorrect names, causing the API to treat text strings as file paths

3. Voice preset configuration not working: Creating a voice preset in the docker config (e.g., f5_tts_paget) doesn't resolve the issue - the endpoint still fails to pass parameters correctly

4. Working alternative exists: The Gradio UI at port 7770 successfully performs voice cloning using a different API structure (array-based parameters with fn_index)

Gradio API Working Example (for reference)

The Gradio UI successfully sends requests in this format:

{
  "data": [
    {"path": "/tmp/gradio/.../audio.mp3", ...},
    "ref_text transcription",
    "text to synthesize",
    "{\"type\": \"F5-TTS_v1\", ...}",
    false,
    0.15,
    32,
    1,
    "4158050146"
  ],
  "fn_index": 85
}

OpenAPI Schema Reference

According to /openapi.json, the endpoint expects:

• Required: model, input, voice
• Optional: response_format, speed, instructions, params (object), stream

The params object is documented as "Additional parameters for the TTS engine" but its structure for F5-TTS voice cloning is not documented.

Recommendation

The OpenAI-compatible endpoint needs either:

1. Proper documentation showing the exact params structure for F5-TTS voice cloning
2. A fix to the parameter mapping between the OpenAI-compatible wrapper and the underlying F5-TTS inference function
3. Support for voice presets that include ref_audio and ref_text configurations