Clarification on Training Data, Licensing, and Building a Commercial Base Model

[ahm3texe]

Hello and thank you very much for this amazing project!

I have a few technical questions regarding the F5-TTS training process:

1. How many total hours of speech data were used to train the F5-TTS base model?

2. Was the training performed only with the Emilia dataset, or were other freely licensed datasets also used (e.g., LibriTTS, LJSpeech, Common Voice, WenetSpeech, etc.)?

3. Since Emilia is licensed under CC-BY-NC (non-commercial), does that mean F5-TTS cannot be used commercially even if we fine-tune it with our own data?

4. I would like to train a base model from scratch for full commercial use.
   Could you please outline the necessary steps and considerations?
   For example:

   • Minimum dataset size required
   • Recommended GPU hardware
   • Estimated training time
   • Any tips for training stability or performance?

Thank you again for your time and for open-sourcing such a great contribution to the community. 🙏

[SWivid]

For Q1,2,4, you could refer to our technical report and also discussion board / issues for successful practice
(If you wish more details than the current ArXiv version, could check https://openreview.net/pdf?id=JiX2DuTkeU)

For Q3, yes CC-BY-NC Emilia trained Base Model cannot be used commercially also after finetuning.

[ahm3texe]

First of all, thank you very much for your quick response.

I have a few more questions regarding dataset preparation and the training process:

1. About the dataset quality: During training, did you include recordings with noise or echo, or did you only use clean, studio-quality recordings?

In your technical report, the following sentence appears. Based on this, can I interpret that you used a mix of recordings?

- We utilize the in-the-wild multilingual speech dataset Emilia (He et al., 2024) to train our base models.

2. For training a base model from scratch, is it sufficient to simply uncheck the “finetune” option in the Gradio interface? Or are there any additional steps needed?

3. After completing the training, is there anything you would have done differently? Was there any aspect that you reconsidered once training was finished?

4. Would having multiple speakers in a single audio clip (even if the voices do not overlap) cause issues during training?

5. If I want to train a base model specifically for Turkish, in your opinion, what would be the minimum number of hours of data required for decent performance?

6. Alternatively, would it be a good idea to pretrain a multilingual model using open-license datasets from various languages and then finetune it specifically for Turkish? Would that lead to better results?

Thank you again for your time and support!

[SWivid]

1. Yes we simply use the raw Emilia dataset, just filter out a few data pairs with ASR hallucination (i.e. wrong transcription).
2. Suggest to use train.py rather than finetune scripts.
3. No additional operations, everything is detailed in our technical report.
4. There actually exists such kind of multi-speaker samples, but not much, so no big influence.
5. Depends on your expectation of inference usage. If in-set inference, 10~100 hrs fine; if zero-shot, more is better, and most importantly ensure the diversity of data (various speakers)
6. Yes, would be better, especially if you want to do zero-shot inference. Could just mix Turkish and other high-quality data (as you want to boost for Turkish) and then finetune on Turkish to improve.

[kyrillosvidor]

1- If I want to train a base model for another language completely from scratch, can I simply uncheck the “Finetune” option in the Gradio interface?

2- Does unchecking “Finetune” mean that the training script will not load the pretrained Emilia dataset or weights, and instead use only my own dataset to build a new model from the ground up?

3- Also, since this new model would not be derived from the Emilia weights or data, would that mean it is not bound by the CC-BY-NC license, and therefore I could license and use it commercially under my own terms

[ahm3texe]

1- Yes, unchecking the "Finetune" option will start the training from scratch without using any pre-trained weights. However, as the maintainer also recommended previously, it is best practice to use the train.py script directly for training a base model from scratch, rather than using the Gradio interface.

2- Yes, your understanding is correct. Unchecking the "Finetune" option means that the training will start from scratch using only the dataset you provide, without loading the weights of an existing model. This way, the training process is independent of the Emilia dataset or any weights derived from it, and a new model is created based entirely on your data.

3- Yes, your conclusion is also correct. As the maintainer stated, the base model trained with the Emilia dataset cannot be used commercially due to its CC-BY-NC license. However, if you train a new model from scratch using your own commercially-permissive dataset—without using any data or weights derived from Emilia—the resulting model will not be bound by the CC-BY-NC license. Therefore, provided the license of your own dataset allows it, you can use this new model commercially and license it under your own terms.