Replies: 43 comments 68 replies
-
|
In the new update, there is an 'experiment' button in the auto train settings. I'm still working on this part so please check setting before start train to make sure all ok for the dataset you use If you encounter memory issues, try using your own settings.
about mult-gpu check al ok about how offer save a checkpoints and last point , note: every checkpoint you need 5G disk space ! About the epoch, the default is now 10 epochs. You may need more or less see what working for you i try make more test and i make some good value soon |
Beta Was this translation helpful? Give feedback.
-
|
Please support Indonesian |
Beta Was this translation helpful? Give feedback.
-
|
What does the Check Vocab step do practically? I'm sorry, but I'm not able to get it; what happens in this part if I want to fine-tune for french or spanish etc.? |
Beta Was this translation helpful? Give feedback.
-
|
This checks all the symbols in the pre-trained model. If any are missing, it will give you a list of the missing symbols. If there are no errors, you can fine-tune it without any issues. Just make sure the project data/project_name/metadata.csv file is in place. Edit : Supported audio formats: format how look like in line 1. i hope this help also check here about finetune #57 |
Beta Was this translation helpful? Give feedback.
-
|
@leoiania for spanish you can train with not problem all symbols wokring i make a quick train test and look like start lear |
Beta Was this translation helpful? Give feedback.
-
|
thank you! for this info and for your work on the gradio interface too! |
Beta Was this translation helpful? Give feedback.
-
|
what you think it's good idea to add info for the system and gpu new tab ?
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this great work. I've managed to fine-tine my first model, but I noob question is how do I test the model whether in cli or gui? |
Beta Was this translation helpful? Give feedback.
-
|
in inference-cli.py to change the model EDIT: make sure you have last update in rp i just make update on this to easy load model EDIT: i create api so now you can simple use make sure you update the main repo make sure you are update the main repo |
Beta Was this translation helpful? Give feedback.
-
|
awesome work are you planning to improver and fix errors? |
Beta Was this translation helpful? Give feedback.
-
|
@lpscr i haven't tested yet actually today learnt but before investing time I wanted to congratulate and ask :D |
Beta Was this translation helpful? Give feedback.
-
|
You must try this! I'm sure you’ll find this amazing repo for high-quality TTS very cool. |
Beta Was this translation helpful? Give feedback.
-
|
@lpscr awesome. last time fine tuning i tried was coqui TTS. so this is better than that you think? |
Beta Was this translation helpful? Give feedback.
-
|
For me, it's the best I've ever seen and my favorite. That's why I also made gradio finetune to make easier to use for beginner users. i have try a lot repo before tts like Coqui TTS, but this repository provides the highest quality TTS I’ve encountered. Try it yourself to see and compare!” |
Beta Was this translation helpful? Give feedback.
-
|
Yeh Doc this is the best |
Beta Was this translation helpful? Give feedback.
-
|
@lpscr |
Beta Was this translation helpful? Give feedback.
-
|
you can use preetrain model to use less data , the only need to have if you use Transcription make all for you automatic only need load audio file or you can place the dataset like this Edit : Supported audio formats: format how look like in line 1. |
Beta Was this translation helpful? Give feedback.
-
|
you should make it so it informs the user if ffmpeg is not available instead of silently failing on the transcribe, this was giving me problems on a runpod instance. |
Beta Was this translation helpful? Give feedback.
-
|
@jazza420 hi thank you for let me know in windows download download https://www.gyan.dev/ffmpeg/builds/ create folder in c: with name ffmpeg run restart and must working fine i hope this help for linux use simple install |
Beta Was this translation helpful? Give feedback.
-
|
hi new update transcribe you can now click button random sample to see how look like the transcribe prepare you can now click button random sample to see how look like the prepare test model |
Beta Was this translation helpful? Give feedback.
-
|
How can I contine training a model? I have the metadata.csv, wavs folder and last_model.pt from spanish model, but I´m not getting good results with my voice, I want to continue training the spanish model to me more accurate with my voice |
Beta Was this translation helpful? Give feedback.
-
|
Can someone explain to me how to practice with languages other than English? |
Beta Was this translation helpful? Give feedback.
-
|
@Bjorjank i still test , to see what best for other langauge , if i success find something good i add in gradio finetune i working on |
Beta Was this translation helpful? Give feedback.
-
|
How many dataset you usually use (how many hours or minutes) to finetune and produce great result?, and I tried it in collab using t4 and needed to lower the batch size per gpu too i guess. Anyway thanks for this easy to use pathway for finetuning F5. |
Beta Was this translation helpful? Give feedback.
-
|
@bastiansurya77 if you click auto setting you must get correct setting about batch size per gpu i dont test in colab |
Beta Was this translation helpful? Give feedback.
-
|
Yeah it automatically provides me the number, but i needed to lower it because it crashing. How about my former question (how many hours or minutes data you usually use to finetune to produce great result, I want to make the results much closer to mine?) |
Beta Was this translation helpful? Give feedback.
-
|
Hello @lpscr, Thank you for sharing this project! I have been following your instructions: After placing WAV files in the newly created project folder and checking user, I clicked the Transcribe button and waited for the process to complete. However, the info field displays “You need to load an audio file.” Could you please help me resolve this issue? Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
@FrnklyN you need to put the file inside to data/project_name/dataset ├── data/ BTW : tomorrow i make big update i change a lot stuff what you like to use to help you more ? |
Beta Was this translation helpful? Give feedback.
-
|
It's is... They are in data/voice_pinyin/database. Or do I need to create a "voice" folder in the data folder? |
Beta Was this translation helpful? Give feedback.
-
no only you need put wave file inside let me check it |
Beta Was this translation helpful? Give feedback.
-
|
@FrnklyN yes you right there is small bug i fix you can simple uncheck the user |
Beta Was this translation helpful? Give feedback.
-
|
@FrnklyN ok update all make sure you have last update repo you can see in first post update video also |
Beta Was this translation helpful? Give feedback.
-
|
usage: finetune_cli.py [-h] [--exp_name {F5TTS_Base,E2TTS_Base}] [--dataset_name DATASET_NAME] Hello @lpscr, It appears that the update for the parameter --file_checkpoint_train has not yet been committed and merged into finetune_cli.py, although it has been merged in finetune_gradio. By the way, is this parameter intended to allow resuming fine-tuning from a previous checkpoint? Could you please provide an example of the exact path to the checkpoint? I assume it would be something like ckpts/project_name/checkpoint.pt, correct? |
Beta Was this translation helpful? Give feedback.
-
|
hi @binhphamthanh in your ckpts/project_name/ In your ckpts/project_name/, if there is no file named For example: When you run the training again, it will continue from where it left off. I also want to update this soon. Essentially, you need to set the last checkpoint to save the model at smaller intervals. I’ll fix this in the next update. Please note that the auto-setting is still experimental. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @lpscr, In the meantime, I identified an issue related to the Pretrain Model textbox in the Train Data tab. This field should be labeled as --pretrain in accordance with the latest finetune_cli.py update. |
Beta Was this translation helpful? Give feedback.
-
|
@binhphamthanh yes i basic you need put the path where the file pt it's and same in gradio finetune use path in checkpoint textbox my_project |
Beta Was this translation helpful? Give feedback.
-
|
To create a public link, set |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Could you please tell me how did you solve this error? |
Beta Was this translation helpful? Give feedback.
-
|
The path |
Beta Was this translation helpful? Give feedback.
-
|
@osmania101 Can you check if you have the latest update repo ? . Did you follow the correct installation steps at the beginning to install |
Beta Was this translation helpful? Give feedback.
-
|
can you send me some line for the metadata.csv you have to check it |
Beta Was this translation helpful? Give feedback.
-
|
audio_segment_1.wav|He is saying that he is equally happy |
Beta Was this translation helpful? Give feedback.
-
|
@osmania101 i dont see any problem |
Beta Was this translation helpful? Give feedback.
-
|
Everything is in the correct path as it should be. It only works when I use the transcribe button, but I don't want to use transcription since I already have the metadata.csv. Why can't it detect my audio files directly, even though they work during transcription? |
Beta Was this translation helpful? Give feedback.
-
|
Did you figure this out? Running into the same problem Edit: @osmania101 I looked at the gradio code and basically it uses torch to measure the duration of your audio files. If the duration isn't found, it assumes there are no files. You might have a bad install. It doesn't use this method for the transcription tab which is why you didn't experience it there. also pip install soundfile==0.12.1 SOLVED: Besides the above, I also noticed the gradio is putting the /wavs/ folder in the directory path for you. So if your data is labeled: it will be imported on the path .../wavs/wavs/audio1.wav so be sure to only include the filename |
Beta Was this translation helpful? Give feedback.
-
|
How can I prepare multi speaker dataset? if i have a multi speaker dataset i should skip speaker id and only keep text and audio pair? |
Beta Was this translation helpful? Give feedback.
-
|
First of all, thank you for your amazing work. I’ve been trying to train a Korean TTS model using F5-TTS (multiple times). Initially, I attempted to train it with a multi-speaker dataset. While there was almost no quality difference between the reference and generated audio during training, the actual inference only managed to copy the voice timbre and failed to read Korean text correctly. Later, I came across your advice in this discussion and tried training again with a single-speaker dataset of about 12 hours. The training settings were as follows: Batch size per GPU: 1600 For reference, my GPU is an NVIDIA GeForce RTX 3080 with 10GB of VRAM. ( yeah, it was hard work ^_T ) Below is the training log. Although the training didn’t complete all the set epochs due to my computer overheating, it reached approximately 300K steps.
The inference results from the trained model are as follows:
In conclusion, I have a few questions for you:
Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
How to train the model? Do you have a video tutorial for this? |
Beta Was this translation helpful? Give feedback.
-
i你好，能请教一下吗，我的GPU都不会怎么被用，但是他报错GPU不足，实际上GPU使用率只有20%，而且我看您的GPU有10，我的只有·8，然后就是我的barch size 的问题了，可以看看您的参数配置吗 |
Beta Was this translation helpful? Give feedback.
-
|
I've never figure it how to train it. Even i tried multiple time the exact order like in the video and other videos. All the time it crashes. So i can't train a new language or any language. |
Beta Was this translation helpful? Give feedback.
-
|
Yes we need explanation. Especially, Indonesian so bad at F5-TTS |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for good work. Everyting works almost fine here, I'm training polish models with success. However, I have a problem with worker processess (maybe a stupid one) - I cannot reduce it from default 16 to 12 (for i7-8700) or 8 (for i7-9700), something is overriding num_workers settings in train.py. This causes problems with bigger datasets. How can I reduce worker number? |
Beta Was this translation helpful? Give feedback.
-
|
thanks for this amazing work!!! I followed the same steps as the video but when started training, I got the following messages. Is there anything that I did wrong? MessagesRuntimeError: Error(s) in loading state_dict for CFM: |
Beta Was this translation helpful? Give feedback.
-
|
Please Help I keep getting this error. I used a couple of audio samples, used the "Transcribe Data" tab, then the "Prepare Data" tab. When I get to the train tab i get this error. run command : The following values were not passed to Word segmentation module jieba initialized. Loading model cost 0.520 seconds. vocab : 2545 vocoder : vocos vocab : 2545 vocoder : vocos |
Beta Was this translation helpful? Give feedback.
-
|
With the help of chatGBT I got this code to use one of my GPUs accelerate launch --mixed_precision=bf16 --num_processes=1 --main_process_port=29500 ^ It seems like it is working but then i get this error I have tried reducing batch size to as low as 128, switching mixed_precision still no luck I have an i9-10900k |
Beta Was this translation helpful? Give feedback.
-
|
hi, you could check |
Beta Was this translation helpful? Give feedback.
-
|
I am new to this. Please how do I do this or is there some resource you could refer me to? I tried running and got this error |
Beta Was this translation helpful? Give feedback.
-
make sure you have proper install pytorch also from your previous error message with gradio
you should check first for cuda and pytorch installation if use prepare dataset first, or modify exisiting scripts to use google_fleurs https://github.com/SWivid/F5-TTS/tree/main/src/f5_tts/train#prepare-dataset then you modify training settings in https://github.com/SWivid/F5-TTS/blob/main/src/f5_tts/configs/F5TTS_Small_train.yaml if you want to train with small model size for example. and you config accelerate and start training with https://github.com/SWivid/F5-TTS/tree/main/src/f5_tts/train#1-training-script-used-for-pretrained-model |
Beta Was this translation helpful? Give feedback.
-
|
Has anyone able to train this on Multiple-4090 GPU's setup (2 or more)?? I am getting this - #728 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Hi When I set the path in Tokenizer File and Path to the Pretrained Checkpoint I get this error: Loading model cost 0.440 seconds. vocab : 2580 vocoder : vocos During handling of the above exception, another exception occurred: Traceback (most recent call last): Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
FileNotFoundError: Directory C:\pinokio\api\e2-f5-tts.git\app\src\f5_tts....\data\my_Language_custom/raw not found "[project name] _custom/raw not found" |
Beta Was this translation helpful? Give feedback.
-
Please guide how to continue the last pre-trained model i rename last check point.pt to last_model.pt but did not work |
Beta Was this translation helpful? Give feedback.
-
|
I have just finished training a Lithuanian model using ~1k hours of multispeaker data using this dataset. The result seems ok, much better than the existing openly available VITS model but still far from perfect. Here’s the settings I used on RTX4090, trained for about 2 days: Can someone please help me gain more insight and improve the model by answering a few questions?
These may be noob questions but I am one in practical machine learning field. Looking forward for answers to improve my knowledge and build the best Lithuanian TTS model out there. |
Beta Was this translation helpful? Give feedback.
-
|
Hi can u tell me, how can I use this in Google colab, bcz I have no GPU, so help me out guys, and I'm not even a coding person so help me to achieve this in Google colab or else can u suggest me some youtube references I can watch it and set this up |
Beta Was this translation helpful? Give feedback.
-
|
I tried to fine-tune a model on the. Indian (Malayalam) Language, and when I tried to input a Malayalam audio. The transcribed text returned Hindi text. I have a metadata.csv in the Malayalam language. Will that work? if i knw where to change the model for transcription i could use one that supports malayalam. or it ppossible to use my metadata which has malaylam text? |
Beta Was this translation helpful? Give feedback.
-
|
transcribe complete samples : 0 |
Beta Was this translation helpful? Give feedback.
-
|
In my case, non zero error files caused by your device has not yet install ffmpeg |
Beta Was this translation helpful? Give feedback.
-
|
Getting the same error as @butterflyoiio "FileNotFoundError: Directory C:\Users\voids\F5-TTS\src\f5_tts....\data\newproject_custom/raw not found" Anyone got a clue what's causing this? Why is it even looking there in an "_custom" folder in a totally different location? I've set up paths for the pretrained model, vocab, etc... P.S. I'm trying to finetune https://huggingface.co/mrfakename/OpenF5-TTS-Base model in English |
Beta Was this translation helpful? Give feedback.
-
|
I have created a simple Python script from the finetune_gradio.py file, which replicates the functionality of a Gradio app for audio generation using a fine-tuned model. The issue I’m facing is that when I generate audio through the Gradio interface, the output sounds perfect. However, when I generate audio using the Python script, the result is completely different and of poor quality. I have verified that all hyperparameters used in the script are identical to those in the Gradio app, yet the generated audio is significantly different. so can you please help me with this? Below is the code snippet for reference: |
Beta Was this translation helpful? Give feedback.
-
|
Hi you should check if you've checked use_ema for the gradio inference. For the api, If I set Initializing F5-TTS model...
Download Vocos from huggingface charactr/vocos-mel-24khz
vocab : F5-TTS/ckpts/mbspeech_finetune_model/vocab.txt
token : custom
model : F5-TTS/ckpts/mbspeech_finetune_model/model_last_pruned.pt
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
[/tmp/ipython-input-2913001611.py](https://localhost:8080/#) in <cell line: 0>()
1 # Initialize TTS API
2 print("Initializing F5-TTS model...")
----> 3 tts_api = F5TTS(
4 model=MODEL_NAME,
5 ckpt_file=CHECKPOINT_PATH,
2 frames
[/content/F5-TTS/src/f5_tts/api.py](https://localhost:8080/#) in __init__(self, model, ckpt_file, vocab_file, ode_method, use_ema, vocoder_local_path, device, hf_cache_dir)
80 cached_path(f"hf://SWivid/{repo_name}/{model}/model_{ckpt_step}.{ckpt_type}", cache_dir=hf_cache_dir)
81 )
---> 82 self.ema_model = load_model(
83 model_cls, model_arc, ckpt_file, self.mel_spec_type, vocab_file, self.ode_method, self.use_ema, self.device
84 )
[/content/F5-TTS/src/f5_tts/infer/utils_infer.py](https://localhost:8080/#) in load_model(model_cls, model_cfg, ckpt_path, mel_spec_type, vocab_file, ode_method, use_ema, device)
270
271 dtype = torch.float32 if mel_spec_type == "bigvgan" else None
--> 272 model = load_checkpoint(model, ckpt_path, device, dtype=dtype, use_ema=use_ema)
273
274 return model
[/content/F5-TTS/src/f5_tts/infer/utils_infer.py](https://localhost:8080/#) in load_checkpoint(model, ckpt_path, device, dtype, use_ema)
223 if ckpt_type == "safetensors":
224 checkpoint = {"model_state_dict": checkpoint}
--> 225 model.load_state_dict(checkpoint["model_state_dict"])
226
227 del checkpoint
KeyError: 'model_state_dict' |
Beta Was this translation helpful? Give feedback.
-
|
After bit tinkering error turned to this: Initializing F5-TTS model...
Download Vocos from huggingface charactr/vocos-mel-24khz
vocab : F5-TTS/ckpts/mbspeech_finetune_model/vocab.txt
token : custom
model : F5-TTS/ckpts/mbspeech_finetune_model/model_last_pruned.safetensors
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[/tmp/ipython-input-2913001611.py](https://localhost:8080/#) in <cell line: 0>()
1 # Initialize TTS API
2 print("Initializing F5-TTS model...")
----> 3 tts_api = F5TTS(
4 model=MODEL_NAME,
5 ckpt_file=CHECKPOINT_PATH,
3 frames
[/content/F5-TTS/src/f5_tts/api.py](https://localhost:8080/#) in __init__(self, model, ckpt_file, vocab_file, ode_method, use_ema, vocoder_local_path, device, hf_cache_dir)
80 cached_path(f"hf://SWivid/{repo_name}/{model}/model_{ckpt_step}.{ckpt_type}", cache_dir=hf_cache_dir)
81 )
---> 82 self.ema_model = load_model(
83 model_cls, model_arc, ckpt_file, self.mel_spec_type, vocab_file, self.ode_method, self.use_ema, self.device
84 )
[/content/F5-TTS/src/f5_tts/infer/utils_infer.py](https://localhost:8080/#) in load_model(model_cls, model_cfg, ckpt_path, mel_spec_type, vocab_file, ode_method, use_ema, device)
270 ).to(device)
271
--> 272 dtype = torch.float32 if mel_spec_type == "bigvgan" else None
273 model = load_checkpoint(model, ckpt_path, device, dtype=dtype, use_ema=use_ema)
274
[/content/F5-TTS/src/f5_tts/infer/utils_infer.py](https://localhost:8080/#) in load_checkpoint(model, ckpt_path, device, dtype, use_ema)
223 if ckpt_type == "safetensors":
224 checkpoint = {"model_state_dict": checkpoint}
--> 225 print(checkpoint["model_state_dict"])
226 model.load_state_dict(checkpoint["model_state_dict"])
227
[/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in load_state_dict(self, state_dict, strict, assign)
2579
2580 if len(error_msgs) > 0:
-> 2581 raise RuntimeError(
2582 "Error(s) in loading state_dict for {}:\n\t{}".format(
2583 self.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for CFM:
Missing key(s) in state_dict: "transformer.time_embed.time_mlp.0.weight", "transformer.time_embed.time_mlp.0.bias", "transformer.time_embed.time_mlp.2.weight", "transformer.time_embed.time_mlp.2.bias", "transformer.text_embed.text_embed.weight", "transformer.text_embed.text_blocks.0.dwconv.weight", "transformer.text_embed.text_blocks.0.dwconv.bias", "transformer.text_embed.text_blocks.0.norm.weight", "transformer.text_embed.text_blocks.0.norm.bias", "transformer.text_embed.text_blocks.0.pwconv1.weight", "transformer.text_embed.text_blocks.0.pwconv1.bias", "transformer.text_embed.text_blocks.0.grn.gamma", "transformer.text_embed.text_blocks.0.grn.beta", "transformer.text_embed.text_blocks.0.pwconv2.weight", "transformer.text_embed.text_blocks.0.pwconv2.bias", "transformer.text_embed.text_blocks.1.dwconv.weight", "transformer.text_embed.text_blocks.1.dwconv.bias", "transformer.text_embed.text_blocks.1.norm.weight", "transformer.text_embed.text_blocks.1.norm.bias", "transformer.text_embed.text_blocks.1.pwconv1.weight", "transformer.text_embed.text_blocks.1.pwconv1.bias", "transformer.text_embed.text_blocks.1.grn.gamma", "transformer.text_embed.text_blocks.1.grn.beta", "transformer.text_embed.text_blocks.1.pwconv2.weight", "transformer.text_embed.text_blocks.1.pwconv2.bias", "transformer.text_embed.text_blocks.2.dwconv.weight", "transformer.text_embed.text_blocks.2.dwconv.bias", "transformer.text_embed.text_blocks.2.norm.weight", "transformer.text_embed.text_blocks.2.norm.bia...
Unexpected key(s) in state_dict: "ema_model.transformer.input_embed.conv_pos_embed.conv1d.0.bias", "ema_model.transformer.input_embed.conv_pos_embed.conv1d.0.weight", "ema_model.transformer.input_embed.conv_pos_embed.conv1d.2.bias", "ema_model.transformer.input_embed.conv_pos_embed.conv1d.2.weight", "ema_model.transformer.input_embed.proj.bias", "ema_model.transformer.input_embed.proj.weight", "ema_model.transformer.norm_out.linear.bias", "ema_model.transformer.norm_out.linear.weight", "ema_model.transformer.proj_out.bias", "ema_model.transformer.proj_out.weight", "ema_model.transformer.rotary_embed.inv_freq", "ema_model.transformer.text_embed.text_blocks.0.dwconv.bias", "ema_model.transformer.text_embed.text_blocks.0.dwconv.weight", "ema_model.transformer.text_embed.text_blocks.0.grn.beta", "ema_model.transformer.text_embed.text_blocks.0.grn.gamma", "ema_model.transformer.text_embed.text_blocks.0.norm.bias", "ema_model.transformer.text_embed.text_blocks.0.norm.weight", "ema_model.transformer.text_embed.text_blocks.0.pwconv1.bias", "ema_model.transformer.text_embed.text_blocks.0.pwconv1.weight", "ema_model.transformer.text_embed.text_blocks.0.pwconv2.bias", "ema_model.transformer.text_embed.text_blocks.0.pwconv2.weight", "ema_model.transformer.text_embed.text_blocks.1.dwconv.bias", "ema_model.transformer.text_embed.text_blocks.1.dwconv.weight", "ema_model.transformer.text_embed.text_blocks.1.grn.beta", "ema_model.transformer.text_embed.text_blocks.1.grn.gamma", "ema_model... |
Beta Was this translation helpful? Give feedback.
-
|
Yes, @MendeBadra, I am facing the same issue. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know if this helps with your situation, but I had a similar problem where I needed to turn off EMA in order for the model to pronounce words correctly. I am using "f5-tts_infer_cli" to process a large number of files, and there is no switch to turn EMA off. Since I want it turned off all the time, I disabled it in the code. In src/f5_tts/infer/infer_cli.py, it loads the model at line 294: Change that to: Now when I run "f5-tts_infer_cli", it does not use EMA and the words are all pronounced correctly. So my batch processing is working properly. Don't know if that will help with your situation, but wanted to document this in case anyone else has similar issues with needing to turn off EMA for batch processing using the CLI version. |
Beta Was this translation helpful? Give feedback.
-
|
Oh that's great. I will test that on my work computer tomorrow. |
Beta Was this translation helpful? Give feedback.
-
|
how am I supposed to use the pruned model, I am not able to browse it in different models in the custom setting under the choose TTS model. |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello everyone!
I have created a Gradio application for easy fine-tuning and training of models. You can find it here:
https://github.com/lpscr/F5-TTS
EDIT : this merge in main repo
NEW: with new version all now automatic you can simple easy finetune anylangaugewith just simple clickHere's a new complete video step by step, there is sound!:
Please make sure the video is not muted click the speaker icon in video ! . enjoy ;)
amazing.tutorial.mp4
BTW :
The man voice in the video was created using
f5tts!you can get from here https://github.com/SWivid/F5-TTS/tree/main/src/f5_tts/infer/examples/basic
note for the new language
Before you start, if you are going to fine-tune a new language, you will need a substantial amount of dataset hours! As I see here, you can fine-tune a single voice with just 10 to 15 hours. but for multiple speakers, you’ll need more—about 50 hours to start. If you want a good model, aim for at least 100 hours; for something perfect, aim for at least 300 hours or more. See what works for you; it might also be possible to achieve good results with fewer hours in your case.
also mentioned success with just 10-15 hours of fine-tuning for one or two voices. follow langauge
Spanish,Indian (Malayalam) with extend tokens,Hungariannote for the English or Chinese
Regarding English or Chinese, if you want to fine-tune a speaker, first check if it's already working because the model is good enough,
and you may not need to fine-tune the speaker. You can test with 2 to 5 hours or more and see what works.
Please share any experiments or results about what works and what doesn’t, so that others can know as well
quick start
first create new project then see what you need
1 . Transcribe Data Option: Skip this if you already have a
metadata.csvandwavsfolder.You can simply click the audio button to open Explorer and select one or multiple audio files.
If you check the
audio from path, you need to place all audio files indata/my_speak/datasetyou can click button random sample to see text and audio
2 . Vocab Check Option: Use this only when you want to train a new language.
If you need to extend the vocab, you can easily click "Check Vocab"
to see all missing symbols or write your symbols like
a,b,c,detc.If you click "Extend," this creates a new
model_1200000.ptandvocab.txtfile:3 . Prepare Data Option: Skip this if you already have
raw.arrow,duration.json, andvocab.txt. You can click the random sample button to see token and audio.If you have the files
raw.arrow,duration.json, andvocab.txt, make sure they are in the correct path:in case you skip the Transcribe , place your dataset (
wavsfolder andmetadata.csvfile):Supported audio formats:
"wav", "mp3", "aac", "flac", "m4a", "alac", "ogg", "aiff", "wma", "amr".format how look like in
metadata.csvline 1.
audio1|text1oraudio1.wav|text1oryour_path/audio1.wav|text1line 2.
audio2|text2oraudio2.mp3|text2oryour_path/audio2.mp3|text2...
Click "Prepare" to create
raw.arrow,duration.json, andvocab.txt.4 . Train Data:
auto setting button this give you best results but you need check if all ok
If you encounter memory issues, try using your own settings.
barch size per gputo lower numberabout how offer save a checkpoints and last point ,
save per updatesey something working for youlast per step# use smaller value for to save model_last.pt more offer like this when crash train or stop you can easy continue where you leftnote: every checkpoint you need 5G disk space !
About the
epoch, the default is now 10 epochs. You may need more or less see what working for youWhen the model trains, you get sample audio every few steps to see how well the model is doing. Click the refresh button or check in the path
ckpts/my_speak/samplefolder.5 . Test Model: Testing your model is simple and easy. Check use_ema to be True or False to see what works best for you.
when you run the train the test model working in
cpumode ! you need stop the train to run ingpuClick the 'Random Sample' button to view get a text and audio. for dataset
You can compare reference (ref) and generated (gen) audio, enter text in 'gen_text,' or load a new reference in 'ref_text.'
To load your audio reference, click the 'X' button. If the ref text is empty, it will automatically transcribe.
6 . Reduce Model Size: You can reduce the model size from 5GB to 1GB.
you find check point to
you see all automatic now ;)
Beta Was this translation helpful? Give feedback.
All reactions