Faster-Whisper-XXL Pro

[Purfview]

Special non-public version for the donators who donated >= £50.

PayPal: https://www.paypal.com/donate?hosted_button_id=JF5BEQE3YQGH2
BuyMeaCoffee: https://buymeacoffee.com/purfview

Includes some new advanced features, here is the log:

r1.xxx.x

New voice extraction options [ --ff_vocal_extract ]:
mdx1_kim2 - Reworked "mdx_kim2" for better quality and to use much less RAM.
mdx2_kim2 - Different implementation of "mdx1_kim2", a little bit better quality.
mb-roformer - The best, SOTA quality, way better than anything.

--roformer_overlap - Lower value - bigger overlap. Bigger overlap - a bit higher quality, but a lot slower.
--roformer_vram - Set VRAM size to fit long audios. Useful if you have low RAM too.

r2.xxx.x

--realign - Very fast, simple realignment, makes the timestamps of .srt more accurate. Works at ~sentences level.
--realign_device - Device for "--realign" [automatic].

r3.256.1

Fixed: --realign "UnicodeDecodeError" error.
Fixed: --realign error if there is more than one line per sub.
Fixed: --realign "File not found" error.
Fixed: ImportError: libctranslate2-d3638643.so.4.4.0 [Linux]
Fixed: ffmpeg: error while loading shared libraries [Linux]
Hide: "not compiled with flash attention" warning.
Fixed: "cuda:1" didn't work with MDX [untested, "r245.3" fix was broken]
New VAD models: silero_v6_fw, silero_v6 [patched], nemo_v2, ten [its threshold is offset by +0.2]
Change: --vad_method default is set to ten
Change: Batched mode has few improvements
Change: --compute_type defaults to default if CUDA, else auto.
Updated: torch to 2.8.0+cu128 [to support 50xx GPUs on torch models]
Updated: onnxruntime_gpu to 1.21.1 [cuDNN 9.x]
Updated: pyinstaller to 6.16.0

NOTE: Please open a new thread if you've issue or questions/discussions, thank you.

Questions/Discussions: https://github.com/Purfview/whisper-standalone-win/discussions/new?category=q-a
Issues: https://github.com/Purfview/whisper-standalone-win/issues/new

[GOvEy1nw]

Ahh that's cool! Just donated via the coffee link :)

[Purfview]

Thank you. Sent you a message.

BTW, no need to notify me here, I get notifications when a donation is made.

[bbqrooster]

I am interested in getting Faster-Whisper-XXL Pro. What information should I include in the Paypal donation? What happens after the donation?

[Purfview]

What information should I include in the Paypal donation?

Add "Faster-Whisper-XXL Pro" into a message, add a notice if you want Linux build.

What happens after the donation?

After the donation wait for my email.

[bbqrooster]

Just donated through Paypal. Will wait for your email.

[Rootduk]

Donated trough PayPal, not sure if it will work within my project but the work only spend on this repo deserves it to be donated. Will await your email and test if it fits my project. Thanks a lot for your effort!

[Purfview]

Thank you. What's your project about?

[Rootduk]

Just for hobby :). To recognize Dutch voice input from a player that will be converted to text and shall be used as input for the response from a (Dutch) AI text model and after all will be converted to audio again, pfiewww....

[wennew82]

Hi, may i know the advantages of using the pro?
is it will make the translation faster than using non pro?
what else? it's good to have some review so more people more interest with the pro, thanks

[Purfview]

If your audios have heavier background noise then Pro with new options can do better, and other things.
Look at the first post, everything is there.

And your financial support for the project enables me to dedicate time to developing it.

[wennew82]

"use much less RAM "
how different the RAM that will be use compare with non pro?

but how about the time ? is it will faster using the pro?

[Purfview]

"use much less RAM " how different the RAM that will be use compare with non pro?

For example, with non-pro you wouldn't be able to process longer than 90 mins audios on 12 GB VRAM, when Pro can do ~180 min on same memory.

but how about the time ? is it will faster using the pro?

No.

[BlackDamag3]

Can I use the Faster-Whisper-XXL Pro version with the subtitle editing program?

[Purfview]

Yes.

[ethansy1995]

I have received the email notification for "Faster-Whisper-XXL Pro". I'd like to ask, how can I download the latest version if there are future updates?

[Purfview]

You'll get a new notification to the same email.

[federicodalzotto]

You simplified my life and improved my work of 90% so donated £50!
For PRO version for Windows

[federicodalzotto]

I missed the instructions about how to obtain the PRO version for Windows. Could you explain me?

[Purfview]

Didn't you get the email? Check your spam folder.

[federicodalzotto]

Oh yeah yeah! It was in the email associated to my PayPal account...not the associated to github!
Thank you!

[apartmandairesi]

Do you have a Pro version for MacOS or only for Windows? Thanks.

[Purfview]

No. Only Windows & Linux.
MacOS is possible, but it doesn't have CUDA, so it would work only on CPU.

[alex65535]

When generating subtitles, some timestamps are not accurate enough, especially for short sentences, especially for single-word sentences, where the start timestamp lags behind. Can the Pro version fix this problem? Can it accurately match the timestamp of each sentence?

[Purfview]

It can.
Email me your audio and I can send you subs made with Pro version for comparison.

[yf94453]

How does the RAM usage of "mb-roformer" compare with non-pro "mdx_kim2"?

[Purfview]

I tried to get numbers using --device cpu but that was ignored and cuda used anyway

That's only for the whisper models. I need to update help...
For voc extraction use --voc_device cpu.

the task manager

Some tool tracking max mem usage by the app would be better, as there can be short high mem usage spikes.
Check this: Timer_v14.zip

[subgrinder]

Yes - there were flashes of spikes - I only gave the "steady state" values as there was no way to know if the spikes were peak or somewhere between.

Thank you for the timer programs. Only max for overall run is provided after completion so here they are.

Using timer64 (with cuda):

................................virtual mem ........ physical mem
mb-roformer .... 16.5Gb .................. 10.6Gb
mdx_kim2 ............ 30.7Gb ................... 25.1Gb

The time to run using cpu is too long for my "interactive machine". Did not think of it until now: I have a server that is currently shutdown that has 64Gb mem, no GPU (normally used as game server for family/friends: all on hiatus giving me time to get subtitles for old shows I want to watch). Looking at our agreement on PRO, it appears I can install on multiple machines for my own use so I am going to copy PRO and non-PRO to that server and run the tests on cpu. Will be a day or two before that info is available.

[Purfview]

mb-roformer on cpu is very slow, doubt that anyone would use that, better test "mdx1_kim2" and "mdx2_kim2" on gpu.

[subgrinder]

yf94453 did not indicate with or without GPU so I ran both ways to get all results
to answer the question posted.

mb-roformer on cpu is very slow, doubt that anyone would use that, better test "mdx1_kim2" and "mdx2_kim2" on gpu

I recorded times for the runs made after the first two but did not re-run those to
get run times. Indeed, mb-roformer on CPU is very slow.

The mdx1_kim2 and mdx2_kim2 runs were made and results included here.

There are two runs on CPU with mb-roformer as I had copied all files from main
machine to the server and forgot to remove the --roformer_vram argument. The
first mb-roformer run broke the audio file into 11 parts as it had on the GPU run.
Interestingly, it made a significant difference in memory usage between the two
mb-roformer runs while the run times are not much different.

Audio file length 3:26:27

Headless server for CPU runs has 64Gb RAM;
......Processor is AMD Ryzen 9 7900X 4701Mhz

GPU computer has 64Gb RAM, NVidia 3080Ti with 12Gb VRam;
......Processor is Intel i9-12900K 3187Mhz

Model.................CPU/GPU........Virtual Memory.....Physical Mem.....Run TIme

mdx_kim2...........GPU........................30.7Gb...................25.1Gb..................xxxxxx
mb-roformer....GPU(vram 4)......16.5Gb...................10.6Gb..................xxxxxx
mdx_kim2...........CPU........................25.5Gb..................24.9Gb..................02:56:54
mb-roformer....CPU(vram 4)......14.5Gb...................11.8Gb...................10:24:02
mb-roformer....CPU.......................29.9Gb....................27.8Gb..................10:33:28

mdx1_kim2.........GPU........................18.3Gb...................12.0Gb...................00:22:29
mdx2_kim2.........GPU........................16.2Gb...................15.8Gb..................00:19:51

[Purfview]

Interestingly, --roformer_vram made a significant difference in memory usage

Yes, it affects ram usage too, as written in the help. It's called "vram" because its values are tuned to vram size.

[steipal]

What does the future of the standard free XXL version look like? I mean, will it get any TLC or will all efforts and new features/improvements only go into the Pro version?

[Purfview]

I mean, will it get any TLC or will all efforts and new features/improvements only go into the Pro version?

Most new features/improvements will go only to Pro version. Regular version will be maintained and improved too.