model large: CUDA fails with out of memory error

[Moritzz00]

Hey, I have a ~1 hour file that I want to transcribe, but after 20 minutes of running Faster-Whisper XXL with --model large cuda fails with out of memory error. Is there any way to reduce or limit the memory usage? I'm also open for other troubleshooting suggestions!

[Purfview]

What is your GPU model and VRAM size?

[Moritzz00]

I am using a NVIDIA GeForce GTX 1050 Ti and have 4 GB of VRAM

[Purfview]

Try --best_of 1 setting.

[Moritzz00]

Okay that worked, thank you! I also just tried using --model medium and that worked with similar results.

[Purfview]

Try to increase it to find highest value that fits into 4GB. [the default is 5]

[Purfview]

I also just tried using --model medium and that worked with similar results.

--model turbo is better than medium, anyway, large-v2 is best and find that max --best_of value for your GPU.