Why Does faster-whisper-large-v2 Use More Than Twice the VRAM on RTX 5060 Compared to RTX 2060?

[ethansy1995]

I am experiencing a significant difference in VRAM usage when running the faster-whisper-large-v2 model on different GPUs.

On an RTX 2060 (6GB VRAM), the quantized version of the model (int8_float16) runs smoothly and uses about 4GB of VRAM.

However, on an RTX 5060 (8GB VRAM), running the same model with the same quantization settings leads to VRAM usage exceeding 8GB, causing out-of-memory errors and preventing the model from running properly.

This is unexpected because the RTX 5060 is a newer generation GPU with more VRAM and better performance, so it should not consume more memory than the older 2060, especially not more than double.

[Purfview]

Sorry, but I can't make sense of your post.

Please post screenshots of what you are doing there with --verbose true

[ethansy1995]

I encountered this issue while using software that integrates Faster-Whisper-XXL. Here is a screenshot of the error log from the software:

Previously, I was using an RTX 2060 to run faster-whisper-large-v2 with FP16, and it only used less than 4GB of VRAM.
Now, after switching to an RTX 5060, the VRAM is no longer sufficient. I also tested with an RTX 5070, and it requires around 8.5GB of VRAM to run.

[Purfview]

I still don't see what you are doing there. Where is the parameters used? Where is verbose output?

while using software that integrates Faster-Whisper-XXL

What software? Maybe you should ask at the place of that software, because your OP doesn't make sense, obviously not the same quantization is used if there is such big difference in the memory usage.