Faster-Whisper-XXL Pro r3.256.1 MB-RoFormer Aray Memory Error

[billz2]

This started happening with the new version, not a problem with the prior version. processing multiple files using one click.bat. When I resubmit the erroed file in the oneclick it's fine I have 12gb VRAM on my nvidia card it works fine. I am suspisious that the the vram is not released when the 1st file completed.

I am using this command

"%dp%faster-whisper-xxl.exe" %file_list% -pp -o source --batch_recursive --check_files --standard -f json srt -m large-v2 --vad_method pyannote_v3 --realign --task translate --ff_vocal_extract mb-roformer --roformer_vram 8 --language ja

This is happening on the next file after the 1st file successfully completed.

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:37<<00:00

Traceback (most recent call last):
File "main.py", line 212, in ffmpeg_audio
File "faster_whisper\roformer_infer.py", line 248, in RoFormer_separator
File "faster_whisper\roformer_infer.py", line 98, in demix_track
File "<array_function internals>", line 180, in nan_to_num
File "numpy\lib\type_check.py", line 515, in nan_to_num
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 429. MiB for an array with shape (1, 2, 225086400) and data type bool

Standalone Faster-Whisper-XXL Pro r3.256.1 running on: CUDA

Starting to process: O:\Avant Browser\data\Video1

Audio filtering is in progress...

Processing Part 1 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:38<<00:00

Processing Part 2 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:38<<00:00

Processing Part 3 out of 3:

MB-RoFormer model running on CUDA: 100% | 590/590 | 03:53<<00:00

Starting sequential faster-whisper inference.

Matplotlib is building the font cache; this may take a moment.
100% | 13732/13732 | 18:41<<00:00 | 12.25 audio seconds/s

Transcription speed: 12.25 audio seconds/s

Starting realignment.
Realignment finished in: 0:00:20.931

Subtitles are written to 'O:\Avant Browser\data' directory.

Starting to process: O:\Avant Browser\data\Video2

Audio filtering is in progress...

Processing Part 1 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:37<<00:00

Traceback (most recent call last):
File "main.py", line 212, in ffmpeg_audio
File "faster_whisper\roformer_infer.py", line 248, in RoFormer_separator
File "faster_whisper\roformer_infer.py", line 98, in demix_track
File "<array_function internals>", line 180, in nan_to_num
File "numpy\lib\type_check.py", line 515, in nan_to_num
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 429. MiB for an array with shape (1, 2, 225086400) and data type bool
Press any key to continue . . .

[Purfview]

This started happening with the new version, not a problem with the prior version. processing multiple files using one click.bat. When I resubmit the erroed file in the oneclick it's fine I have 12gb VRAM on my nvidia card it works fine. I am suspisious that the the vram is not released when the 1st file completed.

I don't remember now if it should be released (from whisper model), and I don't remember how this option works: --model_preload
Try it with both false/true.

[billz2]

Itried your suggestion with both false an true options and didn't make a difference, the command line is below

"%dp%faster-whisper-xxl.exe" %file_list% -pp -o source --batch_recursive --check_files --standard -f json srt -m large-v2 --vad_method pyannote_v3 --realign --task translate --ff_vocal_extract mb-roformer --roformer_vram 8 --language ja --model_preload False

[Purfview]

Itried your suggestion with both false an true options and didn't make a difference,

Didn't it fail on the first file when "--model_preload true"?

[Purfview]

I am suspisious that the the vram is not released when the 1st file completed.

That error is about RAM.

This started happening with the new version, not a problem with the prior version.

Because compute_type default is changed, it takes more vram/ram. Set it to --compute_type auto as in previous version.

[Purfview]

I don't remember now if it should be released (from whisper model) and I don't remember how this option works: --model_preload

By default, if "--ff_vocal_extract" is in use then "--model_preload" defaults to False.
So, whisper model is unloaded from vram at the end of transcription of a file, but it's unloaded to ram. so in a next iteration it can be loaded again faster.
Anyway, --compute_type auto should bring you the ram/vram usage of the previous version.

If you want everything fully unloaded between iterations then "One Click Transcribe.bat" needs to be tweaked to support such.

[billz2]

Unfortunately --compute_type auto did not make difference, the use case is that the 1st file is always succesful, it's on the second that the file that this error occurs the traceback is the same.

[Purfview]

Unfortunately --compute_type auto did not make difference

It should've some difference, as it frees ~1.6 GB RAM, when your error is about 429 MB allocation. Did you test exact same files and exact same settings as in r2.245.4?

I think with r2.245.4 you're already operating at the edge of available RAM, and the updated torch in r3.256.1 uses a bit more RAM by design. How much RAM do you have?

Here I tested 2x 2 hours files:

=========== r2.245.4 ===========
Peak RAM  : 10766 MiB
Peak VRAM : 3928 MiB
================================

=========== r3.256.1 ===========
Peak RAM  : 11194 MiB              + 428
Peak VRAM : 3914 MiB               - 14
================================

+428 MB is expected, if you run there ~4h long files then r3.256.1 probably adds +1 GB [ +1.6 GB different compute_type ].

[billz2]

I have been using the same 2 files and in the same sequence for all the runs. I have 64GB RAM and 12GB VRAM on my CUDA card. Below is the run using r2.245.4 version it worked fine.

Standalone Faster-Whisper-XXL Pro r2.245.4 running on: CUDA

Batch processing 2 files from the filelist: M:\folder for oneclick input\list.txt

Starting to process: O:\Avant Browser\data\File1

Audio filtering is in progress...

Processing Part 1 out of 3:

MB-RoFormer model running on CUDA: 0% | 0/851 | 00:00<<?faster_whisper\roformer_attend.py:73: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
MB-RoFormer model running on CUDA: 100% | 851/851 | 06:01<<00:00

Processing Part 2 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 06:03<<00:00

Processing Part 3 out of 3:

MB-RoFormer model running on CUDA: 100% | 590/590 | 04:08<<00:00

Starting sequential faster-whisper inference.

100% | 13732/13732 | 21:32<<00:00 | 10.63 audio seconds/s

Transcription speed: 10.63 audio seconds/s

Starting realignment.
Realignment finished in: 0:00:22.840

Subtitles are written to 'O:\Avant Browser\data' directory.

Starting to process: O:\Avant Browser\data\File2

Audio filtering is in progress...

Processing Part 1 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:57<<00:00

Processing Part 2 out of 3:

MB-RoFormer model running on CUDA: 100% | 851/851 | 05:57<<00:00

Processing Part 3 out of 3:

MB-RoFormer model running on CUDA: 100% | 193/193 | 01:20<<00:00

Starting sequential faster-whisper inference.

100% | 11353/11353 | 16:42<<00:00 | 11.32 audio seconds/s

Transcription speed: 11.33 audio seconds/s

Starting realignment.
Realignment finished in: 0:00:18.195

Subtitles are written to 'O:\Avant Browser\data' directory.

Operation finished in: 1:15:29.689

Press any key to continue . . .