Stopped in the middle of alignment

[terryops]

I'm aligning a text with a wav file with faster-whisper.
It went well in the first around 30% but it failed at last. Try different files, mp3, wav files, don't work at all. The first 30-40% went well though, they align very well.

Showing logs as follow:

Nov 01 23:47:11 conda[1296381]: result = aligner.align(audio, text)
Nov 01 23:47:11 conda[1296674]: [aost#0:0/pcm_s16le @ 0x562edbf046c0] Error submitting a packet to the muxer: Broken pipe
Nov 01 23:47:11 conda[1296674]: Last message repeated 1 times
Nov 01 23:47:11 conda[1296674]: [out#0/s16le @ 0x562edbf00980] Error muxing a packet
Nov 01 23:47:11 conda[1296674]: [out#0/s16le @ 0x562edbf00980] Task finished with error code: -32 (Broken pipe)
Nov 01 23:47:11 conda[1296674]: [out#0/s16le @ 0x562edbf00980] Terminating thread with return code -32 (Broken pipe)
Nov 01 23:47:11 conda[1296674]: [out#0/s16le @ 0x562edbf00980] Error writing trailer: Broken pipe
Nov 01 23:47:11 conda[1296674]: [out#0/s16le @ 0x562edbf00980] Error closing file: Broken pipe

Could you please tell me what could the issue here?

[jianfch]

This might be a ffmpeg issue. Try to use stream=False to load the entire audio file at once instead of in chunks which causes the "broken pipe" you see when either ffmpeg or stable-ts fails.

[terryops]

Thank you very much! It works now. @jianfch
By the way, I'm trying to align Thai text using stable-ts, but Whisper performs poorly in transcribing Thai. Will this affect the accuracy of the text alignment task?

[terryops]

@jianfch Just bought you some coffee. Thank you.

[jianfch]

It is generally able to align relatively well for text that transcription would otherwise struggle to produce. If you're looking for better transcription performance on Thai, you can try to use Whisper models that were finetuned on Thai.

For example loading https://huggingface.co/biodatlab/whisper-th-medium-combined with stable-ts:

model = stable_whisper.load_hf_whisper('biodatlab/whisper-th-medium-combined')

Model finetuned on Thai might also be better at aligning Thai. I'd suggesting using it with the new alignment algorithm aligner='new' with model.align() (which should work with most Hugging Face models loaded with stable-ts).
Note that for Hugging Face models, alginer='new' will not work with model.transcribe() but it will with model.as_vanilla_model().transcribe().

[terryops]

It is generally able to align relatively well for text that transcription would otherwise struggle to produce. If you're looking for better transcription performance on Thai, you can try to use Whisper models that were finetuned on Thai.

For example loading https://huggingface.co/biodatlab/whisper-th-medium-combined with stable-ts:

model = stable_whisper.load_hf_whisper('biodatlab/whisper-th-medium-combined')
Model finetuned on Thai might also be better at aligning Thai. I'd suggesting using it with the new alignment algorithm aligner='new' with model.align() (which should work with most Hugging Face models loaded with stable-ts). Note that for Hugging Face models, alginer='new' will not work with model.transcribe() but it will with model.as_vanilla_model().transcribe().

The medium model has a higher WER than a model fine-tuned with larger model. Since people always not having authoritative text to align with the audio, which results in failing in most of the time, with stable-ts. I'm considering transcribing with a model with lower WER and align with WhisperX's wav2vec models.
What do you think?

[jianfch]

The medium model has a higher WER than a model fine-tuned with larger model. Since people always not having authoritative text to align with the audio, which results in failing in most of the time, with stable-ts.

Better WER isn't always indicative of better performance but they do have others models sizes for Thai https://github.com/biodatlab/thonburian-whisper.

I'm considering transcribing with a model with lower WER and align with WhisperX's wav2vec models.
What do you think?

The alignment is already part of the transcription process by default, so re-aligning it with WhisperX is unnecessary unless you find it performs better with your data.

[terryops]

The point of using whisperX is that the fine-tuned version of whisper with low WER doesn’t have timestamp while the higher WER one does have timestamp.

…

2025年11月4日 23:43，jian ***@***.***> 写道： jianfch left a comment (jianfch/stable-ts#463) <#463 (comment)> The medium model has a higher WER than a model fine-tuned with larger model. Since people always not having authoritative text to align with the audio, which results in failing in most of the time, with stable-ts. Better WER isn't always indicative of better performance but they do have others models sizes for Thai https://github.com/biodatlab/thonburian-whisper. I'm considering transcribing with a model with lower WER and align with WhisperX's wav2vec models. What do you think? The alignment is already part of the transcription process by default, so re-aligning it with WhisperX is unnecessary unless you find it performs better with your data. — Reply to this email directly, view it on GitHub <#463 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3NLOKKS7EYUZZEFFRLRBL33DCS5AVCNFSM6AAAAACK3RB7RWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIOBWGY2TMNBSGE>. You are receiving this because you authored the thread.