Using Aligner with whisper model

[pierre0x]

I am trying to align a text and I don't get how to use the inference function, when I use this code, I get an error

`import stable_whisper
from stable_whisper import non_whisper
model = stable_whisper.load_model('base')

result = non_whisper.Aligner(model.transcribe,"outvideo.mp3",assembled_text)

myencoder = stable_whisper.load_model('base').encoder()
mydecoder = stable_whisper.load_model('base').decoder()
result = non_whisper.Aligner(model.transcribe,mydecoder,myencoder).align(audio="audio.mp3",
text=assembled_text)`

How to properly use the Aligner function with the model ?

[jianfch]

The Aligner class is intended for implementing alignment for unsupported models. But all models loaded with stable_whisper.load_model() already has it implemented so you don't need use Aligner.
You can use model.align() instead.
See: https://github.com/jianfch/stable-ts?tab=readme-ov-file#alignment

[varn0]

hello @jianfch, looks like model.align() is a bit broken. Passing the audio, text and language does not work. Passing the audio and a WisperResult object does work.

import stable_whisper

model = stable_whisper.load_model('base')
aligned = model.align('walking.wav', 'I am walking down the road, looking at the sky.', language='en')
aligned.save_as_json('aligned.json')

root@25d02ced0f29:/app# python align.py 
...some FutureWarning
Traceback (most recent call last):
  File "/app/align.py", line 4, in <module>
    aligned = model.align('walking.wav', 'I am walking down the road, looking at the sky.', language='en')
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/stable_whisper/alignment.py", line 185, in align
    raise TypeError('expected argument for language')
TypeError: expected argument for language

Example of working code:

import stable_whisper

model = stable_whisper.load_model('base')
result = model.transcribe('walking.wav')
aligned = model.align('walking.wav', result)
aligned.save_as_json('aligned.json')

Output:

root@25d02ced0f29:/app# python align.py 
...some FutureWarning
Detected language: english                                                                                                                    
Transcribe: 100%|█| 7.21/7.21 [00:02<00:00,  3.32sec/s]
Align: 100%|█| 7.21/7.21 [00:00<00:00,  9.48sec/s]
Adjustment: 100%|█| 6.4/6.4 [00:00<00:00, 21037.26sec/s]
Saved: /app/aligned.json

[jianfch]

@varn0 thanks for pointing this out!
Fixed in 8e12ee4.