linto-ai/whisper-timestamped won't work extracting word-by-word

[matthias33890]

Hi, i have following Problem:
I wan to extract subtitles from a video word by word.
I use following code in python:

subs_ai = SubsAI()
model = subs_ai.create_model(model_name='linto-ai/whisper-timestamped', model_config={'model_type': 'base'})
print(subs_ai.model_info('linto-ai/whisper-timestamped'))
subs = subs_ai.transcribe(media_file=file, model=model)
subs.save('./ram/output.ass')

To troubleshoot I used this command:
subsai ./ram/withaudio.mp4 --model 'linto-ai/whisper-timestamped' --model-configs '{"model_type": "small", "verbose":true}'
In the terminal I get:

[00:00.000 --> 00:00.160] If
[00:00.160 --> 00:00.320] this
[00:00.320 --> 00:00.540] is a
[00:00.540 --> 00:00.960] Problem

That is how I want it:

But the saved .srt file has following contents:

1
00:00:00,000 --> 00:00:04,320
If this is a problem that you are facing then you need to take some classes

Which is not the word-for-word format I want

Anyone any suggestions on what I can do, so it works?

[matthias33890]

I fixed the issue by adding 'segment_type':'word' to my code:

subs_ai = SubsAI()
model = subs_ai.create_model(model_name='linto-ai/whisper-timestamped', model_config={'model_type': 'base','segment_type':'word'})
print(subs_ai.model_info('linto-ai/whisper-timestamped'))
subs = subs_ai.transcribe(media_file=file, model=model)
subs.save('./ram/output.ass')

[absadiki]

Hi @matthias33890,

Yes, you can switch between word or sentence using the segment_type config.

FYI, you can also extract the word-level timestamps using faster-whisper.
It gives good results as well:

subs_ai = SubsAI()
model = subs_ai.create_model(model_name='guillaumekln/faster-whisper', model_config={'model_type': 'base', "word_timestamps": True})
subs = subs_ai.transcribe(media_file=file, model=model)
subs.save('./ram/output.ass')

Hope this helps.