Real-time Speech Recognition with PyAudio and Faster Whisper (Issue with Temporary Files)

[jagarlamudisrinath]

I'm building a real-time speech recognition system using PyAudio for recording and Faster Whisper for transcription. While I've achieved progress, I'm facing an issue with temporary audio files.

Here's the current approach:

• PyAudio captures audio in chunks.
• Audio data is accumulated until a specific duration is reached.
• A temporary WAV file is created for each chunk.
• Faster Whisper transcribes the temporary file.

The problem:

The use of temporary files is creating unnecessary disk I/O, potentially impacting performance.

• Are there alternative ways to handle audio data for real-time transcription that avoid temporary files?
• Is there a more efficient way to work with Faster Whisper that accepts raw audio data directly?
• Any suggestions for optimizing performance and minimizing latency in a real-time speech recognition system?

[kvrban]

here you can get inspiration:
https://github.com/gaborvecsei/whisper-live-transcription/blob/main/standalone-poc/live-transcribe.py

[piyushmaharana]

mount the audio files folder to tmpfs

[MahmoudAshraf97]

Faster whisper accepts both buffers and raw data directly assuming it is in float32 and sampled at 16khz