How often reset_states() should be called in never-ending streaming scenarios with 512 sample chunks?

[progmars]

I am using Silero VAD in a similar way to pyaudio-streaming-examples.ipynb feeding it with 512 sample chunks from Python sounddevice library input.

I noticed that in about 30 seconds it started giving very low confidences for clear voices with some spurious recoveries when it suddenly detected voice chunks again.

The problem was almost solved when I added vad_model.reset_states() call for every chunk. But to be honest I don't fully understand what I'm doing.

Is there a suggested logic behind resetting and any recommendations how often it is reasonable to call it, especially in streaming cases when there is no clear "audio end" for the never-ending input stream?

I have recorded a wav file that triggers this issue. Here are matplotlib graphs. The top is the graph of the audio file itself with quite clear voice peaks. The second graph is VAD confidences with calling reset_states after every 512 sample chunk. The third one is VAD confidences without reset. As you may notice, without reset it starts with giving very high confidences and then suddenly starts failing. With reset, the confidences are lower but the results match the waveform more closely. However, with reset it starts missing some voice fragments, too, albeit behaves more consistently. So, clearly, resetting every 32ms is overkill and hurts detection. But then how often to reset?

Here is my testing code (pardon my clunky Python, I come from another language):

import numpy as np
import wave
import torch
import matplotlib
import matplotlib.pylab as plt
from silero_vad import load_silero_vad, read_audio, get_speech_timestamps

INPUT_FILE = "dumps/long_input.wav"
SAMPLE_RATE = 16000
BLOCK_SIZE_SAMPLES = 512 # to match Silero VAD defaults, 32ms buffer

vad_model = load_silero_vad()

# read by 32ms chunks and feed manually to simulate audio input
wf = wave.open(INPUT_FILE, "rb")
nframes = wf.getnframes()
raw = wf.readframes(nframes)
wf.close()

# Convert raw bytes to numpy float32 in range -1..1
# The test file is known to be int16 mono 16kHZ
wav = np.frombuffer(raw, np.int16).astype(np.float32) / 32768.0

total = wav.shape[0]
pos = 0

voiced_confidences_no_reset = []

while True:
    end = pos + BLOCK_SIZE_SAMPLES
    if end <= total:
        chunk = wav[pos:end]
    else:
        break
    pos = end    

    # as in pyaudio-streaming-examples.ipynb
    new_confidence = vad_model(torch.from_numpy(chunk), SAMPLE_RATE).item()
    voiced_confidences_no_reset.append(new_confidence)

#========================
# reset once before the test with resets in the loop
vad_model.reset_states()
pos = 0

voiced_confidences_with_reset = []

while True:
    end = pos + BLOCK_SIZE_SAMPLES
    if end <= total:
        chunk = wav[pos:end]
    else:
        break
    pos = end    

    # as in pyaudio-streaming-examples.ipynb
    new_confidence = vad_model(torch.from_numpy(chunk), SAMPLE_RATE).item()
    voiced_confidences_with_reset.append(new_confidence)

    # important - this makes the results much better in this case
    vad_model.reset_states()


#------------------------
# plot the confidences for the speech
plt.figure(figsize=(10,4))
plt.plot(voiced_confidences_with_reset)
plt.figure(figsize=(10,4))
plt.plot(voiced_confidences_no_reset)
plt.figure(figsize=(10,4))
plt.plot(wav)
plt.show()

[snakers4]

Is there a suggested logic behind resetting and any recommendations how often it is reasonable to call it, especially in streaming cases when there is no clear "audio end" for the never-ending input stream?
But then how often to reset?

reset_states() should be called when the new audio stream starts and / or the old one ends (e.g. at the begging of each new session).

pyaudio-streaming-examples.ipynb

The more likely culprit is that we changed the VAD interface in v5 and the example may have not been updated accordingly.
For PyTorch we tucked the changes inside of the model, so it is strange that is is misbehaving,

I have recorded a wav file that triggers this issue. Here are matplotlib graphs.

There is a much simpler way to debug how the VAD works. Just use the standard interface:

silero-vad/src/silero_vad/utils_vad.py

Lines 212 to 227 in 3ca476e

	def get_speech_timestamps(audio: torch.Tensor,
	model,
	threshold: float = 0.5,
	sampling_rate: int = 16000,
	min_speech_duration_ms: int = 250,
	max_speech_duration_s: float = float('inf'),
	min_silence_duration_ms: int = 100,
	speech_pad_ms: int = 30,
	return_seconds: bool = False,
	time_resolution: int = 1,
	visualize_probs: bool = False,
	progress_tracking_callback: Callable[[float], None] = None,
	neg_threshold: float = None,
	window_size_samples: int = 512,
	min_silence_at_max_speech: int = 98,
	use_max_poss_sil_at_max_speech: bool = True):

Then just use the visualize_probs = True. If this works fine, then you can work backwards from the example and find an issue.
If it does not, please send your audio file, we will take a look.

[progmars]

Thank you for reply.
In my case, I did not need timestamps at all, that's why I went with the direct model call examples.

I now tried get_speech_timestamps with visualize_probs for the same file. The result is the same issue - after about 30 seconds it stops recognizing voice.

It might be just a coincidence that reset somehow helps, but it's still strange - one would imagine that with a known previous state the model should always be able to detect the next chunk better (as it usually is for most cases I tested), but in this case it needed the resets.

I tried fiddling around with the frequency of resets in my code. So, resetting once around 30 seconds was not enough - it detected better after the reset, but then again started failing. However, when I added logic to reset every 10 seconds, it seemed to work the best. Here is the probs result from my direct model calls with reset every 10 seconds:

So, I think, just to be safe, I will reset it every 10 seconds or whenever the user's input gets committed further to ASR - whichever comes first.

Here is the tested audio file.
long_input2.wav
It is a recording of short words in Latvian. I'm working on a voice command tool for people who cannot use their hands (similar to Windows 11 Voice Access but in Latvian and, hopefully, with more functionality). The tester got frustrated after 40 seconds when the system could not recognize "lapa lejup", so he repeated it many times, until finally it was detected.

The testing system is Windows 11. Here is full info, just in case:

CUDA available: True
Torch environment:
Collecting environment information...
PyTorch version: 2.9.1+cu128
Is debug build: False
CUDA used to build PyTorch: 12.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro (10.0.26200 64-bit)
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.19 | packaged by conda-forge | (main, Oct 22 2025, 22:23:22) [MSC v.1944 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.26200-SP0
Is CUDA available: True
CUDA runtime version: 12.9.41
CUDA_MODULE_LOADING set to:
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 581.57
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Name: Intel(R) Core(TM) i7-14700

Versions of relevant libraries:
[pip3] numpy==2.0.2
[pip3] onnxruntime==1.23.1
[pip3] pytorch-lightning==2.5.5
[pip3] pytorch-metric-learning==2.9.0
[pip3] torch==2.9.1+cu128
[pip3] torch-audiomentations==0.12.0
[pip3] torch_pitch_shift==1.2.5
[pip3] torchaudio==2.9.1+cu128
[pip3] torchcodec==0.8.1 (I had to stub this with https://github.com/FaridPF/torchcodec_fallback/blob/main/_torchcodec.py  for read_audio to work because torchcodec has issues on Windows)
[pip3] torchmetrics==1.8.2
[pip3] torchvision==0.24.1+cu128

[snakers4]

I confirm, that the VAD breaks on this file, we need to take a closer look

[snakers4]

We cannot understand why this happens
On other long test files the VAD works fine
The only hack that kind of works is resampling to 8 kHz

[progmars]

Thank you for checking this. Yeah, that's strange. I saw it happen a few times during tests. Might be something specific to this person's voice or microphone or the fact that there are lots of silences between short words. However, we tried it with a record of counting up for a minute and there were no issues with that file.
Anyway, the "reset at least every 10 seconds" trick works, and your 8kHz suggestion also will be good.