Skip to content

Add Nemotron ASR transcription support#741

Open
ncylich wants to merge 11 commits into
mainfrom
nemotron-asr-integration
Open

Add Nemotron ASR transcription support#741
ncylich wants to merge 11 commits into
mainfrom
nemotron-asr-integration

Conversation

@ncylich

@ncylich ncylich commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add Nemotron ASR model-family detection, conversion, transpile, and runtime wiring
  • add RNNT decoding support with file and stream transcription parity
  • harden graph save/load parameter persistence for ASR-relevant ops
  • restore corrected causal depthwise SIMD while preserving left-padded causal convolution semantics
  • restore Core ML audio encoder fast paths and warm audio encoder variants used by file and stream transcription
  • add focused regression tests for detection, graph serialization, and reused decoder state outputs

Benchmarks

Measured on cactus-engine/tests/assets/test.wav using mean engine time over 5 fresh CLI runs.

Model Path Mean engine time Realtime factor Speed vs CPU
Parakeet CPU 738 ms 0.0368x baseline
Parakeet Apple/NPU 214 ms 0.0107x 3.45x faster
Nemotron CPU 2178 ms 0.1087x baseline
Nemotron Apple/Core ML 586 ms 0.0293x 3.72x faster

Testing

  • cmake --build cactus-kernels/build --target test_conv && ./cactus-kernels/build/test_conv
  • source ./venv/bin/activate && cactus build && PYTHONPATH=python pytest python/cactus/convert/tests/test_policy.py::test_nemotron_asr_config_detection_extraction_and_plan python/cactus/convert/tests/test_policy.py::test_prompt_tdt_config_detection_stays_parakeet_tdt python/cactus/convert/tests/test_policy.py::test_nemotron_prompt_dim_preserves_sparse_prompt_ids python/tests/test_graph.py::TestGraphSaveLoad::test_save_load_roundtrip_glu_axis python/tests/test_transpile_aten_ops.py::test_reused_lstm_state_output_is_materialized -q
  • source ./venv/bin/activate && cactus build && cactus transcribe nvidia/nemotron-3.5-asr-streaming-0.6b --file cactus-engine/tests/assets/test.wav
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stt --transcription-model nvidia/nemotron-3.5-asr-streaming-0.6b
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stream --transcription-model nvidia/nemotron-3.5-asr-streaming-0.6b
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stt --transcription-model /Users/noahcylich/Documents/Desert/cactus/weights/nemotron-3.5-asr-streaming-0.6b-cq4-apple
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stream --transcription-model /Users/noahcylich/Documents/Desert/cactus/weights/nemotron-3.5-asr-streaming-0.6b-cq4-apple
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stt --transcription-model nvidia/parakeet-tdt-0.6b-v3
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stream --transcription-model nvidia/parakeet-tdt-0.6b-v3
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stt --transcription-model /Users/noahcylich/Documents/Desert/cactus/weights/parakeet-tdt-0.6b-v3-cq4-apple
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stream --transcription-model /Users/noahcylich/Documents/Desert/cactus/weights/parakeet-tdt-0.6b-v3-cq4-apple
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stt --transcription-model openai/whisper-base
  • source ./venv/bin/activate && cactus build && cactus test --component engine --suite stream --transcription-model openai/whisper-base
  • git diff --check

ncylich added 11 commits June 18, 2026 20:41
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Signed-off-by: Noah Cylich <noahcylich@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant