Add optional FunASR/SenseVoice voice transcription

[LauraGPT]

Summary

• adds an optional local FunASR/SenseVoice STT provider alongside the existing faster-whisper path
• exposes SenseVoice Small in the local Talk and Dictation model catalog when FunASR is installed
• wires the selected local STT model into the active voice runtime so Dictation/Talk actually use the chosen backend
• makes the voice extra self-contained by installing compatible CPU torch and torchaudio builds with FunASR
• preserves the Row-Bot 4.7.0 platform, app, and extensibility documentation while documenting the FunASR/SenseVoice option

Closes #152.

Validation

Exact signed head: d3de864b743d650c1afe5d9c5d77181ac35d81b2, merged with main@ab3997d74852ea18f1e4ecad8295052034333ab2.

• voice/dependency suite: 38 passed
• complete deterministic suite: 3,300 passed, 2 skipped, 1 deselected
• contracts: 60 passed
• subsystem: 1,196 passed
• migrated-subsystem coverage: 67.85% (55% required); the 9 coverage-only failures are unchanged workflow tests whose background SQLite work outlives pytest's shared basetemp under coverage, while the same four files pass 21/21 outside coverage
• installer contracts: 63 passed
• legacy inventory/source map: 20 passed
• lockfile, generated requirements, 79 runtime imports, compileall, Ruff safety/format, diff checks, and app HTTP smoke: passed
• Node 22 landing-page runtime matrix: passed
• fresh voice-only environment: torch 2.12.1+cpu and torchaudio 2.11.0+cpu import together
• real iic/SenseVoiceSmall CPU inference: every checkpoint key matched and the official 16 kHz sample transcribed as 欢迎大家来体验达摩院推出的语音识别模型。

[LauraGPT]

Resynced this branch with Row-Bot 4.7.0 main@ab3997d and restored mergeability at GitHub-verified signed+DCO head d3de864b. The prior exact head remains on rollback branch LauraGPT:codex/backup-row-bot-289-2b07667e-20260813.

The new upstream overlap was confined to README. The resolution retains the complete 4.7.0 platform/app/extensibility rows and the FunASR/SenseVoice voice entry. Real-provider validation also exposed that the existing voice extra installed FunASR without its required torchaudio import path; the final head now locks compatible CPU torch and torchaudio packages for both voice and all.

Fresh exact-tree evidence:

• voice/dependency: 38/38
• deterministic: 3,300 passed / 2 skipped / 1 deselected
• contracts: 60/60; subsystem: 1,196/1,196
• coverage: 67.85% (55% required)
• installer: 63/63; inventory/source map: 20/20
• lock, generated requirements, 79 runtime imports, compileall, Ruff safety/format, Node 22 runtime test, diff, and app HTTP smoke: passed
• clean voice-only environment imported torch 2.12.1+cpu with torchaudio 2.11.0+cpu
• real SenseVoiceSmall CPU run loaded all keys and transcribed 欢迎大家来体验达摩院推出的语音识别模型。

The coverage command still exposes an upstream test-fixture race: 9 unchanged workflow tests can let background SQLite work outlive pytest's shared basetemp when coverage slows execution. Those exact four test files pass 21/21 independently, and the complete non-coverage deterministic suite is green. No production workflow code was changed for that unrelated fixture behavior.

Fork workflows may require maintainer approval before Actions jobs appear.

[siddsachar]

Implemented and pushed the saved SenseVoice hardening plan to this existing contributor branch (no force-push, no new PR, and not merged).

Commits:

• 91cfc6bb — fix(voice): harden optional SenseVoice runtime
• 27f02277 — normal merge of current upstream main (a063ccaa)
• PR head is now 27f02277254e7f808be5b35778940906d47fe7b3; the contributor baseline d3de864b remains intact in history.

What changed:

• SenseVoice acquisition is now an explicit Voice settings action with source/size/license/data disclosure.
• Package probing and readiness use offline metadata checks. Startup and transcription refuse to construct FunASR unless a persisted, verified app-owned snapshot contains the required local files.
• Normal FunASR construction passes only the verified local path to AutoModel, forces CPU, and sets disable_update=True.
• Missing packages, missing model installation, corrupt/missing local data, ready state, and unsupported Intel macOS are distinct, recoverable statuses.
• Talk and Dictation retain both local Whisper and SenseVoice selection. Whisper behavior is unchanged.
• Torch and Torchaudio are exactly matched at 2.11.0 using the existing CPU index; FunASR/ModelScope are explicit voice dependencies. Intel macOS excludes only the unsupported SenseVoice stack, leaving Faster-Whisper available.
• Runtime dependency verification now covers FunASR, ModelScope, Torch, and Torchaudio.
• Added deterministic offline/install/cache/corruption/selection/Whisper/dependency/lock/source-map/UI coverage and removed stale formatting/generated-site churn.

Post-merge validation:

uv run python -m pytest tests/test_voice_sensevoice.py tests/test_voice_providers.py tests/test_voice_provider_catalog.py tests/test_voice_coordinator.py tests/test_voice_runtime.py tests/test_dependency_metadata.py tests/test_settings_overhaul_contracts.py tests/docs/test_public_docs_automation.py::test_generated_mdx_pages_are_current tests/docs/test_public_docs_automation.py::test_public_docs_metadata_validates tests/subsystem/test_source_test_map.py -q
88 passed in 5.87s

uv lock --check
Resolved 304 packages

python scripts/export_locked_requirements.py --check
requirements.txt is current

uv sync --locked --all-extras --group test
Resolved 304 packages; checked 290 packages

uv run python scripts/verify_runtime_dependencies.py all
Passed all groups; 84 imports checked

uv run python scripts/run_test_matrix.py changed --base origin/main
1239 passed, 5 warnings
legacy/source-map: 21 passed

uv run python scripts/run_test_matrix.py pr
contracts: 60 passed
subsystem: 1197 passed, 5 warnings
migrated coverage: 1257 passed, 748 warnings; 67.93% (55% required)
deterministic: 3313 passed, 5 skipped, 1 deselected, 5 warnings
installer contracts: 63 passed
app smoke: process start, launcher ping, HTTP 200, and clean termination all passed
legacy/source-map: 21 passed
lock/export/sync/runtime imports/compileall/ruff-safety all passed

Python 3.12 target resolution using uv pip compile pyproject.toml --extra voice --python-version 3.12 --python-platform <target>:

• Windows x64: torch==2.11.0+cpu, torchaudio==2.11.0+cpu
• macOS arm64: torch==2.11.0, torchaudio==2.11.0
• Linux x64: torch==2.11.0+cpu, torchaudio==2.11.0+cpu
• macOS x64: SenseVoice dependencies excluded; faster-whisper==1.2.1 remains resolvable

Warnings/known local-environment notes:

• Runtime verification notes that FFmpeg is not installed and uses Torchaudio for audio loading.
• FAISS AVX2 import falls back successfully to standard FAISS.
• The matrix reports existing Starlette/httpx, LangGraph, langchain-community, NumPy reload, and SQLite resource warnings.
• compileall prints the existing Can't list 'debug_tools.py' notice but exits successfully.
• Initial sandbox runs had Windows ACL failures on a reused pytest temp directory and uv shared cache; reruns with an isolated pytest temp path and authorized cache access passed.
• One intermediate deterministic run found two generated-doc freshness failures after the new settings controls shifted source-line inventory. The UI was tightened into a local renderer to preserve the upstream generated inventory; the two docs checks and the final full matrix pass.

Manual/live validation:

• A fresh isolated explicit install reached ModelScope and began writing the app-owned snapshot (21 files / 14.6 MB), but the external model.pt transfer stalled at 11 MB for over ten minutes, then its HTTPS connection closed while the SDK remained waiting. The stalled process was stopped and the incomplete isolated cache was not accepted as valid.
• Therefore a completed first live download, a second real cached launch with network disabled, and a short real CPU transcription remain pending. Their deterministic equivalents (including no downloader call from readiness/startup/transcription, persisted verified path, local-path-only AutoModel, network-blocked cache behavior, and corrupt cache recovery) pass.
• Windows installer contracts and local Windows runtime imports/app smoke passed. A built Windows installer install/upgrade smoke, macOS installer/runtime execution, signing, notarization, and clean-machine UX remain platform-specific manual work.

No approval or merge was performed.

[siddsachar]

Live app + real-cache follow-up validation (Windows x64)

Pushed follow-up commit 116f2c0c (fix(voice): guard SenseVoice install action) to the existing contributor branch. PR head is now 116f2c0c5164b211d3fb20b469b06fd808c7f8fd. No rebase, force-push, approval, or merge.

Real user workflow

Tested the ordinary local owner app in the in-app browser against the existing user data directory.

• Before installation, Local Whisper small remained selected/ready; SenseVoice was marked setup; the ~940 MB ModelScope/privacy/license disclosure and explicit Install SenseVoice Small action were visible.
• SenseVoice was present in both Dictation and Local Talk model selectors without triggering acquisition.
• The explicit UI action downloaded and verified a 936,291,369-byte model.pt. The saved SenseVoice path remained absent while the snapshot was incomplete and was persisted only after verification.
• A real live retry exposed a duplicate-click race: overlapping install callbacks could target a Settings element deleted by the first successful reopen. Commit 116f2c0c adds an in-flight guard, disables the action immediately, restores it on failure, and completes notifications through the captured client context.
• After a full app stop/start, SenseVoice was immediately ready, no install button was shown, and the pre-test OpenAI Realtime Talk / Local Whisper small selections were restored.
• Started/stopped installed SenseVoice in both Dictation and Local Talk from a real conversation UI. No prompt was sent, no workflow/channel action was triggered, and the composer remained empty for the silent capture.
• Local Kokoro test playback, Providers, Models, Home, and read-only conversation navigation remained functional. SenseVoice did not appear in normal chat model settings.
• All test app and tunnel helper processes started by this validation were stopped afterward.

Real offline CPU inference

Using the real persisted VoiceService path, all socket connection APIs were replaced with a function that raises on any connection attempt. The cached model loaded with all checkpoint keys matched and transcribed the bundled English sample as:

The tribal chieftain called for the boy and presented him with 50 pieces of gold.

The probe completed successfully in about 11 seconds end to end with no network attempt.

Automated validation

Focused commands:

.venv\Scripts\python.exe -m pytest tests\test_settings_overhaul_contracts.py -q
.venv\Scripts\python.exe -m pytest tests\test_voice_sensevoice.py tests\test_voice_runtime.py tests\test_voice_provider_catalog.py tests\test_voice_providers.py tests\test_settings_overhaul_contracts.py tests\test_browser_local_voice.py tests\subsystem\test_source_test_map.py -q

Results: 15 passed; then 61 passed.

Changed-source gate:

uv run python scripts/run_test_matrix.py changed --base origin/main
$env:PYTEST_ADDOPTS='--basetemp=.tmp/pytest_tmp_pr289_live_changed'
uv run python scripts/run_test_matrix.py changed --base origin/main
.venv\Scripts\python.exe -m pytest tests\test_agent_runner.py::test_child_skill_snapshot_starts_profile_skills_and_loads_others_task_locally -q --basetemp=.tmp\pytest_tmp_pr289_agent_isolated -p no:cacheprovider

• First attempt: pytest could not clean the repository shared temp root (WinError 5); no product assertion was reached for affected cases.
• Isolated-temp rerun: 1,238 passed, 1 unrelated order-dependent agent-skill isolation failure, 7 warnings.
• The lone failure passed in isolation: 1 passed, 1 warning.
• All selected SenseVoice/Voice/UI tests passed.

Full PR gate:

$env:PYTEST_ADDOPTS='--basetemp=.tmp/pytest_tmp_pr289_live_pr -p no:cacheprovider'
uv run python scripts/run_test_matrix.py pr

Passed in 914.4 seconds:

• lock check and generated requirements check
• locked all-extras/test sync
• runtime dependency verification: 84 imports
• compileall and Ruff safety
• contracts: 60 passed
• subsystem: 1,197 passed (5 warnings)
• migrated coverage gate: passed (55% threshold)
• deterministic: 3,313 passed, 5 skipped, 1 deselected (5 warnings)
• installer contracts: 63 passed
• app smoke: process start, launcher ping, HTTP 200, clean termination
• legacy inventory/source map: 21 passed

Warnings/limitations:

• Runtime verification reported that ffmpeg is absent and used the supported Torchaudio fallback.
• The matrix emitted existing deprecation/resource warnings and compileall reported the already-absent optional debug_tools.py; the gate exited 0.
• The real app logged unrelated Windows keyring cleanup/resource warnings and duplicate-ngrok/SMS tunnel warnings while multiple test launches overlapped; cached Voice startup and the owner UI remained healthy.
• The headless browser could exercise microphone start/stop but had no human speech input. Real transcription quality was therefore validated with the bundled speech sample through the actual persisted VoiceService.
• macOS installer/runtime and a clean-machine packaged-installer UX check remain platform-specific and were not run here.

[siddsachar]

OSV follow-up pushed as 4606bde1 (fix(deps): resolve OSV scanner findings).

Root cause: the OSV baseline exceptions expired on 2026-07-31. The failed scan then exposed both newly fixable dependency findings and a few findings whose compatible upstream fixes are not available yet.

What changed:

• Upgraded the fixable Python findings, including aiohttp 3.14.3, cryptography 50.0.0, h2 4.4.1, httplib2 0.32.0, langgraph-checkpoint-sqlite 3.1.1, MCP 1.29.0, Pillow 12.3.0, pyasn1 0.6.4, pypdf 6.16.0, Pytest 9.1.1, Requests 2.33.1, and Transformers 5.15.0.
• Moved arxiv to 3.0.0 so fixed Requests can resolve, and moved Hugging Face Hub to 1.27.0 for Transformers 5.x compatibility.
• Regenerated uv.lock and requirements.txt with repository tooling.
• Updated the vulnerable docs lock entries and added narrow npm overrides for fixed serialize-javascript 7.0.5 and uuid 11.1.1.
• Refreshed the WhatsApp bridge lock so Baileys/libsignal leaves protobufjs 6.8.8; npm ci now reports 0 vulnerabilities there.
• Replaced the expired broad OSV baseline with four exact exceptions expiring 2026-09-30: setuptools 81.0.0, Torch 2.11.0, Torch 2.11.0+cpu, and docs-only image-size 2.0.2. Torch 2.13 is fixed, but the official Torchaudio CPU index currently stops at the matching 2.11 release across supported platforms; Torch 2.11 also requires setuptools <82. OSV lists no fixed image-size release.

Validation:

• Official OSV Scanner 2.3.8 Windows binary, published SHA-256 verified (cb04e79dd9698a7bc821bbfdddec916a416d1409fda79c927c509d37d00c9716): exact workflow scan command over all three lockfiles -> No issues found, exit 0.
• uv lock --check -> pass.
• python scripts/export_locked_requirements.py --check -> pass (requirements.txt is current).
• uv sync --locked --all-extras --group test -> pass (309 resolved, 296 checked).
• uv run python scripts/verify_runtime_dependencies.py all -> pass, 84 imports checked across all runtime groups. Notice only: ffmpeg absent, Torchaudio fallback available.
• Focused voice/dependency/embedding/WhatsApp/browser tests -> 72 passed.
• uv run python scripts/run_test_matrix.py changed --base origin/main -> 1,186 selected tests passed plus 21 inventory tests; 0 failures. Warnings: existing Starlette/LangGraph deprecations and Windows notification-thread warnings.
• uv run python scripts/run_test_matrix.py pr -> pass: contracts 60; subsystem 1,197; migrated coverage gate passed (>=55%); deterministic 3,313 passed, 5 skipped, 1 deselected; installer contracts 63; app smoke 4/4; inventory 21. Existing deprecation/resource warnings remained; compileall also printed the pre-existing optional debug_tools.py absence while exiting 0.
• Docs: clean npm ci, then npm run build:ci -> pass; 86 pages and 3,196 words indexed. npm audit still expands the two unfixed image-size advisories through 17 Docusaurus build dependencies.
• WhatsApp: clean npm ci -> 0 vulnerabilities; node --check bridge.js -> pass.
• Real default data directory/cache: SenseVoice status ready; real cached CPU model loaded with socket connections blocked and transcribed a generated sample as Robot offline speech recognition is working perfectly. No network connection was attempted.
• Real app on a separate local port: Voice settings showed SenseVoice ready; SenseVoice was selectable and persisted for both Talk and Dictation; original Realtime Talk / Whisper Dictation preferences were restored; browser console had 0 warnings/errors. The temporary app and its child processes were stopped; the pre-existing app instance was left running.

GitHub status: PR head is 4606bde15ce75cec951d1461449f621ee9559566. The newly triggered fork workflows currently show action_required with zero jobs and await maintainer approval, so the remote OSV conclusion is not yet available. No PR merge was performed.