Voxtral Journal Windows

Local Windows 11 + WSL2 Ubuntu project for batch transcribing personal audio journals with mistralai/Voxtral-Mini-3B-2507.

What is it?

I’ve been using audio journaling for a while, and one of the best parts is being able to talk naturally instead of forcing myself to sit down and write.

I used to use Gemini for transcribing and summarizing my audio journals, and honestly, it worked pretty well. But the more I used it, the more uncomfortable I got with the privacy side of it.

These are personal journals. They can include thoughts, stress, relationships, work stuff, health stuff, and things I’m still processing. Even if a cloud provider has good policies, I just didn’t love the idea that my most private reflections had to leave my machine to become useful.

So I made Voxtral Journal Windows.

It works locally, without needing a cloud transcription API, and the core workflow keeps everything on your computer.

This project is intentionally local-first:

• vLLM runs in WSL2 Ubuntu.
• Gradio runs in WSL2 Ubuntu.
• The Windows browser opens http://localhost:7860.
• No Mistral/OpenAI/cloud API is required for the core flow.
• The project does not use mistralai/Voxtral-Mini-4B-Realtime-2602 and does not implement realtime streaming.

Architecture

1. Start vLLM in WSL2 on 0.0.0.0:8000.
2. Start the Gradio app in WSL2 on 0.0.0.0:7860.
3. Upload an audio file from the browser.
4. The app writes:
   • data/uploads
   • data/normalized
   • data/chunks
   • data/raw_transcripts
   • data/final_transcripts
   • data/history/index.json
   • data/logs

Voxtral Request Route

The main compatible path is based on the official Voxtral Mini 3B Hugging Face model card:

• Raw transcription uses the OpenAI-compatible /v1/audio/transcriptions client call generated by mistral_common.protocol.transcription.request.TranscriptionRequest.
• Prompted audio tasks use Mistral audio chat objects (AudioChunk, TextChunk, UserMessage) against /v1/chat/completions.
• Voxtral Mini 3B documentation notes that true system prompts are not yet supported, so this app stores your system prompt in app/prompts.py but sends it as user instruction text for compatibility.

References:

• https://huggingface.co/mistralai/Voxtral-Mini-3B-2507
• https://docs.vllm.ai/en/v0.11.1/examples/offline_inference/audio_language/
• https://docs.mistral.ai/models/deployment/local-deployment/vllm

Windows 11 Prerequisites

• Windows 11 with WSL2 enabled.
• Ubuntu installed in WSL2.
• Recent NVIDIA Windows driver with WSL CUDA support.
• A 16 GB NVIDIA GPU can work with the included safe launcher profile.
• 32 GB system RAM.

Install WSL2 Ubuntu from elevated PowerShell if needed:

wsl --install -d Ubuntu

Restart Windows if prompted.

Verify GPU In WSL

Open Ubuntu in WSL and run:

nvidia-smi

You should see your NVIDIA GPU. Do not install a separate Linux NVIDIA kernel driver inside WSL. WSL uses the Windows NVIDIA driver.

From this repo:

make check-gpu

Ubuntu Build Tools

vLLM/Triton can need a working C/C++ compiler at runtime. Install the standard Ubuntu toolchain inside WSL:

sudo apt update
sudo apt install -y build-essential gcc g++ make python3-dev

Verify:

gcc --version
g++ --version
cc --version
make --version

Setup

From PowerShell:

cd C:\path\to\voxtral-journal-windows
powershell -ExecutionPolicy Bypass -File .\scripts\windows_setup_wsl.ps1

Or from WSL Ubuntu:

cd /mnt/c/path/to/voxtral-journal-windows
bash scripts/wsl_install_deps_ubuntu.sh
make setup
cp .env.example .env

If your Ubuntu image does not include Python 3.11, install Python 3.11+ for your release and rerun make setup.

One-click Windows start

The easiest Windows flow starts vLLM and the Gradio UI inside WSL2, waits for both health checks, then opens the browser:

cd C:\path\to\voxtral-journal-windows
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\windows_one_click_start.ps1

You can also double-click:

start_voxtral_webui.cmd

The default one-click profile is safe, which is tuned for 16 GB GPUs:

VLLM_MAX_MODEL_LEN=8192
VLLM_GPU_MEMORY_UTILIZATION=0.90
VLLM_MAX_NUM_SEQS=1
VLLM_ENFORCE_EAGER=true
VLLM_EXTRA_ARGS=

--enforce-eager reduces CUDA graph memory at the cost of some speed. Lowering max-model-len reduces KV-cache memory. The safe profile intentionally uses --gpu-memory-utilization 0.90, not 0.98, because Windows and WSL can reserve enough VRAM that 0.98 fails on otherwise compatible 16 GB GPUs.

Optional launchers:

start_voxtral_webui_safe.cmd
start_voxtral_webui_balanced.cmd
start_voxtral_webui_high_context.cmd

Profiles:

• safe: max-model-len=8192, --enforce-eager, --max-num-seqs 1, --gpu-memory-utilization 0.90
• balanced: max-model-len=8192, --max-num-seqs 1, --gpu-memory-utilization 0.90
• high-context: max-model-len=16384, --max-num-seqs 1, --gpu-memory-utilization 0.90, may fail on 16 GB GPUs

Fast WSL runtime copy

The source repo stays on Windows:

C:\path\to\voxtral-journal-windows

The ML runtime runs from a WSL-native copy for speed:

~/apps/voxtral-journal-windows

Running Python, PyTorch, and vLLM from /mnt/s or /mnt/c can be extremely slow because every import crosses the Windows filesystem boundary. The one-click launcher first syncs the Windows source repo into ~/apps/voxtral-journal-windows, then starts vLLM and Gradio from that native WSL path.

Start from Windows:

cd C:\path\to\voxtral-journal-windows
.\start_voxtral_webui.cmd

Stop from Windows:

cd C:\path\to\voxtral-journal-windows
.\stop_voxtral_webui.cmd

Runtime logs are written in the WSL runtime copy:

~/apps/voxtral-journal-windows/data/logs/

Source files remain in:

C:\path\to\voxtral-journal-windows

The one-click launcher uses the known working 16 GB VRAM fallback:

VLLM_MAX_MODEL_LEN=8192

It does not change the canonical default in scripts/start_vllm_mini3b.sh, which remains 32768 for manual launches.

The launcher writes logs to:

• ~/apps/voxtral-journal-windows/data/logs/vllm.log
• ~/apps/voxtral-journal-windows/data/logs/vllm_launcher.log
• ~/apps/voxtral-journal-windows/data/logs/gradio.log
• ~/apps/voxtral-journal-windows/data/logs/one_click_start.log

If startup fails, it prints the relevant log tail and exits with a non-zero status. The .cmd wrapper keeps the console open on error.

To stop vLLM and Gradio without deleting logs or outputs:

cd C:\path\to\voxtral-journal-windows
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\windows_stop_stack.ps1

Or double-click:

stop_voxtral_webui.cmd

vLLM Install Notes

requirements.txt installs mistral-common[audio], but vLLM is large and CUDA-specific. Install vLLM inside the same WSL virtual environment:

cd ~/apps/voxtral-journal-windows
source .venv/bin/activate
pip install -U "vllm[audio]"
python -c "import mistral_common; print(mistral_common.__version__)"

If Hugging Face requires auth for the model files:

source .venv/bin/activate
pip install -U huggingface_hub
huggingface-cli login

Start vLLM

Use this exact model and flags:

vllm serve mistralai/Voxtral-Mini-3B-2507 \
  --host 0.0.0.0 \
  --port 8000 \
  --tokenizer_mode mistral \
  --config_format mistral \
  --load_format mistral \
  --gpu-memory-utilization 0.90 \
  --max-model-len 32768

Or run:

make start-vllm

From PowerShell:

cd C:\path\to\voxtral-journal-windows
powershell -ExecutionPolicy Bypass -File .\scripts\windows_start_vllm.ps1

Keep this terminal open.

Logged background launcher:

cd ~/apps/voxtral-journal-windows
pkill -f "vllm serve" || true
nohup bash scripts/run_vllm_logged.sh > data/logs/vllm_launcher.log 2>&1 &
tail -f data/logs/vllm.log

scripts/run_vllm_logged.sh uses the canonical command above. If you want the optional higher-context profile on hardware with more free VRAM, you can use:

VLLM_MAX_MODEL_LEN=16384 VLLM_GPU_MEMORY_UTILIZATION=0.90 VLLM_MAX_NUM_SEQS=1 bash scripts/run_vllm_logged.sh

The canonical requested command remains the default.

Health Check

From WSL:

bash scripts/test_vllm_health.sh

Expected: JSON containing mistralai/Voxtral-Mini-3B-2507.

Real Voxtral Smoke Test

After /v1/models works, run a real 30-second local transcription smoke test:

cd ~/apps/voxtral-journal-windows
. .venv/bin/activate
python scripts/smoke_vllm_audio_real.py "/mnt/c/path/to/sample-audio.m4a" --seconds 30 --language en

Or:

make smoke-vllm AUDIO="/mnt/c/path/to/sample-audio.m4a"

PowerShell helper:

powershell -ExecutionPolicy Bypass -File .\scripts\windows_smoke_vllm.ps1 -AudioPath C:\path\to\sample-audio.m4a

The script writes:

• data/smoke/smoke_30s.wav
• data/smoke/smoke_vllm_transcript.txt

Start The WebUI

Open a second terminal.

From WSL:

cd ~/apps/voxtral-journal-windows
make start-ui

scripts/start_ui.sh disables Gradio/Hugging Face telemetry environment variables for the local-first UI process.

From PowerShell:

cd C:\path\to\voxtral-journal-windows
powershell -ExecutionPolicy Bypass -File .\scripts\windows_start_ui.ps1

Then open:

http://localhost:7860

PowerShell helper:

powershell -ExecutionPolicy Bypass -File .\scripts\windows_open_ui.ps1

Processing Modes

• full-file first, fallback to chunks: sends the normalized WAV plus the journal prompt to Voxtral audio chat. If that fails due to OOM, request size, timeout, or server error, it automatically chunks and runs a second polishing pass.
• always chunk: creates 5-minute chunks with 30-second overlap by default, transcribes chunks, merges them, then polishes globally.
• raw transcription only: uses the dedicated transcription endpoint and skips journal polishing. If full-file raw transcription fails, it falls back to chunks.
• polished journal transcript: gets a raw transcript first, then applies the journal prompt to the merged text. This is useful if direct prompted audio output is less stable than raw ASR on your vLLM build.

Output Files

Each run creates a session folder under data/final_transcripts with:

• final_journal_transcript.md
• transcript.json
• raw_merged_transcript.md
• chunks.zip when chunking was used

Raw per-chunk transcript files are kept under data/raw_transcripts/<session>.

Tests

make test

The included tests validate configuration, prompt assembly, chunk range logic, output writing, and vLLM health-client behavior. They do not require a live GPU server.

Troubleshooting

CUDA Or GPU Not Visible

Run inside WSL:

nvidia-smi

If it fails, update the Windows NVIDIA driver, restart Windows, then retry WSL. Do not install a normal Linux desktop NVIDIA driver inside WSL.

vLLM Fails To Install

Make sure you are in WSL, not native Windows Python:

uname -a
python --version
pip install -U "vllm[audio]"

Use Python 3.11+ and a clean virtual environment.

vLLM Fails With Missing C Compiler

If data/logs/vllm.log contains Failed to find C compiler, install the standard Ubuntu build tools:

sudo apt update
sudo apt install -y build-essential gcc g++ make python3-dev

Then restart vLLM.

OOM Or KV Cache Errors

The default command uses --gpu-memory-utilization 0.90 and --max-model-len 32768. If vLLM fails on your current desktop load, close other GPU applications or temporarily reduce GPU utilization:

vllm serve mistralai/Voxtral-Mini-3B-2507 ... --gpu-memory-utilization 0.85

The WebUI will also fall back from full-file processing to chunks when requests fail.

vLLM uses VRAM for model weights, CUDA graphs, and KV cache. A 16 GB GPU can run this model, but it may need the safe profile to leave enough memory for the cache blocks.

If data/logs/vllm.log says the available KV cache cannot serve the chosen context, start vLLM with a smaller context and the safe profile:

VLLM_MAX_MODEL_LEN=8192 VLLM_GPU_MEMORY_UTILIZATION=0.90 VLLM_MAX_NUM_SEQS=1 VLLM_ENFORCE_EAGER=true bash scripts/run_vllm_logged.sh

If you need a higher context and have more free VRAM, try the balanced or high-context profile. If vLLM still reports the estimated maximum model length is below 8192, use a value below the estimate, for example:

VLLM_MAX_MODEL_LEN=4096 VLLM_GPU_MEMORY_UTILIZATION=0.90 VLLM_MAX_NUM_SEQS=1 VLLM_ENFORCE_EAGER=true bash scripts/run_vllm_logged.sh

If stop_voxtral_webui.cmd reports that matching processes remain, run this inside WSL and retry the launcher:

pkill -KILL -f "[v]llm serve|[m]istralai/Voxtral-Mini-3B-2507|[r]un_vllm_logged"

ffmpeg Errors

Install ffmpeg in WSL:

sudo apt-get update
sudo apt-get install -y ffmpeg
ffmpeg -version

You can manually normalize a file:

bash scripts/normalize_audio.sh path/to/audio.m4a

WSL Networking

Both services bind to 0.0.0.0. On current WSL2, Windows usually forwards localhost automatically:

• vLLM: http://localhost:8000/v1/models
• WebUI: http://localhost:7860

If Windows cannot connect, confirm the services are running in WSL and check firewall prompts.

Hugging Face Auth

If model download fails with authorization or gated-repo errors:

source .venv/bin/activate
huggingface-cli login

Then restart vLLM.

Assumptions And TODOs

• This project assumes current vLLM Voxtral support follows the official Mistral/Voxtral examples using mistral-common[audio].
• True system-role prompts are not supported by Voxtral Mini 3B per the model card, so journal instructions are sent as user-message instructions.
• If a future vLLM release changes audio request serialization, update app/vllm_client.py.