Add on-demand GPU acceleration for AMD GPUs (ROCm/HIP) — parity with NVIDIA/CUDA

[csmashe]

Summary

TypeWhisper for Linux currently supports on-demand GPU acceleration for NVIDIA cards via automatic CUDA runtime provisioning. There is no equivalent path for AMD GPUs. This issue tracks adding the same "it just works" local GPU acceleration experience for AMD hardware (ROCm/HIP), so AMD users get the same speedups that NVIDIA users already enjoy for both whisper.cpp and sherpa-onnx (Parakeet) backends.

⚠️ Help wanted — I do not own an AMD GPU, so I can't develop or test this myself. This needs a contributor with AMD hardware to implement and validate it. Design discussion, partial PRs, and testing reports are all welcome.

What NVIDIA already does (the bar to match)

On Linux, NVIDIA acceleration provisions its CUDA libraries on demand instead of requiring the user to install a full system CUDA toolkit:

• Detects which CUDA .sos the host already has and downloads only the missing ones into a managed bundle directory under ~/.local/share/TypeWhisper/Runtimes/.
• Loads the native runtime in the correct dependency order before the engine starts, so both whisper.cpp and sherpa-onnx pick up GPU execution automatically.
• Falls back cleanly to CPU when no compatible GPU is present.

The goal is to deliver the same automatic, low-friction experience for AMD users.

Proposed scope

• Detect AMD GPUs and a usable ROCm/HIP environment on Linux.
• Provision the required AMD runtime libraries on demand (mirroring the NVIDIA provisioner pattern), or clearly document any unavoidable system prerequisites.
• Wire AMD GPU execution into the whisper.cpp backend (e.g. ROCm/HIP build) for transcription.
• Wire AMD GPU execution into the sherpa-onnx (Parakeet) backend, depending on what ONNX Runtime exposes for AMD (ROCm / MIGraphX execution providers).
• Graceful CPU fallback and a clear status surface when AMD acceleration is unavailable or misconfigured.
• Documentation for AMD users on what's supported and any requirements.

Open questions / things to investigate

• Which AMD compute stack to target — ROCm/HIP is the obvious candidate, but coverage varies a lot by GPU generation and distro.
• Whether the upstream prebuilt binaries (whisper.cpp, sherpa-onnx / ONNX Runtime) ship AMD-capable builds we can download, or whether we'd need to build/host them ourselves.
• Minimum supported GPUs and ROCm versions.
• How to keep this consistent with the existing NVIDIA provisioning architecture so the two share as much code as possible.

Reference

The existing NVIDIA implementation can serve as the template for the AMD equivalent (runtime detection → on-demand download → ordered native load → backend wiring → CPU fallback).