fix(core): prevent cross-origin Web Audio capture from silencing audio

[desenmeng]

What

Prevent AudioContext.createMediaElementSource() from silently taking over and zeroing cross-origin <audio> elements that were loaded without a crossorigin opt-in.

• Classify each audio element before Web Audio capture as web-audio, decode-only, or explicitly native.
• Keep the existing fetch/decode fallback for automatically bypassed cross-origin media, preserving the FX graph when the server permits CORS.
• Add data-native-audio as a per-element escape hatch for redirects and other cases URL inspection cannot settle.
• Surface preview-only diagnostics through hyperframes check, including processing that native fallback cannot reproduce.
• Document the new attribute and add regression coverage across core runtime, transport, and CLI checks.

Why

The Web Audio specification requires a MediaElementAudioSourceNode created from media that fails the CORS-cross-origin check to output silence. The call succeeds and does not throw, so the existing try/catch cannot recover. Creating the node also permanently reroutes the element away from native output for its lifetime.

The result is a composition whose timeline and visuals continue normally while its audio is completely silent, with no error or diagnostic. Guarding before node creation preserves audible native playback instead.

Fixes #3458.

How

The runtime now uses a pure route classifier at discovery and scheduling time:

• Same-origin, CORS-opted-in, and non-HTTP(S) sources keep the existing Web Audio path.
• Cross-origin HTTP(S) sources without a crossorigin attribute skip createMediaElementSource() but still attempt fetch plus decodeAudioData; native playback remains the final fallback.
• data-native-audio skips both Web Audio capture and decode.

Source selection follows currentSrc, then src, then <source> candidates. The transport also rechecks the route immediately before the irreversible capture call. Diagnostics are emitted during preview discovery so hyperframes check can observe them without playback, but are suppressed during export rendering, where audio is mixed offline. The existing non-unit-rate fail-closed policy remains limited to data-fx-chain and data-automation.

Test plan

• Unit tests added/updated
• Manual testing performed
• Documentation updated (if applicable)

Verified locally:

• Core focused runtime tests: 183 passed
• CLI checkBrowser tests: 16 passed
• Core and CLI typechecks passed
• Changed-file formatting and lint checks passed

[miga-heygen]

Reviewed the diff (docs/reference/html-schema.mdx, packages/cli/src/utils/checkBrowser.ts(.test.ts), packages/core/src/runtime/init.ts, webAudioRoute.ts (new), webAudioTransport.ts(.test.ts), init.test.ts) plus the un-diffed decodeAudioElement/scheduleWebAudioForActiveClips call sites to check a perf hypothesis below.

Summary

Fixes #3458: AudioContext.createMediaElementSource() silently outputs silence (no throw) for cross-origin <audio> with no crossorigin opt-in, and permanently steals the element's native output on the way. The fix introduces a pure classifyWebAudioMediaRoute classifier (web-audio / decode-only / native) consulted at three points — discovery (loadedmetadata + immediate call), schedule time in init.ts, and as an enforcement backstop inside WebAudioTransport.acquireMediaElementSource — plus a data-native-audio escape hatch and a hyperframes check-visible web_audio_bypass finding.

Correctness

• Verified the classifier's origin/candidate-selection logic against the HTML resource-selection algorithm (currentSrc > src > <source> children) — matches spec priority and is well tested, including the "browser already committed via currentSrc" and "selection still unsettled, be conservative across all <source> candidates" cases.
• crossorigin enumerated-attribute handling (presence, not value, is the opt-in) is correct and tested against anonymous/use-credentials/empty/garbage values.
• data-native-audio correctly short-circuits both Web Audio capture and decode (a decode success would otherwise re-mute the element via a buffer source) — this is enforced independently in webAudioTransport.ts, not just init.ts, so a direct caller (studio/player) can't reopen the one-way door. Good defense-in-depth, tested (webAudioTransport.test.ts).
• The non-unit-playback-rate fail-closed mute rule is correctly scoped to stay web-audio-route-only (route.kind === "web-audio" && hasProcessing) so it doesn't newly mute grouped/above-unity-volume tracks on the new decode-only route — good catch on scope creep, and it's explicitly tested against regressing pre-#3458 behavior.
• Chased a perf hypothesis: since "decode-only" is now a mainline path for ordinary cross-origin audio (not just a rare fallback), does webAudio.decodeAudioElement() get called repeatedly (e.g. once per animation frame)? Confirmed no — decodeAudioElement has an internal _bufferCache/_failedSrcs keyed by src, and scheduleWebAudioForActiveClips() (the only caller of this scheduling path) is invoked only on discrete transitions (play, playback-rate change, hidden-audio dirty), not per rAF tick. No redundant fetch/decode risk.
• Render-mode gating (isRenderMode() checked before the diagnostic and before the latch) matches the documented rationale — render mixes offline and reproduces the FX chain, so reporting a "bypass" there would be a false claim, and it's confirmed the once-per-element latch isn't consumed by a render-mode call (tested).

Type safety

• RuntimeJson recursively permits string[], so lostProcessing: string[] in the diagnostic details typechecks against Record<string, RuntimeJson>. No unsafe casts introduced; getAttr/hasAttr guard against non-Element inputs defensively.

Architecture / SSOT

• classifyWebAudioMediaRoute is called at up to three points (discovery in init.ts, schedule loop in init.ts, and acquireMediaElementSource in webAudioTransport.ts). This reads as intentional defense-in-depth (comment: "this stays the enforcement point so a direct caller cannot reopen the one-way door") rather than duplicated decision logic — it's the same pure function called from multiple call sites, not reimplemented, and reportWebAudioMediaRoute's WeakSet latch keeps the diagnostic single-fire regardless of how many times it's (re)classified. No SSOT violation.
• nativeUnexpressibleProcessing() is intentionally wider (adds audio-group, above-unity-gain) than the schedule loop's own hasProcessing check (fx-chain/automation only) — the PR comments explicitly call out why these answer different questions (diagnostic = "what's lost", fail-closed rule = "is losing FX worse than silence"). Documented rather than accidental drift.

Edge cases

• Non-http(s) schemes (blob:, data:, file:) correctly fall through to the pre-existing web-audio behavior rather than guessing — reasonable, avoids new regressions on schemes the origin check can't meaningfully judge.
• <video> is correctly excluded (reportWebAudioRoute checks instanceof HTMLAudioElement) since only <audio> reaches createMediaElementSource per the existing capture path — tested.
• One minor question, not a blocker: reportWebAudioMediaRoute returns early (before adding to the diagnosedElements latch) when route.kind === "native" and nothing is lost. That's correct behavior (no spam, "honoring a request isn't a finding"), but it does mean that specific case recomputes nativeUnexpressibleProcessing() on every call for the lifetime of the element instead of latching a no-op result — cheap, so not worth blocking on, just noting for anyone chasing a future perf ticket.

Tests / docs

• Coverage is thorough: classifier unit tests, transport-level enforcement tests, CLI scraper prefix-anchoring test (verifies a composition author's own console.info that merely mentions the code isn't promoted into a finding), and an end-to-end init.test.ts suite covering the full cross-origin/native/same-origin/video matrix plus the fail-closed-rule regression guard.
• Docs (html-schema.mdx) accurately describe the new attribute and match nativeUnexpressibleProcessing()'s actual dropped-feature list (fx-chain, automation, audio-group, above-unity volume).

Verdict

Solid, well-reasoned fix for a nasty silent-failure class of bug (spec-mandated silence instead of a throw). Didn't find correctness, type-safety, or SSOT issues worth blocking on — the multi-call-site classification is deliberate defense-in-depth, and the perf concern I chased down doesn't materialize since decode is cached by src and the scheduling path only runs on discrete state transitions.

— Miga

[miga-heygen]

Changes Requested

Remove the data-native-audio escape hatch — the automatic detection is sufficient.

The classifyWebAudioMediaRoute classifier and isCorsSilenced origin check already cover the core bug (#3458): cross-origin <audio> elements without a crossorigin attribute are correctly routed to decode-only or native fallback. That's the fix, and it's solid.

The data-native-audio attribute handles a narrow edge case (same-origin URLs that redirect to a cross-origin CDN at request time), but URL inspection can't detect that scenario anyway — and authors who hit it can use the standard HTML solution: add crossorigin to the <audio> element, assuming the CDN sends CORS headers.

What to remove:

• The HF_NATIVE_AUDIO_ATTR constant and all references to data-native-audio
• The authored_opt_out reason branch in classifyWebAudioMediaRoute
• The nativeUnexpressibleProcessing function (only used for the native opt-out diagnostic)
• The web_audio_bypass finding in hyperframes check (tied to the attribute)
• All test cases specific to data-native-audio behavior
• The doc-table row describing the attribute

Keep everything else — the automatic origin detection, the decode-only route, the transport-layer enforcement, the WeakSet diagnostic latch, and the existing tests for cross-origin behavior are all good.

— Miga