voice/otel: trace export ships inline base64 audio in span attributes (separate path from #451)

[drewdrewthis]

Background

While verifying the voice audio-externalization fix (#451 / commit 180bab4 + follow-up on the event-reporter), wire-capturing the SDK's actual POSTs during a voice scenario run surfaced a second, independent inline-audio channel that neither fix addresses: the OpenTelemetry trace export.

Current-state violations

Captured from a live scenario.run() (TS SDK, OpenAI Realtime voice agent), POSTs to /api/otel/v1/traces:

• 3 trace events of 0.95–1.7 MB each, carrying raw audio inline as the AI-SDK file part: {"type":"file","mediaType":"audio/pcm16","data":"<~507,000-char base64>"}.
• This is the un-translated file shape (not even input_audio) — the OTEL path predates and bypasses the AG-UI content converter entirely.
• javascript/src/tracing/filters.ts has no audio handling (no redaction, externalization, or size cap), and javascript/src/tracing/setup.ts sets no span-attribute size limit. So full PCM16 audio flows into span attributes → ClickHouse spans.

Per scenario this is ~3–5 MB of inline audio on the OTEL channel in addition to the scenario-events channel; across a 10-scenario set it compounds (the same class of bloat as the 90 MB getSuiteRunData issue, different table/path).

Why now

The scenario-events inline-audio path is being fixed (array + input_audio + backend extractor externalization). The OTEL trace path is the other half of "audio is flooding production inline" and is currently untracked. A voice agent's spans will keep carrying multi-MB base64 audio until the span exporter has an audio filter/externalizer.

Scope notes (for investigation to refine)

• Add an audio-aware span-attribute filter/processor in tracing/ that redacts or externalizes file/audio/* (and input_audio) payloads before export — mirror whatever externalization the scenario-events extractor uses, or strip to a reference.
• Consider a general span-attribute size cap as defense-in-depth.
• Verify with the same wire-capture method (point OTEL exporter at a local sink, assert no >100KB base64 in exported spans).

---

Surfaced via wire-capture during #451 verification. Separate path from the scenario-events fix.

[drewdrewthis]

Correction / re-scope after deeper prod + backend-source verification:

The original framing ("ships inline base64 audio → 90MB storage leak") is inaccurate for storage. Pulling real prod traces via /api/trace/{id} shows the langwatch backend truncates large span I/O on store to [truncated: N bytes] / [AUDIO: ~N bytes] placeholders — so the OTEL trace path is not a 90MB storage leak.

What remains true and worth tracking:

• The SDK does put full base64 audio on the wire to /api/otel/v1/traces (~0.95–1.7MB/event in my capture) — a bandwidth cost, not storage.
• The traces path is not the see+listen surface anyway — the conversation panel reads scenario-events → simulation_runs, where audio externalization is handled by #4493 (langwatch backend) turning audio into /api/files/<id> refs.

Re-scoped ask: add an audio-aware filter to the span exporter so multi-MB base64 isn't shipped over the wire in trace attributes (externalize/strip before export), purely as a bandwidth/efficiency fix. This is independent of and lower-priority than the scenario-events audio path (#4493), which is what actually drives playable audio.