Skip to content

Add support for raw RTP over UDP video - #2961

Draft
rafaellehmkuhl wants to merge 2 commits into
bluerobotics:masterfrom
rafaellehmkuhl:add-udp-video-ingest
Draft

Add support for raw RTP over UDP video#2961
rafaellehmkuhl wants to merge 2 commits into
bluerobotics:masterfrom
rafaellehmkuhl:add-udp-video-ingest

Conversation

@rafaellehmkuhl

Copy link
Copy Markdown
Member

Adds the video source type that ArduPilot and PX4 companion computers actually emit, matching QGroundControl's "UDP h.264 / h.265 Video Stream" options. The video configuration page gains a listen address, a port and a codec selector, and the stream then behaves like any other: usable by video widgets, recordable, and visible in stats-for-nerds.

Standalone only, since a browser cannot open a UDP socket. The README limitation table is updated accordingly.

How it works

Cockpit already bundles both ffmpeg and go2rtc, and the go2rtc service already forwards an arbitrary source string, so no new dependency or sidecar is involved. Raw RTP describes neither its codec nor its clock rate, which is why the codec has to be picked by hand rather than discovered, exactly as QGC hardcodes it into its GStreamer caps. That choice becomes a synthesized SDP which reaches go2rtc inline as a base64 data: URI.

Two things forced that design, both found by testing against the real binary rather than the docs:

  • go2rtc's HTTP API rejects exec: sources outright, so the ffmpeg invocation cannot travel in the source string and lives in a named ffmpeg: input template in the config instead.
  • go2rtc also rejects any source containing whitespace, so the SDP could never be referenced as a temp file whose path has a space in it. Hence the inline data: URI, which also means there is no temp file to clean up.

Both codecs are passed through without re-encoding.

Commits

  • electron: go2rtc: pin the RTSP and WebRTC servers to free local ports — go2rtc was started with only its api block configured, leaving RTSP on :8554 and WebRTC on :8555 bound to every interface, where both collide with MediaMTX or a second go2rtc. This already affected today's RTSP users, so it is separated out as its own reviewable and backportable change.
  • feat: video: add raw RTP over UDP stream sources — the feature itself, purely additive.

The three sites that inferred a stream's protocol by string-sniffing an rtsp:// prefix now share one helper, and the protocol chip and the stats panel treat every go2rtc-backed protocol alike instead of special-casing RTSP.

Why this is a draft

Verification

  • Fed a synthetic sender into the real bundled go2rtc for both codecs and confirmed frames flow with the expected codec and rising packet counts. The H.265 receiver reports hevc, which is the proof of passthrough: had ffmpeg re-encoded, it would have reported h264 there.
  • Unit tests in src/tests/libs/rtp-source.test.ts cover the URI round-trip, config validation, SDP contents for both codecs, and that the generated source never contains whitespace.
  • yarn lint clean at --max-warnings=0, production build passes.

Note when testing recording on this branch: recordings go through the same MediaRecorder path as every other stream, and that path records in passthrough mode, so a UDP H.265 stream produces an H.265 recording. There is an open report of H.265 recording failing (related to #1725) which is not specific to this PR but will be reachable through it.

Test plan

  • H.264 stream from a real companion computer (... -f rtp rtp://<topside>:5600) plays, records, and reports sane stats
  • H.265 stream, on top of Bump Electron to 43.4.1 and enable H.265 (HEVC) support #2672
  • A wrong codec choice, or nothing sending, fails visibly rather than silently
  • Existing RTSP and WebRTC streams still work, given the go2rtc port change

go2rtc was started with only its api block configured, leaving the RTSP
server on :8554 and the WebRTC one on :8555, bound to every interface.
Both collide with MediaMTX or with a second go2rtc, and neither needs to
be reachable from outside this machine.
Adds the source type ArduPilot and PX4 companion computers actually
emit, matching QGroundControl's UDP h.264/h.265 options. The video
configuration page gains a listen address, port and codec, and the
bundled ffmpeg ingests the stream through the go2rtc sidecar, so no new
dependency or sidecar is involved. Standalone only, as the browser
cannot open a UDP socket.

Raw RTP describes neither its codec nor its clock rate, which is why the
codec has to be picked by hand rather than discovered. It becomes a
synthesized SDP, the same information QGroundControl hardcodes into its
GStreamer caps, and reaches go2rtc inline as a base64 data: URI because
go2rtc rejects sources containing whitespace and so could never
reference a temp file whose path has a space in it.

Both codecs are passed through without re-encoding. H.265 therefore
depends on the Chromium HEVC switches, so it needs the Electron 41 bump
of bluerobotics#2672 to reach the renderer.

The three sites that inferred a stream's protocol by string-sniffing an
rtsp:// prefix now share one helper, and the protocol chip and the
stats-for-nerds panel treat every go2rtc-backed protocol alike instead
of special-casing RTSP.
@rafaellehmkuhl rafaellehmkuhl changed the title Receive raw RTP over UDP video, as ArduPilot and PX4 companions send it Add support for raw RTP over UDP video Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant