Skip to content

Add opt-in timestamp-synced publishing for GMSL cameras - #447

Open
khaterma wants to merge 1 commit into
stereolabs:masterfrom
khaterma:timestamp-synced-publishing
Open

Add opt-in timestamp-synced publishing for GMSL cameras#447
khaterma wants to merge 1 commit into
stereolabs:masterfrom
khaterma:timestamp-synced-publishing

Conversation

@khaterma

@khaterma khaterma commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Adds an opt-in general.sync_pub_to_timestamp parameter. When enabled, image/depth/point-cloud
publishing is decimated on the synchronized hardware image timestamp instead of each node's
free-running wall-clock publish timer. Multiple GMSL hardware-synced ZED cameras then select the
same grabbed frame and publish in phase (sub-millisecond cross-camera offset).

Defaults to false, so single-camera behavior is unchanged.

Motivation

When pub_frame_rate < grab_frame_rate, publishing is decimated by a per-node, free-running
wall-clock timer (the sleep-throttle in handleVideoDepthPublishing / handlePointCloudPublishing).
Each zed_node independently decides which grabbed frame to publish based on its own timer phase,
which is not tied to the camera's hardware capture timestamp and not coordinated between nodes.

As a result, two cameras that are hardware-synced and expose at the same instant still publish
different grabbed frames, offset by whole multiples of the grab period. The downstream effect is
a per-frame cross-camera timestamp mismatch of one or more grab periods.

  • The problem does not appear when pub_frame_rate == grab_frame_rate — every grabbed frame is
    published, so there's nothing to phase-misalign.
  • It appears specifically when publishing/sampling slower than grab (e.g. 15 Hz grab, 5 Hz publish).
  • Under system load the phase drift grows and the timestamp gap gets larger.

What changed

In sync_pub_to_timestamp mode:

  1. Timestamp bucket gate (grab thread) — publish only the first grabbed frame of each
    floor(hw_timestamp / pub_period) bucket. Because GMSL-synced cameras share the same hardware
    timestamp, they select the same frame and publish in phase.
  2. Predictive depth gating — depth (GPU) is computed only on the frames that will actually be
    published, so inference is throttled to pub_frame_rate rather than running at full grab rate.
  3. Grab runs at full rate — the SDK grab-compute cap is lifted so every frame's hardware
    timestamp is visible to the bucket gate.
  4. Free-running publish throttle bypassed — in sync mode the bucket gate is the sole decimator
    (the old sleep-throttle would fight it and drop boundary frames).

No separate inter-node sync/handshake logic is needed: alignment falls out automatically from the
shared hardware timestamp as long as both cameras run at the same pub_frame_rate.

New parameter

Parameter Default Description
general.sync_pub_to_timestamp false Decimate publishing on the synchronized hardware image timestamp so GMSL hardware-synced cameras publish in phase. Grab runs at full rate; depth compute stays at pub_frame_rate. Leave false for single-camera setups.

Scope / limitations

  • Applies to cameras that share a hardware timestamp source (same GMSL sync). It aligns cameras
    that are already hardware-synced; it does not itself synchronize independent, unsynced cameras.
  • Both cameras must run at the same pub_frame_rate.

Testing

Setup

Item Value
Compute NVIDIA Jetson AGX Orin 64 GB
ROS ROS 2 Humble
Cameras 2× ZED, GMSL hardware-synced (shared capture timestamp) — zed_A, zed_B
grab_frame_rate 15 Hz (grab period ≈ 66.7 ms)
pub_frame_rate 5 Hz (publish period = 200 ms)
Measured topic point_cloud/cloud_registered
Samples ~530 publish intervals / ~531 frame pairs per run
System idle

Cross-camera offsets appear as multiples of the 66.7 ms grab period — i.e. publish-decimation
phase between independent zed_node timers, not capture desync.

Per-camera publish-period stability (target 200 ms @ 5 Hz)

Camera Before — mean Before — max |div| After — mean After — max |div|
zed_A 199.9 ms 133.8 ms 200.0 ms 1.3 ms
zed_B 200.0 ms 68.4 ms 200.0 ms 0.3 ms

Before, each camera's own publish interval swings between ~133 / 200 / 267 ms (±1–2 grab periods).
After, both hold a steady 200 ms.

Cross-camera per-frame offset (zed_A − zed_B)

Metric Before (off) After (on)
mean −39.8 ms +0.0 ms
min −134.0 ms −0.4 ms
max +133.4 ms +1.3 ms
within 1 ms 132 / 531 (24.9 %) 525 / 526 (99.8 %)

Offset distribution (clusters)

Offset Before (count) After (count)
−133 ms 18
−67 ms 337
0 ms 133 526
+67 ms 30
+133 ms 13

Before: offsets cluster at ±1–2 grab periods; only ~25 % of frame pairs land within 1 ms.
After: every frame pair lands in the 0 ms bucket (99.8 % within 1 ms), and per-camera cadence is
rock-steady — confirming both cameras now publish the same hardware-timestamped frame in phase, with
GPU/depth compute throttled to the publish rate.

Add general.sync_pub_to_timestamp: when enabled, decimate publishing on
the synchronized hardware image timestamp so multiple GMSL hardware-synced
ZED cameras select the same grabbed frame and publish in phase (sub-ms
cross-camera offset). The grab runs at full rate; a predictive enable_depth
gate keeps depth compute at pub_frame_rate, and the free-running publish
throttles are bypassed so the timestamp bucket gate is the sole decimator.
Defaults off, so single-camera behavior is unchanged.
@Myzhar
Myzhar requested a review from RodolpheP July 8, 2026 15:02
@Myzhar Myzhar self-assigned this Jul 8, 2026
@Myzhar Myzhar added the feature_request New feature or request label Jul 8, 2026
@Myzhar

Myzhar commented Jul 8, 2026

Copy link
Copy Markdown
Member

@RodolpheP can you please review this and test it with our internal tools?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature_request New feature or request

Development

Successfully merging this pull request may close these issues.

2 participants