Add opt-in timestamp-synced publishing for GMSL cameras - #447
Open
khaterma wants to merge 1 commit into
Open
Conversation
Add general.sync_pub_to_timestamp: when enabled, decimate publishing on the synchronized hardware image timestamp so multiple GMSL hardware-synced ZED cameras select the same grabbed frame and publish in phase (sub-ms cross-camera offset). The grab runs at full rate; a predictive enable_depth gate keeps depth compute at pub_frame_rate, and the free-running publish throttles are bypassed so the timestamp bucket gate is the sole decimator. Defaults off, so single-camera behavior is unchanged.
Member
|
@RodolpheP can you please review this and test it with our internal tools? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in
general.sync_pub_to_timestampparameter. When enabled, image/depth/point-cloudpublishing is decimated on the synchronized hardware image timestamp instead of each node's
free-running wall-clock publish timer. Multiple GMSL hardware-synced ZED cameras then select the
same grabbed frame and publish in phase (sub-millisecond cross-camera offset).
Defaults to
false, so single-camera behavior is unchanged.Motivation
When
pub_frame_rate < grab_frame_rate, publishing is decimated by a per-node, free-runningwall-clock timer (the sleep-throttle in
handleVideoDepthPublishing/handlePointCloudPublishing).Each
zed_nodeindependently decides which grabbed frame to publish based on its own timer phase,which is not tied to the camera's hardware capture timestamp and not coordinated between nodes.
As a result, two cameras that are hardware-synced and expose at the same instant still publish
different grabbed frames, offset by whole multiples of the grab period. The downstream effect is
a per-frame cross-camera timestamp mismatch of one or more grab periods.
pub_frame_rate == grab_frame_rate— every grabbed frame ispublished, so there's nothing to phase-misalign.
What changed
In
sync_pub_to_timestampmode:floor(hw_timestamp / pub_period)bucket. Because GMSL-synced cameras share the same hardwaretimestamp, they select the same frame and publish in phase.
published, so inference is throttled to
pub_frame_raterather than running at full grab rate.timestamp is visible to the bucket gate.
(the old sleep-throttle would fight it and drop boundary frames).
No separate inter-node sync/handshake logic is needed: alignment falls out automatically from the
shared hardware timestamp as long as both cameras run at the same
pub_frame_rate.New parameter
general.sync_pub_to_timestampfalsepub_frame_rate. Leavefalsefor single-camera setups.Scope / limitations
that are already hardware-synced; it does not itself synchronize independent, unsynced cameras.
pub_frame_rate.Testing
Setup
zed_A,zed_Bgrab_frame_ratepub_frame_ratepoint_cloud/cloud_registeredPer-camera publish-period stability (target 200 ms @ 5 Hz)
zed_Azed_BBefore, each camera's own publish interval swings between ~133 / 200 / 267 ms (±1–2 grab periods).
After, both hold a steady 200 ms.
Cross-camera per-frame offset (
zed_A − zed_B)Offset distribution (clusters)
Before: offsets cluster at ±1–2 grab periods; only ~25 % of frame pairs land within 1 ms.
After: every frame pair lands in the 0 ms bucket (99.8 % within 1 ms), and per-camera cadence is
rock-steady — confirming both cameras now publish the same hardware-timestamped frame in phase, with
GPU/depth compute throttled to the publish rate.