fix(core): fall back to non-multicast zenoh discovery when scouting bind fails - #2850
fix(core): fall back to non-multicast zenoh discovery when scouting bind fails#2850bobdingAI wants to merge 1 commit into
Conversation
…ind fails `open_zenoh_session_with_listen` opens the tuned zenoh session, and on failure retried with a plain `zenoh::Config::default()`. But the default keeps multicast scouting enabled, so when the *cause* of the failure is the scouting socket failing to bind its multicast group (`224.0.0.224:7446`), the retry re-attempts the identical bind, fails again, and the whole dataflow aborts at startup. This happens whenever a busy DDS/ROS 2 multicast graph shares the network with dora — e.g. running the ROS 2 bridge alongside a full TurtleBot3 / nav2 stack (dozens of FastDDS participants). Discovery on the DDS side is fine; it's dora's own zenoh scouting bind that fails, so the dataflow intermittently never starts. Disable multicast scouting on the fallback retry: gossip + explicit connect endpoints (`DORA_ZENOH_CONNECT`) still cover discovery for co-located and connect-configured setups, so degrading to non-multicast keeps dora running where it would otherwise fail outright. The primary open is unchanged, so normal (uncontended) environments still get multicast by default. Repro (dora ROS2 bridge example vs a 25-participant DDS graph, x12 runs): - busy graph, multicast on (before): 0/12 started - busy graph, multicast on (after): 12/12 started - clean graph (after): unchanged, fallback never triggers Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
No issues found. Disabling multicast scouting only on the fallback retry is a strictly-safer degradation: the fallback runs only after the primary tuned open already failed to bind, so a multicast-dependent setup wasn't getting multicast on that attempt anyway, and gossip / explicit Generated by Claude Code |
|
Automated review. The fallback direction is reasonable for the co-located / dev-container multicast-contention case this targets, and the The retry disables multicast unconditionally, but the Gating the multicast-disable on the error actually being the 🤖 This is a fully automated review by Claude (Claude Code). No human has vetted these findings — please confirm before acting on them. Generated by Claude Code |
Summary
open_zenoh_session_with_listen(libraries/core/src/topics.rs, called from thedaemon at
binaries/daemon/src/lib.rs:1225) opens the tuned zenoh session and,on failure, retried with a plain
zenoh::Config::default(). The default keepsmulticast scouting enabled, so when the failure's cause is the scouting
socket failing to bind its multicast group (
224.0.0.224:7446), the retryre-attempts the identical bind, fails again, and the dataflow aborts at startup.
This bites when dora shares one network namespace with a busy DDS/ROS 2 multicast
graph — i.e. dev containers / CI / any co-located
docker-netns setup (theexisting comment in
topics.rsalready notes multicast scouting is "broken indev containers and many CI environments"). Reproduced in
docker/ros2dev(dora +ROS 2 in one netns) alongside a ~25-participant DDS graph.
Fix
Disable multicast scouting on the fallback retry only. Gossip + explicit
connect endpoints (
DORA_ZENOH_CONNECT) still cover discovery for co-located andconnect-configured setups, so degrading to non-multicast keeps dora running where
it would otherwise fail outright. The primary open is unchanged — normal
(uncontended) environments still get multicast by default and the fallback never
triggers. It's strictly safer than today: the fallback runs only after the
primary already failed to bind multicast, so a multicast-dependent setup wasn't
getting multicast on that attempt anyway.
Scope — what this does and does NOT fix
This is the dora daemon zenoh-transport layer only. It does not touch the
ROS 2 bridge's DDS path (
Ros2Transport::Dds/Ros2Contextdefault), whichopens no zenoh session at all.
There is a separate discovery bug with a confusingly similar ~50% rate: the
ROS 2 bridge's RustDDS discovery going silently empty under DDS domain
contention (process healthy, subscription just receives nothing — no bind error,
no daemon required). This PR does not address that; it lives at the
ros2_client/rustddslayer, not in dora's zenoh session. The two share anenvironmental cause (multicast contention) but are distinct code paths.
Host deployments where dora and ROS 2 do not share a netns are not exposed to
the failure this PR fixes (measured: 10/10 dataflow starts with and without
ZENOH_CONFIGagainst a live 32-node graph, zero bind errors) — for those, theDDS-layer bug above is the relevant one.
Validation
rust-ros2-dataflowbridge example vs a ~25-participant DDS graph, 12 runs each,in
docker/ros2dev(dora + ROS 2 share one netns):224.0.0.224:7446 Address already in useretrying with multicast scouting disabled, dataflow finishescargo fmt/cargo clippy -p dora-core -- -D warningsclean.A deterministic unit test isn't practical (triggering the fallback needs a
contended-multicast environment where the primary bind fails), so this is
validated by the reproduction above.
Workaround (no rebuild)
Set
ZENOH_CONFIGto a config withscouting: { multicast: { enabled: false } },or set
DORA_ZENOH_CONNECT, to opt out of multicast scouting directly.