Commit 34a2e00
committed
fix(config): two hiroz nodes on two hosts delivered nothing to each other
Two hiroz processes on two hosts, with an `hu router` both of them reached,
exchanged nothing and reported nothing. Measured, 20 messages published per
cell, subscriber started 3s ahead, router asserted alive throughout:
| subscriber | publisher | published | received |
|---|---|---|---|
| shipped default | shipped default | 20 | 0 |
| with_mode("client") | with_mode("client") | 20 | 20 |
| with_mode("client") | shipped default | 20 | 20 |
| shipped default | with_mode("client") | 20 | 20 |
Both sessions opened, the publisher published all 20, the router stayed up,
and no error surfaced anywhere. Only the configuration a user gets by
writing `ZContextBuilder::default()` failed.
The shipped session config asked for three things that cannot hold together:
`mode: "peer"`, `scouting/multicast/enabled: false`, and
`listen/endpoints: ["tcp/localhost:0"]`. A zenoh router does not route
between two peers connected to it; the setting that used to make it do so,
`routing.router.peers_failover_brokering`, is deprecated in zenoh 1.9 and has
no effect — the router logged exactly that on every start, and this crate's
own router config was still setting it. So the two peers had to link directly
to each other, and a peer that advertises only `tcp/localhost:0` cannot be
dialled from another host. On one machine they link over loopback, which is
why every local run looked fine.
Fix: listen on `tcp/[::]:0`, which is what rmw_zenoh_cpp's own session config
leaves it at, and drop the dead `peers_failover_brokering` override. Same 2x2
after the change: 20 received in every cell, and the router's deprecation
warning count drops from 1 to 0.
Candidates rejected, with the measurement that rejected them:
- Default the session to `client`. It fixes the general case, including two
nodes that can neither dial the other. It also breaks router-less
operation: a client session FAILS TO OPEN with no router, where a peer
opens. Measured on the whole `hiroz` suite with no router running —
baseline 423 passed / 0 failed, client-mode default 8 targets failed
(tests/service 5, tests/shm 7, and more). Those tests are single-session
and legitimately need no network. Rejected on that blast radius; it is a
call for the maintainers, not a defect fix.
- Gossip multihop / peer autoconnect. Does not address the cause: discovery
already worked, the advertised locator was the unreachable part. It would
also undo the tuning next to it ("greater-zid to avoid redundant
connections", "peers send gossip only to router").
- Configure the router to broker. Not available: zenoh 1.9 deprecated the
setting with no replacement, which is the root of this.
What this fix does NOT cover, stated plainly: two peers that can neither
accept an inbound connection still cannot reach each other, because nothing
relays for them. That case needs client mode and its cost.
Blast radius of what landed: nodes now accept inbound connections on every
interface instead of loopback only — the upstream rmw_zenoh_cpp default, and
the price of peer mode working off-host. Whole `hiroz` suite after the
change: 425 passed, 0 failed.
Regression test: crates/hiroz/tests/two_session_default_config.rs. Every
pub/sub test in this crate put publisher and subscriber on ONE ZContext, so
the message never crossed a session boundary; the integration suite that uses
two endpoints builds them through create_hiroz_context_with_endpoint, which
calls .with_mode("client") and so tests the row that worked. The shipped
default had no two-session test at all. The new file adds one, plus the
assertion that actually fails at the broken revision: a shipped-default
session must advertise a listen endpoint another host can dial. Reproducing
the delivery failure itself needs two hosts, which the suite cannot arrange —
the two-session delivery test passes on one machine either way, and its doc
comment says so.1 parent 1fe6b3d commit 34a2e00
2 files changed
Lines changed: 246 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| |||
195 | 195 | | |
196 | 196 | | |
197 | 197 | | |
198 | | - | |
| 198 | + | |
199 | 199 | | |
200 | 200 | | |
201 | 201 | | |
| |||
214 | 214 | | |
215 | 215 | | |
216 | 216 | | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | 217 | | |
223 | 218 | | |
224 | 219 | | |
| |||
246 | 241 | | |
247 | 242 | | |
248 | 243 | | |
249 | | - | |
250 | | - | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
251 | 251 | | |
252 | 252 | | |
253 | 253 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
0 commit comments