docs(agents): realtime A2A connection guide for deployed agents - #2161
Merged
Conversation
Agents deployed in taOS are expected to hold a live bus connection rather than chatting in their own CLI: a TUI conversation is invisible to memory, audit and observability, which is the reason to run an agent inside the OS at all. Documents the connection pattern proven in a live Hermes deployment (curl subprocess per channel piped into a parser, cheap cron pre-check so quiet ticks cost no model tokens) along with the parts that are easy to get wrong: http.client hangs on SSE keepalives, the proxy's 25s ': ping' comments must be ignored rather than parsed as events, and channel is required. Also records two field incidents as rules. A shared user account means a shared crontab, and an installer that dedups cron lines by script basename silently deleted a peer agent's backup watch, so dedup by full path and verify the whole table after writing. And a reply posted to the wrong channel left an agent blocked for 40 minutes, so answer where you were asked or redirect explicitly. Stream durability is stated as a requirement rather than advice: SSE connections die on network blips, so a supervisor plus a ~2 hourly backup poll are both mandatory, and health is measured by a watermark advancing rather than by the process being alive.
|
Warning Review limit reached
Next review available in: 36 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
PR Summary by QodoDocs: add realtime A2A bus connection guide for deployed agents
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1. Ping heartbeat guarantee wrong
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+95
to
+99
| - **Deduplicate your own cron entry by full path**, never by basename: | ||
| `grep -v "taos-website-agent/backup_watch"`, not `grep -v "backup_watch"`. | ||
| - **After any crontab write, print the whole table and confirm entries you did | ||
| not write are still present.** A write that clobbers a peer is not detectable | ||
| from your own entry looking correct. |
There was a problem hiding this comment.
2. Unsafe cron dedupe example 🐞 Bug ☼ Reliability
Section 4 warns to deduplicate cron entries by full path, but the example uses an unanchored substring grep (grep -v "taos-website-agent/backup_watch") that is not a full path match and can still delete a peer agent’s cron line. This undermines the incident-driven safety rule the doc is trying to enforce.
Agent Prompt
## Issue description
The documentation instructs readers to dedup cron entries by full path, but the provided `grep -v` example is a substring match (and not even a full absolute path), so it can still remove other agents’ entries in shared-account setups.
## Issue Context
This section is specifically documenting a production incident caused by unsafe dedup logic, so the example should be copy/paste safe.
## Fix Focus Areas
- docs/agent-join-kit/realtime-a2a.md[95-99]
## Suggested fix direction
- Replace the example with an exact-match pattern (e.g., `grep -Fv -- "/abs/path/to/backup_watch.py"`).
- Even safer: recommend writing a unique marker comment/tag (e.g., `# taos-agent:<canonical_id>`) and dedup by that exact marker rather than any path substring.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+125
to
+128
| - The proxy injects a `: ping` comment every 25 seconds so an idle stream is | ||
| distinguishable from a dead one. **Your parser must ignore comment lines** | ||
| (anything starting with `:`) rather than treating them as malformed events. | ||
| - Absence of pings is a positive signal that the stream is dead. Worth acting on. |
There was a problem hiding this comment.
3. Ping heartbeat guarantee wrong 🐞 Bug ≡ Correctness
The new guide claims the proxy injects a : ping every 25s and that absence of pings indicates a dead stream, but the current SSE proxy only checks/emits the heartbeat inside the upstream aiter_lines() loop. If the upstream is truly idle (emits no lines), the proxy cannot emit periodic pings, so “no pings == dead” is not a reliable signal.
Agent Prompt
## Issue description
`docs/agent-join-kit/realtime-a2a.md` asserts a strict heartbeat behavior (`: ping` every 25s) and recommends treating missing pings as a dead-stream signal. The current implementation in `tinyagentos/routes/a2a_bus.py` cannot guarantee emitting pings during true upstream idleness because the heartbeat check only runs when an upstream line is received.
## Issue Context
This is not just a wording nuance: agents following the doc may falsely declare the stream dead (or fail to get the intended intermediary keepalive behavior), because the promised periodic pings may never arrive.
## Fix Focus Areas
- docs/agent-join-kit/realtime-a2a.md[125-128]
- tinyagentos/routes/a2a_bus.py[45-48]
- tinyagentos/routes/a2a_bus.py[168-193]
## Fix options
1) **Preferred:** Fix the proxy to emit `: ping` on a timer independent of upstream lines (e.g., `asyncio.wait()` between a read task and a sleep task; yield ping when the sleep wins).
2) **Doc-only fallback:** Soften/remove the guarantee and the “absence of pings implies dead” rule (e.g., say pings *may* appear and should be ignored, but health should be measured by watermark/progress).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+138
to
+140
| The 30-minute cron is the floor, not the target. Agents deployed in taOS should | ||
| hold a live bus connection and use polling as the backstop: see | ||
| [realtime-a2a.md](realtime-a2a.md). |
There was a problem hiding this comment.
4. Polling vs realtime unclear 🐞 Bug ⚙ Maintainability
docs/agent-join-kit/README.md now says deployed agents “should hold a live bus connection,” but the same document’s introduction still frames the join-kit as a polling-only model with no live transport required. This mixed messaging can lead to agents skipping SSE when it’s operationally expected, or thinking SSE is mandatory when the join-kit loop still supports polling.
Agent Prompt
## Issue description
The join-kit README now contains both:
- an introduction emphasizing the 30-minute cron/polling model without live transport
- a newly added paragraph saying taOS-deployed agents should hold a live connection
This is likely intended as “minimum supported” vs “recommended for deployed agents,” but it reads like a contradiction.
## Issue Context
The new `realtime-a2a.md` guide is being positioned as authoritative for deployed agents.
## Fix Focus Areas
- docs/agent-join-kit/README.md[3-6]
- docs/agent-join-kit/README.md[132-140]
## Suggested doc change
Add a one-sentence clarification in the intro or near the new paragraph: polling-only is the minimum viable model, while SSE is recommended/expected for agents actually deployed inside taOS (with polling as backstop).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Hermes revised the pattern after the first writeup: the parser now fires the session directly on each new message rather than a cron noticing it later, so latency is a second rather than up to a minute. Records the two edges they hit. The curl subprocess does not inherit a useful PATH, so the trigger needs an absolute path or it fails silently. And one curl per channel means a message in two watched channels triggers twice, which is only safe because their scheduler rejects the duplicate; any harness without that dedup needs its own lock or a burst spawns racing sessions. The incoming file stays even though the trigger is immediate: it is what makes the wake durable and what the backup poll reconciles against. Polling is kept as a documented fallback for harnesses that cannot be triggered externally, and explicitly distinguished from the 2 hourly backup poll, which covers a different failure.
Field correction from a second agent hit by the same shared-crontab wipe. The guide already said to measure health by a watermark advancing rather than by liveness, which was right but incomplete: their watermark check only ran as part of the backup job itself, so when the schedule was deleted the job never ran, the check never ran, and the silence read as calm. They found out only because a peer reported the incident on the bus. So the check has to run on its own schedule and confirm the job is still scheduled, not just that its last run looked healthy. Also records why nobody saw the deletions for hours: every check on that box printed the crontab through a grep matching only its own lines. A filter that hides peers hides the damage you did to them, and it had additionally concealed an unrelated scheduled job for days. Hence: print unfiltered, pin the expected table with owners so 'confirm the others survived' is actionable, and never quietly remove a block you do not recognise.
Reported from the field as 'since does not suppress replay'. It does; the parameter is a float epoch seconds cursor and the clients were passing message ids into it. since=1444 is read as 1970, which means everything, so a reconnect pulls the whole retained history. Verified against a live bus rather than taken on report: since=<id> replayed from id 1247, no since at all replayed nothing, and since=<now-120> returned only the message from the last two minutes. The proxy signature confirms it, since: float | None. Worth calling out because of how it fails. A client that passes an id works perfectly in testing, where there is no history, and floods only in production where there is. Also tells clients to drop already-seen ids on receipt regardless, since a correct timestamp can still straddle a boundary and re-deliver.
Code Review Roast 🔥Verdict: No New Issues Found | Recommendation: Address existing comments before merge Overview
My independent review of all changed code found no additional issues beyond the 4 existing Qodo inline comments. The existing findings cover the real problems:
Assessment🏆 Best part: The field-incident-driven approach. Documenting real production failures (shared crontab wipe, wrong-channel replies, watermark checker that died with its subject) is exactly the right way to write operational docs. This isn't hypothetical advice — it's scar tissue. 💀 Worst part: The ping heartbeat guarantee. If the proxy can't actually emit periodic pings during upstream idleness, the doc is teaching agents to trust a signal that doesn't exist. That's the kind of thing that only fails at 3 AM in production. 📊 Overall: Like a senior engineer's war stories transcribed into markdown — valuable, specific, and occasionally tripping over its own assumptions. Fix the four flagged issues and this is solid. Files Reviewed (2 files)
Reviewed by kat-coder-pro-v2.5 · Input: 125.8K · Output: 14.1K · Cached: 146.4K |
hognek
pushed a commit
to hognek/tinyagentos
that referenced
this pull request
Jul 29, 2026
…fc#2161) * docs(agents): realtime A2A connection guide for deployed agents Agents deployed in taOS are expected to hold a live bus connection rather than chatting in their own CLI: a TUI conversation is invisible to memory, audit and observability, which is the reason to run an agent inside the OS at all. Documents the connection pattern proven in a live Hermes deployment (curl subprocess per channel piped into a parser, cheap cron pre-check so quiet ticks cost no model tokens) along with the parts that are easy to get wrong: http.client hangs on SSE keepalives, the proxy's 25s ': ping' comments must be ignored rather than parsed as events, and channel is required. Also records two field incidents as rules. A shared user account means a shared crontab, and an installer that dedups cron lines by script basename silently deleted a peer agent's backup watch, so dedup by full path and verify the whole table after writing. And a reply posted to the wrong channel left an agent blocked for 40 minutes, so answer where you were asked or redirect explicitly. Stream durability is stated as a requirement rather than advice: SSE connections die on network blips, so a supervisor plus a ~2 hourly backup poll are both mandatory, and health is measured by a watermark advancing rather than by the process being alive. * docs(agents): direct session trigger instead of a polling cron Hermes revised the pattern after the first writeup: the parser now fires the session directly on each new message rather than a cron noticing it later, so latency is a second rather than up to a minute. Records the two edges they hit. The curl subprocess does not inherit a useful PATH, so the trigger needs an absolute path or it fails silently. And one curl per channel means a message in two watched channels triggers twice, which is only safe because their scheduler rejects the duplicate; any harness without that dedup needs its own lock or a burst spawns racing sessions. The incoming file stays even though the trigger is immediate: it is what makes the wake durable and what the backup poll reconciles against. Polling is kept as a documented fallback for harnesses that cannot be triggered externally, and explicitly distinguished from the 2 hourly backup poll, which covers a different failure. * docs(agents): a monitor that dies with its subject is not a monitor Field correction from a second agent hit by the same shared-crontab wipe. The guide already said to measure health by a watermark advancing rather than by liveness, which was right but incomplete: their watermark check only ran as part of the backup job itself, so when the schedule was deleted the job never ran, the check never ran, and the silence read as calm. They found out only because a peer reported the incident on the bus. So the check has to run on its own schedule and confirm the job is still scheduled, not just that its last run looked healthy. Also records why nobody saw the deletions for hours: every check on that box printed the crontab through a grep matching only its own lines. A filter that hides peers hides the damage you did to them, and it had additionally concealed an unrelated scheduled job for days. Hence: print unfiltered, pin the expected table with owners so 'confirm the others survived' is actionable, and never quietly remove a block you do not recognise. * docs(agents): since is an epoch timestamp, not a message id Reported from the field as 'since does not suppress replay'. It does; the parameter is a float epoch seconds cursor and the clients were passing message ids into it. since=1444 is read as 1970, which means everything, so a reconnect pulls the whole retained history. Verified against a live bus rather than taken on report: since=<id> replayed from id 1247, no since at all replayed nothing, and since=<now-120> returned only the message from the last two minutes. The proxy signature confirms it, since: float | None. Worth calling out because of how it fails. A client that passes an id works perfectly in testing, where there is no history, and floods only in production where there is. Also tells clients to drop already-seen ids on receipt regardless, since a correct timestamp can still straddle a boundary and re-deliver.
hognek
pushed a commit
to hognek/tinyagentos
that referenced
this pull request
Jul 30, 2026
…fc#2161) * docs(agents): realtime A2A connection guide for deployed agents Agents deployed in taOS are expected to hold a live bus connection rather than chatting in their own CLI: a TUI conversation is invisible to memory, audit and observability, which is the reason to run an agent inside the OS at all. Documents the connection pattern proven in a live Hermes deployment (curl subprocess per channel piped into a parser, cheap cron pre-check so quiet ticks cost no model tokens) along with the parts that are easy to get wrong: http.client hangs on SSE keepalives, the proxy's 25s ': ping' comments must be ignored rather than parsed as events, and channel is required. Also records two field incidents as rules. A shared user account means a shared crontab, and an installer that dedups cron lines by script basename silently deleted a peer agent's backup watch, so dedup by full path and verify the whole table after writing. And a reply posted to the wrong channel left an agent blocked for 40 minutes, so answer where you were asked or redirect explicitly. Stream durability is stated as a requirement rather than advice: SSE connections die on network blips, so a supervisor plus a ~2 hourly backup poll are both mandatory, and health is measured by a watermark advancing rather than by the process being alive. * docs(agents): direct session trigger instead of a polling cron Hermes revised the pattern after the first writeup: the parser now fires the session directly on each new message rather than a cron noticing it later, so latency is a second rather than up to a minute. Records the two edges they hit. The curl subprocess does not inherit a useful PATH, so the trigger needs an absolute path or it fails silently. And one curl per channel means a message in two watched channels triggers twice, which is only safe because their scheduler rejects the duplicate; any harness without that dedup needs its own lock or a burst spawns racing sessions. The incoming file stays even though the trigger is immediate: it is what makes the wake durable and what the backup poll reconciles against. Polling is kept as a documented fallback for harnesses that cannot be triggered externally, and explicitly distinguished from the 2 hourly backup poll, which covers a different failure. * docs(agents): a monitor that dies with its subject is not a monitor Field correction from a second agent hit by the same shared-crontab wipe. The guide already said to measure health by a watermark advancing rather than by liveness, which was right but incomplete: their watermark check only ran as part of the backup job itself, so when the schedule was deleted the job never ran, the check never ran, and the silence read as calm. They found out only because a peer reported the incident on the bus. So the check has to run on its own schedule and confirm the job is still scheduled, not just that its last run looked healthy. Also records why nobody saw the deletions for hours: every check on that box printed the crontab through a grep matching only its own lines. A filter that hides peers hides the damage you did to them, and it had additionally concealed an unrelated scheduled job for days. Hence: print unfiltered, pin the expected table with owners so 'confirm the others survived' is actionable, and never quietly remove a block you do not recognise. * docs(agents): since is an epoch timestamp, not a message id Reported from the field as 'since does not suppress replay'. It does; the parameter is a float epoch seconds cursor and the clients were passing message ids into it. since=1444 is read as 1970, which means everything, so a reconnect pulls the whole retained history. Verified against a live bus rather than taken on report: since=<id> replayed from id 1247, no since at all replayed nothing, and since=<now-120> returned only the message from the last two minutes. The proxy signature confirms it, since: float | None. Worth calling out because of how it fails. A client that passes an id works perfectly in testing, where there is no history, and floods only in production where there is. Also tells clients to drop already-seen ids on receipt regardless, since a correct timestamp can still straddle a boundary and re-deliver.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jay asked for a proper connect-and-use guide now that Hermes has realtime bus messaging working, so deployed agents have something authoritative to follow.
Adds
docs/agent-join-kit/realtime-a2a.mdand wires it into the join-kit README.Why agents must use the bus, not their own CLI chat. A conversation in a TUI is not in memory, not in the audit log, and not visible to observability. The same conversation on the bus is persisted by taOSmd, indexed by the librarian and visible in taOStalk everywhere. An agent chatting in its own terminal has opted out of the OS.
The pattern is one taken from a working deployment, not invented here: one curl subprocess per channel piped into a parser, writing to an incoming file, with a short cron pre-check that exits silently when there is nothing, so quiet ticks cost no model tokens. Credit to Hermes for running it and reporting back the sharp edges, which are the parts worth documenting: Python's
http.clienthangs on SSE keepalives, andchannelis required (400 without it).Verified against the code rather than assumed:
/api/a2a/bus/streamrequiresa2a_receive, rejects a missing channel with 400, and injects a: pingcomment every 25s. That last one is why the guide tells parsers to ignore comment lines rather than treat them as malformed events.Two field incidents are written up as rules.
Shared account means shared crontab. A backup-watch installer deduped existing cron lines by script basename, matched a peer agent's identically-named script, and deleted its line. That agent's watch ran once and vanished, caught only because its watermark stopped advancing. The rule: dedup by full path, and print the whole table after any write to confirm you did not clobber a peer.
Wrong-channel replies. An agent answered in its own channel instead of the one it was asked in, and the asker sat blocked for 40 minutes.
Durability is stated as a requirement, not advice. SSE connections die on blips and restarts, and a dead watcher is indistinguishable from nobody talking to you, so a supervisor plus a ~2 hourly backup poll are both mandatory and health is measured by a watermark advancing rather than by liveness.