pixel agents with orca - #342
Closed
caste77an wants to merge 25 commits into
Closed
Conversation
existingAgents buffered agents into pendingAgents, which is only flushed by the layoutLoaded handler. But handleWebviewReady sends layoutLoaded BEFORE existingAgents, so the buffer flushed empty and any agent that already existed when the page connected (e.g. a session running before the office is opened) never rendered. Add agents directly when the layout is already loaded; keep buffering for the case existingAgents arrives first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live activity feed + status detail panel, resizable top-canvas/bottom-panel split, per-character project labels. Standalone surface for v1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10 TDD tasks: protocol (3 messages), server activity ring buffer + dispatch tee + requestActivity handler, client feed state + lifted selection + resizable split panel + project labels, standalone e2e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n select Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…standalone) Add the standalone browser dashboard UI: a resizable bottom AgentDetailPanel (overview list + selected-agent activity feed), per-character ProjectLabels, and an isBrowserRuntime-gated split layout. The VS Code surface is unchanged. Extract the world->screen overlay math into a shared overlayPositioning helper (computeOverlayTransform + characterToScreen) and refactor ToolOverlay to use it so ProjectLabels shares the exact same transform. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y + clear stale selection Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Standalone split layout with a resizable agent detail panel (live activity feed + status), per-character project labels, and the agentActivity / agentActivityHistory / requestActivity protocol additions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the standalone-office UI decisions locked this session in the Orca integration design doc, and commit the self-contained pixel-art preview. - docs/orca-integration.md: new §12 "UI / Office layout — design preview". 16:5 cinematic office band (desktop) / current stacked layout (mobile); single shared office grouped into per-workspace rooms; visual->milestone mapping (badge+hue tint=M2, decision-gate bubble & lead+teammates=M5, waiting/working/despawn=M4, live activity feed=M2/M4); open questions (mobile office size, badge density, done-agent treatment). - docs/orca-integration-preview.html: interactive canvas mockup of the integrated office (no external assets; click an agent to drive the AgentDetailPanel + live activity feed; 17-type legend). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Route each hook event to the provider named by the `:providerId` path segment of POST /api/hooks/:providerId, instead of ignoring it and always using the single injected provider. Fulfils the provider-registry architecture CLAUDE.md describes and unblocks the Orca `orca` provider. - HookEventHandler now takes Map<providerId, HookProvider> and resolves the provider per event via resolveProvider(); the resolved provider is threaded explicitly through every handler (no shared mutable state). Falls back to the sole provider when exactly one is registered, so Claude behaviour is identical; an unknown providerId is dropped once 2+ providers are registered. - AgentRuntime builds the provider Map (primary provider auto-registered) and exposes registerProvider() for adding the Orca bridge later; the handler shares the same Map instance. - Tests: 4 routing tests (id match / unknown-drop / single-provider fallback / unsupported-protocol-drop). All 228 server tests pass; full compile (types + lint + esbuild + vite) is green. Refs docs/orca-integration.md §11 (M1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Thread a per-agent `agentType` from the server to the office so each agent can
render a type badge (and, later, a hue tint) — the identity that lets Orca's 17
CLI types read apart. Claude agents are tagged 'claude'; the field is
provider-agnostic so the Orca provider (M4) will set 'codex'/'cursor'/etc.
Protocol
- New fully-typed `agentProviderInfo` ServerMessage (mirrors agentTeamInfo):
{ id, agentType, providerId? }. Added to core/asyncapi.yaml + regenerated
core/src/messages.ts (59 models).
Server
- AgentState.agentType + PersistedAgent.agentType (server + core schemas);
persisted and restored on both surfaces (standalone + VS Code).
- Broadcast agentProviderInfo right after agentCreated on the WS (httpServer)
and postMessage (PixelAgentsViewProvider) transports, and on VS Code webview
reconnect resync. Defaults to DEFAULT_AGENT_TYPE ('claude') when a provider
hasn't set one, so today's Claude agents are tagged without touching every
creation site.
Webview
- Character.agentType; OfficeState.setProviderInfo (mirrors setTeamInfo);
agentProviderInfo handler in useExtensionMessages.
- ToolOverlay renders a colored type badge (AGENT_TYPE_COLORS — the 17-type
palette from the office preview); AgentDetailPanel shows the type in its header.
Verification
- New WS integration test: connecting a client and adding agents broadcasts
agentProviderInfo with agentType 'claude' (default) and 'codex' (explicit).
- 229 server tests pass; full compile (asyncapi gen + types + lint + esbuild +
vite) green.
Known limitation: the standalone reconnect snapshot (existingAgents) doesn't
replay agentType yet — same gap team info already has; the badge shows for
agents created while the client is connected. Follow-up (with the team-info gap).
Refs docs/orca-integration.md §11 (M2), §12 (type badge).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the `orca` HookProvider: a pure translation layer that maps Orca's already- normalized 17-CLI event stream onto pixel-agents' AgentEvent union. No live Orca connection — Orca (Source A) will push to POST /api/hooks/orca in M4. - providers/hook/orca/orca.ts: normalizeHookEvent dispatches Orca kinds → AgentEvent (session.start→sessionStart with cwd parsed from the ptyId, tool.call/result→toolStart/toolEnd, agent.end/session.end→sessionEnd, permission.request→permissionRequest) and drives turn-end via the `state` field (idle→turnEnd(awaitingInput), done→sessionEnd, working→drop). Exposes parseOrcaWorkspacePath (ptyId → workspace path) and orcaAgentTypeFromEvent (validated agent_type, consumed at M4 adoption). Cross-CLI formatToolStatus. Push-based, so installer methods are no-ops and there is no team / file fallback / terminal launch. - providers/hook/orca/constants.ts: the 17 agent types, event vocabulary, state model, ptyId separators, and best-effort cross-CLI tool buckets. - Registered in providers/index.ts and wired via runtime.registerProvider in cli.ts (the M1 provider map), so POST /api/hooks/orca now routes to it. Tests: __tests__/orca.test.ts — 110 tests (metadata + registry, ptyId parse unix/windows, all 17 agent types recognized, a 17×5 type×kind matrix, the state-driven lifecycle, and cross-CLI formatToolStatus). 339 server tests pass; full compile (types + lint + esbuild + vite) green. No protocol change. Refs docs/orca-integration.md §11 (M3), §10 (push contract), §6 (mapping). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopt Orca agents pushed to POST /api/hooks/orca as first-class office
characters, tagged with their CLI type. This is the pixel-agents half of M4;
the Orca-side forwarder (a few lines in the Orca host, per §10) is the other
half and lives outside this repo.
- Thread the CLI identity through adoption: PendingExternalSession gains
providerId + agentType; the SessionStart handler captures them (providerId
from the route, agent_type from the raw event); onExternalSessionDetected
forwards them; adoptExternalSessionFromHook sets AgentState.agentType. So an
adopted Orca agent broadcasts agentProviderInfo (M2) with its real type
('codex', 'cursor', ...) and the office renders the type badge.
- Always-adopt push-based providers: a provider with no file fallback
(getSessionDirs === undefined, i.e. Orca) bypasses the isTrackedProjectDir /
Watch-All gate, since every pushed session is intentional. File-scanning
providers (Claude) are unchanged.
- Orca agents flow through the existing hooks-only adoption path (no transcript
→ jsonlFile '', hooksOnly true, projectDir/folderName from the ptyId's
workspace path).
Tests: __tests__/orcaAdoption.test.ts — 5 runtime-level integration tests
driving the real path (handleHookEvent → orcaProvider → pending/confirm →
adopt): session.start pends, a confirming event adopts a hooks-only agent
tagged by CLI type, adoption works in untracked workspaces, and active/idle
transitions land. Two hookEventHandler assertions updated for the new
onExternalSessionDetected arg. 344 server tests + full compile green.
Refs docs/orca-integration.md §11 (M4), §10 (push contract).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the status line to reflect M1–M4 landing, and document the one remaining piece — the Orca-side forwarder — as a concrete wire contract + Node reference implementation, with the adoption rules the implemented provider/handler require (session.start-first, state=idle for turn end, agent.end to despawn). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gate Orca adoption behind a per-namespace `orcaEnabled` setting (default off / opt-in), surfaced as a "Show Orca Agents" toggle in the standalone Settings modal. Makes the integration discoverable in the UI and gives a kill switch. - Setting: orcaEnabled added to AdapterSettings (config persistence — interface, key list, default=false, coercion). - Protocol: new SetOrcaEnabled ClientMessage + optional orcaEnabled on SettingsLoaded (asyncapi + regenerated messages.ts, 60 models). - Runtime: AgentRuntime.orcaEnabled ref; onExternalSessionDetected adopts a push-based (Orca) session only when orcaEnabled is on (file-scanning providers like Claude unchanged). Wired through clientMessageHandler (setOrcaEnabled case + settingsLoaded read/send/ref-sync) and cli.ts startup sync. - Webview: "Show Orca Agents" checkbox in SettingsModal, gated to standalone via isBrowserRuntime (Orca isn't registered in the VS Code adapter); useExtensionMessages state + settingsLoaded guard + return; App wiring sends setOrcaEnabled. Added data-testid="agent-type-badge" to the M2 type badge for future e2e assertions. Tests: orcaAdoption.test.ts gains a decision-gate case (permission.request → agentToolPermission bubble, i.e. §6 decision_gates → permission "…" bubble via push) and an off-gate case (orcaEnabled=false → no adoption); beforeEach enables the setting. 346 server tests + full compile (types + lint + esbuild + vite) green. Deferred M5 items (need more than pixel-agents-side work): coordinator/tasks → Lead+Teammates (needs an Orca team-push contract), group-by-workspace rooms (larger office-engine change), mock-orca E2E (Playwright browsers unavailable in this environment — author + run in CI). Refs docs/orca-integration.md §11 (M5), §6 (mapping). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M1 multi-provider hook routing · M2 agentType identity + type badge · M3 orca provider (pure translation) · M4 Orca push adoption · M5 orcaEnabled setting. Design + Orca forwarder reference in docs/orca-integration.md. Remaining to go live: the Orca-side forwarder (§13).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Merges two feature branches into
main(25 commits, 47 files):1. Agent activity dashboard (standalone browser surface only)
agentActivity,agentActivityHistory,requestActivity(core/asyncapi.yaml, regeneratedcore/src/messages.ts).AgentStateStore, abuildActivityEntryfeed mapper, activity recording tapped into hook-event dispatch, and arequestActivityhandler that replies with history.AgentDetailPanel(overview list + selected-agent feed) plus per-characterProjectLabels. Gated to the standalone runtime (isBrowserRuntime) — VS Code surface is unchanged. Overlay math was extracted into a sharedoverlayPositioninghelper soProjectLabelsandToolOverlayshare the same transform.layoutLoadedfired beforeexistingAgentsflushed the pending buffer.docs/superpowers/specs/2026-06-30-agent-activity-dashboard-design.md,.../plans/2026-06-30-agent-activity-dashboard.md.2. Orca multi-provider integration, milestones M1–M5 (see
docs/orca-integration.md§11):providerIdsegment ofPOST /api/hooks/:providerIdthrough aMap<providerId, HookProvider>, instead of always hitting one injected provider (falls back to the sole provider, so Claude behavior is unchanged).agentTypethreaded end-to-end (newagentProviderInfomessage, persisted + restored state, type badge onToolOverlay/AgentDetailPanel) so Orca's 17 CLI types can be told apart.orcaHookProvider(server/src/providers/hook/orca/): pure translation from Orca's normalized event stream to the internalAgentEventunion; push-based, no live connection yet.orcaEnabledsetting (default off), with a "Show Orca Agents" toggle in the standalone Settings modal and a kill switch at the runtime level.Type of change
Related issues
Not linked to a GitHub issue. Scope is tracked in
docs/superpowers/plans/2026-06-30-agent-activity-dashboard.mdanddocs/orca-integration.md(§11 milestone log).Screenshots / GIFs
Interactive design mockup also included at
docs/orca-integration-preview.html(§12 of the Orca doc).Test plan
E2E coverage
under
pixel-agents/e2e/tests/. Rannpm run e2e:inventoryandcommitted the regenerated
pixel-agents/e2e/README.md.behavior change, no e2e test is needed. Explain below.
What e2e tests cover this change?
e2e/tests/standalone/hooks.spec.ts:102— "shows agent activity in the detail panel" (new;@area:standalonewent 1 → 2 tests, 50 → 51 total). The Orca M1–M5 work is covered by server-side tests only (server/__tests__/orca.test.ts,orcaAdoption.test.ts, plus updatedhookEventHandler.test.ts/server.test.ts); a browser-level mock-Orca E2E is explicitly deferred — blocked on Playwright browsers not being available in the dev sandbox (to be authored/run in CI).