Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 93 additions & 2 deletions PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ Dispatch a user message to a session.
| customInstructions | string? | Account-level instructions snapshotted onto this task. Updated daemons append this text to the Pi system prompt. |
| disableSkills | boolean? | `true` disables Claude skill discovery and explicit skill file loading for the task. |
| planCapability | PlanCapability? | Task-scoped bearer capability for Plan Mode tools. The daemon passes it to the Pi process as environment variables and never places it in the prompt. |
| browserGrant | BrowserGrantContext? | Task-scoped browser grant context. Cloud creates the grant before dispatch; the daemon opens the local browser lazily on the first browser tool call. |

`ContextRef`:

Expand All @@ -80,6 +81,18 @@ Dispatch a user message to a session.
| expiresAt | string | ISO timestamp. |
| snapshot | object? | Capability metadata snapshot used by cloud-side authorization. |

`BrowserGrantContext`:

| Field | Type | Notes |
|---|---|---|
| grantId | string | Browser grant record identifier. |
| projectId | uuid | Project that owns the grant. |
| sessionId | uuid | Session that owns the grant. |
| taskId | uuid | Task that owns the grant. |
| channelId | string | Browser channel that receives lifecycle events. |
| machineId | uuid | Daemon machine expected to consume the grant. |
| expiresAt | iso8601 string | Grant expiry. |

`TaskDeadlines`:

| Field | Type | Notes |
Expand Down Expand Up @@ -572,6 +585,17 @@ Daemon-to-browser lifecycle message for manual and automatic compaction.
| previewWebSocketProtocols | boolean? | Daemon forwards requested WebSocket subprotocols. |
| localServerDetection | boolean? | Daemon reports verified loopback web servers started by task tools. |
| skills | boolean? | Daemon accepts `listSkills` and can pass explicit Claude skill files into Pi. |
| browserRuntimeInstalled | boolean? | `gsd-browser` binary is present on the daemon machine. |
| browserRuntimeVersion | string? | Installed `gsd-browser` version. |
| browserRuntimeMinVersion | string? | Minimum runtime version required by the daemon. |
| browserRuntimeMinVersionOk | boolean? | Installed runtime satisfies `browserRuntimeMinVersion`. |
| browserRuntimePath | string? | Runtime binary path used by daemon probing and execution. |
| browserRuntimeErrorCode | string? | Stable unavailable reason, e.g. `browser_not_installed`, `version_too_old`, `manifest_invalid`, or `chrome_missing`. |
| browserRuntimeErrorMessage | string? | Human-readable unavailable reason. |
| browserCloudMethodsVersion | int? | Cloud method manifest version reported by `gsd-browser cloud-methods --json`. |
| browserChromeAvailable | boolean? | Chrome/Chromium is available to the runtime. |

Old daemons omit browser runtime fields and browser lifecycle messages. Cloud treats omitted runtime fields as unknown or unavailable for ambient browser use.

### `welcome` (relay → daemon, response to hello)

Expand Down Expand Up @@ -830,6 +854,15 @@ Browser support is advertised in `hello.capabilities`.
| browserIdentities | bool | Daemon can isolate browser state by identity scope and key. |
| browserSensitiveActionApproval | bool | Daemon can pause browser tool execution for approval. |
| browserMaxFrameBytes | int | Maximum encoded browser frame size the daemon can send. |
| browserRuntimeInstalled | bool | `gsd-browser` binary is installed. |
| browserRuntimeVersion | string | Installed `gsd-browser` version. |
| browserRuntimeMinVersion | string | Minimum compatible runtime version. |
| browserRuntimeMinVersionOk | bool | Installed runtime is compatible. |
| browserRuntimePath | string | Runtime binary path. |
| browserRuntimeErrorCode | string | Stable runtime unavailable code. |
| browserRuntimeErrorMessage | string | Runtime unavailable detail. |
| browserCloudMethodsVersion | int | Cloud method manifest version. |
| browserChromeAvailable | bool | Chrome/Chromium is available. |

### Lifecycle

Expand Down Expand Up @@ -1135,7 +1168,7 @@ to the approved loopback target.

### Tool Calls

`browserToolCall` and `browserToolResult` represent agent browser tool execution. The daemon validates active grant state before executing each call.
`browserToolCall`, `browserToolCallStarted`, `browserToolCallUpdated`, `browserArtifactCreated`, and `browserToolResult` represent agent browser tool execution. The daemon validates active grant state before executing each call and sends only redacted safe result data to Cloud/model consumers.

#### `browserToolCall`

Expand All @@ -1149,18 +1182,76 @@ to the approved loopback target.
| method | string |
| paramsJson | json? |

#### `browserToolCallStarted`

| Field | Type |
|---|---|
| type | "browserToolCallStarted" |
| browserId | string? |
| grantId | string? |
| sessionId | uuid |
| channelId | string |
| taskId | uuid? |
| toolUseId | string? |
| method | string |
| category | string? |
| summary | string |
| intent | string? |
| metadata | json? |
| at | iso8601 string |

#### `browserToolCallUpdated`

| Field | Type |
|---|---|
| type | "browserToolCallUpdated" |
| browserId | string? |
| grantId | string? |
| sessionId | uuid |
| channelId | string |
| taskId | uuid? |
| toolUseId | string? |
| status | string |
| summary | string? |
| metadata | json? |
| at | iso8601 string |

#### `browserArtifactCreated`

| Field | Type |
|---|---|
| type | "browserArtifactCreated" |
| browserId | string |
| grantId | string? |
| sessionId | uuid |
| channelId | string |
| taskId | uuid? |
| artifactId | string |
| kind | string |
| title | string? |
| url | string? |
| metadata | json? |
| createdAt | iso8601 string |

#### `browserToolResult`

| Field | Type |
|---|---|
| type | "browserToolResult" |
| browserId | string |
| grantId | string |
| sessionId | uuid? |
| channelId | string? |
| taskId | uuid |
| toolUseId | string |
| ok | bool |
| resultJson | json? |
| resultJson | json? — Redacted safe result data only. |
| error | string? |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
| errorCode | string? |
| recoveryHint | string? |
| sensitivity | string? |
| redactionStatus | string? |
| localArtifactPointer | string? |

### Sensitive Actions

Expand Down
6 changes: 6 additions & 0 deletions envelope.go
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,12 @@ func payloadForType(msgType string) (any, error) {
return &BrowserToolCall{}, nil
case MsgTypeBrowserToolResult:
return &BrowserToolResult{}, nil
case MsgTypeBrowserToolCallStarted:
return &BrowserToolCallStarted{}, nil
case MsgTypeBrowserToolCallUpdated:
return &BrowserToolCallUpdated{}, nil
case MsgTypeBrowserArtifactCreated:
return &BrowserArtifactCreated{}, nil
case MsgTypeBrowserControlClaim:
return &BrowserControlClaim{}, nil
case MsgTypeBrowserControlRelease:
Expand Down
167 changes: 121 additions & 46 deletions messages.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ const (
MsgTypeBrowserAction MessageType = "browserAction"
MsgTypeBrowserToolCall MessageType = "browserToolCall"
MsgTypeBrowserToolResult MessageType = "browserToolResult"
MsgTypeBrowserToolCallStarted MessageType = "browserToolCallStarted"
MsgTypeBrowserToolCallUpdated MessageType = "browserToolCallUpdated"
MsgTypeBrowserArtifactCreated MessageType = "browserArtifactCreated"
MsgTypeBrowserControlClaim MessageType = "browserControlClaim"
MsgTypeBrowserControlRelease MessageType = "browserControlRelease"
MsgTypeBrowserUserInput MessageType = "browserUserInput"
Expand Down Expand Up @@ -145,6 +148,16 @@ type PlanCapability struct {
Snapshot json.RawMessage `json:"snapshot,omitempty"`
}

type BrowserGrantContext struct {
GrantID string `json:"grantId"`
ProjectID string `json:"projectId"`
SessionID string `json:"sessionId"`
TaskID string `json:"taskId"`
ChannelID string `json:"channelId"`
MachineID string `json:"machineId"`
ExpiresAt string `json:"expiresAt"`
}

type PlanningEvent struct {
Type MessageType `json:"type"`
EventID string `json:"eventId"`
Expand Down Expand Up @@ -254,30 +267,31 @@ const (

// Task is sent from the browser to the daemon to dispatch a user message.
type Task struct {
Type string `json:"type"`
TaskID string `json:"taskId"`
SessionID string `json:"sessionId"`
ChannelID string `json:"channelId"`
AttemptID string `json:"attemptId,omitempty"`
AttemptNumber int `json:"attemptNumber,omitempty"`
LeaseExpiresAt string `json:"leaseExpiresAt,omitempty"`
DeadlineProfile TaskDeadlines `json:"deadlineProfile,omitempty"`
TurnKind TurnKind `json:"turnKind,omitempty"`
Prompt string `json:"prompt"`
Engine string `json:"engine,omitempty"` // "pi"; empty defaults to pi
Provider string `json:"provider,omitempty"` // Pi provider; empty defaults to claude-cli
Model string `json:"model"`
Effort string `json:"effort"`
PermissionMode string `json:"permissionMode"`
CWD string `json:"cwd"`
ClaudeSessionID string `json:"claudeSessionId,omitempty"` // passed to --resume
RequestID string `json:"requestId,omitempty"`
Traceparent string `json:"traceparent,omitempty"` // W3C trace context
ImageURLs []string `json:"imageUrls,omitempty"` // user-attached image URLs
ContextRefs []ContextRef `json:"contextRefs,omitempty"`
CustomInstructions string `json:"customInstructions,omitempty"`
DisableSkills bool `json:"disableSkills,omitempty"`
PlanCapability *PlanCapability `json:"planCapability,omitempty"`
Type string `json:"type"`
TaskID string `json:"taskId"`
SessionID string `json:"sessionId"`
ChannelID string `json:"channelId"`
AttemptID string `json:"attemptId,omitempty"`
AttemptNumber int `json:"attemptNumber,omitempty"`
LeaseExpiresAt string `json:"leaseExpiresAt,omitempty"`
DeadlineProfile TaskDeadlines `json:"deadlineProfile,omitempty"`
TurnKind TurnKind `json:"turnKind,omitempty"`
Prompt string `json:"prompt"`
Engine string `json:"engine,omitempty"` // "pi"; empty defaults to pi
Provider string `json:"provider,omitempty"` // Pi provider; empty defaults to claude-cli
Model string `json:"model"`
Effort string `json:"effort"`
PermissionMode string `json:"permissionMode"`
CWD string `json:"cwd"`
ClaudeSessionID string `json:"claudeSessionId,omitempty"` // passed to --resume
RequestID string `json:"requestId,omitempty"`
Traceparent string `json:"traceparent,omitempty"` // W3C trace context
ImageURLs []string `json:"imageUrls,omitempty"` // user-attached image URLs
ContextRefs []ContextRef `json:"contextRefs,omitempty"`
CustomInstructions string `json:"customInstructions,omitempty"`
DisableSkills bool `json:"disableSkills,omitempty"`
PlanCapability *PlanCapability `json:"planCapability,omitempty"`
BrowserGrant *BrowserGrantContext `json:"browserGrant,omitempty"`
}

type TaskLifecycle struct {
Expand Down Expand Up @@ -906,14 +920,66 @@ type BrowserToolCall struct {
}

type BrowserToolResult struct {
Type MessageType `json:"type"`
BrowserID string `json:"browserId"`
GrantID string `json:"grantId"`
SessionID string `json:"sessionId,omitempty"`
ChannelID string `json:"channelId,omitempty"`
TaskID string `json:"taskId"`
ToolUseID string `json:"toolUseId"`
OK bool `json:"ok"`
ResultJSON json.RawMessage `json:"resultJson,omitempty"`
Error string `json:"error,omitempty"`
ErrorCode string `json:"errorCode,omitempty"`
RecoveryHint string `json:"recoveryHint,omitempty"`
Sensitivity string `json:"sensitivity,omitempty"`
RedactionStatus string `json:"redactionStatus,omitempty"`
LocalArtifactPointer string `json:"localArtifactPointer,omitempty"`
}

type BrowserToolCallStarted struct {
Type MessageType `json:"type"`
BrowserID string `json:"browserId,omitempty"`
GrantID string `json:"grantId,omitempty"`
SessionID string `json:"sessionId"`
ChannelID string `json:"channelId"`
TaskID string `json:"taskId,omitempty"`
ToolUseID string `json:"toolUseId,omitempty"`
Method string `json:"method"`
Category string `json:"category,omitempty"`
Summary string `json:"summary"`
Intent string `json:"intent,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"`
At string `json:"at"`
}

type BrowserToolCallUpdated struct {
Type MessageType `json:"type"`
BrowserID string `json:"browserId,omitempty"`
GrantID string `json:"grantId,omitempty"`
SessionID string `json:"sessionId"`
ChannelID string `json:"channelId"`
TaskID string `json:"taskId,omitempty"`
ToolUseID string `json:"toolUseId,omitempty"`
Status string `json:"status"`
Summary string `json:"summary,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"`
At string `json:"at"`
}

type BrowserArtifactCreated struct {
Type MessageType `json:"type"`
BrowserID string `json:"browserId"`
GrantID string `json:"grantId"`
TaskID string `json:"taskId"`
ToolUseID string `json:"toolUseId"`
OK bool `json:"ok"`
ResultJSON json.RawMessage `json:"resultJson,omitempty"`
Error string `json:"error,omitempty"`
GrantID string `json:"grantId,omitempty"`
SessionID string `json:"sessionId"`
ChannelID string `json:"channelId"`
TaskID string `json:"taskId,omitempty"`
ArtifactID string `json:"artifactId"`
Kind string `json:"kind"`
Title string `json:"title,omitempty"`
URL string `json:"url,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"`
CreatedAt string `json:"createdAt"`
}

type BrowserControlClaim struct {
Expand Down Expand Up @@ -1050,22 +1116,31 @@ type BrowserSensitiveActionResponse struct {

// HelloCapabilities describes optional daemon protocol support.
type HelloCapabilities struct {
Stop bool `json:"stop,omitempty"`
Terminal bool `json:"terminal,omitempty"`
AgentTerminalJobs bool `json:"agentTerminalJobs,omitempty"`
ContextRefs bool `json:"contextRefs,omitempty"`
PreviewTunnel bool `json:"previewTunnel,omitempty"`
PreviewMaxFrameBytes int `json:"previewMaxFrameBytes,omitempty"`
PreviewChunkBytes int `json:"previewChunkBytes,omitempty"`
PreviewWebSocketProtocols bool `json:"previewWebSocketProtocols,omitempty"`
LocalServerDetection bool `json:"localServerDetection,omitempty"`
Skills bool `json:"skills,omitempty"`
BrowserSessions bool `json:"browserSessions,omitempty"`
BrowserFrameStream bool `json:"browserFrameStream,omitempty"`
BrowserUserControl bool `json:"browserUserControl,omitempty"`
BrowserIdentities bool `json:"browserIdentities,omitempty"`
BrowserSensitiveActionApproval bool `json:"browserSensitiveActionApproval,omitempty"`
BrowserMaxFrameBytes int `json:"browserMaxFrameBytes,omitempty"`
Stop bool `json:"stop,omitempty"`
Terminal bool `json:"terminal,omitempty"`
AgentTerminalJobs bool `json:"agentTerminalJobs,omitempty"`
ContextRefs bool `json:"contextRefs,omitempty"`
PreviewTunnel bool `json:"previewTunnel,omitempty"`
PreviewMaxFrameBytes int `json:"previewMaxFrameBytes,omitempty"`
PreviewChunkBytes int `json:"previewChunkBytes,omitempty"`
PreviewWebSocketProtocols bool `json:"previewWebSocketProtocols,omitempty"`
LocalServerDetection bool `json:"localServerDetection,omitempty"`
Skills bool `json:"skills,omitempty"`
BrowserSessions bool `json:"browserSessions,omitempty"`
BrowserFrameStream bool `json:"browserFrameStream,omitempty"`
BrowserUserControl bool `json:"browserUserControl,omitempty"`
BrowserIdentities bool `json:"browserIdentities,omitempty"`
BrowserSensitiveActionApproval bool `json:"browserSensitiveActionApproval,omitempty"`
BrowserMaxFrameBytes int `json:"browserMaxFrameBytes,omitempty"`
BrowserRuntimeInstalled bool `json:"browserRuntimeInstalled,omitempty"`
BrowserRuntimeVersion string `json:"browserRuntimeVersion,omitempty"`
BrowserRuntimeMinVersion string `json:"browserRuntimeMinVersion,omitempty"`
BrowserRuntimeMinVersionOK bool `json:"browserRuntimeMinVersionOk,omitempty"`
BrowserRuntimePath string `json:"browserRuntimePath,omitempty"`
BrowserRuntimeErrorCode string `json:"browserRuntimeErrorCode,omitempty"`
BrowserRuntimeErrorMessage string `json:"browserRuntimeErrorMessage,omitempty"`
BrowserCloudMethodsVersion int `json:"browserCloudMethodsVersion,omitempty"`
BrowserChromeAvailable bool `json:"browserChromeAvailable,omitempty"`
}

// Hello is the first frame sent by the daemon after connecting.
Expand Down
Loading