Skip to content

Extend browser workbench protocol #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
glittercowboy merged 3 commits into from
May 1, 2026
Merged

Extend browser workbench protocol #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a8ee03b
feat: extend browser workbench protocol
glittercowboy May 1, 2026
ab5e273
test: cover browser workbench envelope compatibility
glittercowboy May 1, 2026
8b3e1d9
test: cover browser approval and ack compatibility
glittercowboy May 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 164 additions & 1 deletion PROTOCOL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -881,6 +881,10 @@ Browser support is advertised in `hello.capabilities`.
| channelId | string |
| machineId | string |
| identityId | uuid? |
| identityScope | string? |
| identityKey | string? |
| identityProjectId | uuid? |
| identitySessionId | uuid? |
| mode | string |
| expiresAt | iso8601 string |

Expand Down Expand Up @@ -952,7 +956,18 @@ Browser support is advertised in `hello.capabilities`.
| height | int |
| viewportWidth | int? |
| viewportHeight | int? |
| viewportCssWidth | int? |
| viewportCssHeight | int? |
| capturePixelWidth | int? |
| capturePixelHeight | int? |
| devicePixelRatio | float64? |
| captureScaleX | float64? |
| captureScaleY | float64? |
| encodedBytes | int? |
| quality | int? |
| capturePixelRatio | float64? |
| latencyMs | int64? |
| latestAcceptedFrameSeq | int64? |
| capturedAt | iso8601 string |
| droppedPriorCount | int? |

Expand Down Expand Up @@ -1021,7 +1036,7 @@ Sent daemon to browser clients for visible interactive browser references.

### Control

`browserControlClaim`, `browserControlRelease`, and `browserUserInput` coordinate Lex control. Tool calls execute while control owner is `agent`.
`browserControlClaim`, `browserControlRelease`, `browserControlClaimRequest`, `browserControlState`, `browserClaimAndInput`, and `browserUserInput` coordinate Lex control. Tool calls execute while control owner is `agent`.

#### `browserControlClaim`

Expand All @@ -1035,6 +1050,47 @@ Sent daemon to browser clients for visible interactive browser references.
| reason | string? |
| controlVersion | int64? |

#### `browserControlClaimRequest`

| Field | Type |
|---|---|
| type | "browserControlClaimRequest" |
| claimId | string |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| owner | string |
| requestedBy | string |
| requestedAt | iso8601 string |
| reason | string? |
| metadata | json? |

#### `browserControlState`

| Field | Type |
|---|---|
| type | "browserControlState" |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| owner | string |
| controlVersion | int64 |
| accepted | bool |
| reason | string? |
| at | iso8601 string |
| metadata | json? |

#### `browserClaimAndInput`

| Field | Type |
|---|---|
| type | "browserClaimAndInput" |
| claimId | string |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| input | BrowserUserInput |

#### `browserControlRelease`

| Field | Type |
Expand All @@ -1058,12 +1114,26 @@ Sent daemon to browser clients for visible interactive browser references.
| channelId | string |
| owner | string |
| kind | string |
| phase | string? |
| button | string? |
| buttons | int? |
| clickCount | int? |
| pointerType | string? |
| modifiers | string[]? |
| x | float64? |
| y | float64? |
| text | string? |
| key | string? |
| code | string? |
| location | int? |
| repeat | bool? |
| commitMode | string? |
| mimeTypes | string[]? |
| deltaX | float64? |
| deltaY | float64? |
| url | string? |
| action | string? |
| capturedAt | iso8601 string? |
| frameSeq | int64? |
| controlVersion | int64? |
| coordinateSpace | string? |
Expand All @@ -1086,11 +1156,32 @@ Sent daemon to browser clients for visible interactive browser references.
| sessionId | uuid |
| channelId | string |
| inputId | string? |
| kind | string? |
| phase | string? |
| accepted | bool |
| frameSeq | int64? |
| acceptedFrameSeq | int64? |
| reasonCode | string? |
| safeRetry | bool? |
| message | string? |
| reason | string? |
| controlVersion | int64? |
| ackedAt | iso8601 string |

#### `browserViewportSet`

| Field | Type |
|---|---|
| type | "browserViewportSet" |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| viewportCssWidth | int |
| viewportCssHeight | int |
| deviceScaleFactor | float64 |
| requestedBy | string |
| requestedAt | iso8601 string |

#### `browserTransportStatus`

| Field | Type |
Expand Down Expand Up @@ -1118,6 +1209,57 @@ Browser frame producers may drop, coalesce, downscale, or reference frames when
encoded frames exceed transport limits. Control and approval messages have
priority over frame delivery.

### Browser Identity And Evidence

Browser identity messages describe local-first profile availability, binding, use, and revocation. Browser evidence messages describe redacted, durable events and local-only debug bundles.

#### `browserEvidenceCreated`

| Field | Type |
|---|---|
| type | "browserEvidenceCreated" |
| evidenceId | string |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| taskId | uuid? |
| toolUseId | string? |
| actor | string |
| status | string |
| eventType | string |
| summary | string |
| origin | string? |
| redactionStatus | string |
| metadata | json? |
| createdAt | iso8601 string |

#### `browserDebugBundleCreated`

| Field | Type |
|---|---|
| type | "browserDebugBundleCreated" |
| bundleId | string |
| browserId | string |
| sessionId | uuid |
| channelId | string |
| redactionStatus | string |
| residency | string |
| localArtifactPointer | string? |
| metadata | json? |
| createdAt | iso8601 string |

#### Identity messages

| Message | Required fields |
|---|---|
| browserIdentityAvailable | identityId, machineId, identityScope, displayName, status |
| browserIdentitySaved | identityId, machineId, identityScope, identityKey, displayName, savedAt |
| browserIdentityRevoked | identityId, machineId, identityScope, identityKey, revokedAt, acknowledged |
| browserIdentityUsed | identityId, browserId, sessionId, channelId, origin, usedAt |
| browserIdentityBind | bindingId, identityId, machineId, projectId, originPattern, boundBy, boundAt |
| browserIdentityUnbind | bindingId, identityId, machineId, projectId, revokedBy, revokedAt |
| browserIdentityUseApproved | grantId, identityId, browserId, sessionId, channelId, origin, expiresAt |

### Preview Bridge Access

Preview-origin bridge access is short-lived, grant-bound, and separate from
Expand Down Expand Up @@ -1267,10 +1409,18 @@ to the approved loopback target.
| sessionId | uuid |
| channelId | string |
| taskId | uuid |
| approvalId | string |
| nonce | string |
| actorUserId | string |
| grantId | string |
| toolUseId | string |
| method | string |
| category | string |
| summary | string |
| origin | string? |
| parameterHash | string |
| expectedExternalEffect | string |
| sensitivity | string |
| expiresAt | iso8601 string |

#### `browserSensitiveActionResponse`
Expand All @@ -1282,5 +1432,18 @@ to the approved loopback target.
| requestId | string |
| sessionId | uuid |
| channelId | string |
| approvalId | string |
| nonce | string |
| actorUserId | string |
| grantId | string |
| toolUseId | string |
| method | string |
| category | string |
| origin | string? |
| parameterHash | string |
| expectedExternalEffect | string |
| sensitivity | string |
| expiresAt | iso8601 string |
| approved | bool |
| deniedReason | string? |
| reason | string? |
26 changes: 26 additions & 0 deletions envelope.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,12 @@ func payloadForType(msgType string) (any, error) {
return &BrowserControlClaim{}, nil
case MsgTypeBrowserControlRelease:
return &BrowserControlRelease{}, nil
case MsgTypeBrowserControlState:
return &BrowserControlState{}, nil
case MsgTypeBrowserControlClaimRequest:
return &BrowserControlClaimRequest{}, nil
case MsgTypeBrowserClaimAndInput:
return &BrowserClaimAndInput{}, nil
case MsgTypeBrowserUserInput:
return &BrowserUserInput{}, nil
case MsgTypeBrowserUserInputAck:
Expand All @@ -237,6 +243,26 @@ func payloadForType(msgType string) (any, error) {
return &BrowserSensitiveActionRequest{}, nil
case MsgTypeBrowserSensitiveActionResponse:
return &BrowserSensitiveActionResponse{}, nil
case MsgTypeBrowserEvidenceCreated:
return &BrowserEvidenceCreated{}, nil
case MsgTypeBrowserDebugBundleCreated:
return &BrowserDebugBundleCreated{}, nil
case MsgTypeBrowserIdentityAvailable:
return &BrowserIdentityAvailable{}, nil
case MsgTypeBrowserIdentitySaved:
return &BrowserIdentitySaved{}, nil
case MsgTypeBrowserIdentityRevoked:
return &BrowserIdentityRevoked{}, nil
case MsgTypeBrowserIdentityUsed:
return &BrowserIdentityUsed{}, nil
case MsgTypeBrowserIdentityBind:
return &BrowserIdentityBind{}, nil
case MsgTypeBrowserIdentityUnbind:
return &BrowserIdentityUnbind{}, nil
case MsgTypeBrowserIdentityUseApproved:
return &BrowserIdentityUseApproved{}, nil
case MsgTypeBrowserViewportSet:
return &BrowserViewportSet{}, nil
case MsgTypePlanningEvent:
return &PlanningEvent{}, nil
case MsgTypePlanningEventAck:
Expand Down
Loading