Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/brave-cassettes-record.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@opencode-ai/http-recorder": minor
---

Publish the initial beta of the Effect HTTP and WebSocket record/replay library.
11 changes: 11 additions & 0 deletions .changeset/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"$schema": "https://unpkg.com/@changesets/config@3.1.1/schema.json",
"changelog": "@changesets/cli/changelog",
"commit": false,
"fixed": [],
"linked": [],
"access": "public",
"baseBranch": "dev",
"updateInternalDependencies": "patch",
"ignore": []
}
35 changes: 35 additions & 0 deletions .github/workflows/http-recorder-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: http-recorder release

on:
workflow_dispatch:

concurrency: http-recorder-release

permissions:
contents: read
id-token: write

jobs:
release:
if: github.repository == 'anomalyco/opencode' && github.ref == 'refs/heads/dev'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1

- uses: ./.github/actions/setup-bun

- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: "24"
registry-url: "https://registry.npmjs.org"

- name: Verify package
working-directory: packages/http-recorder
run: |
bun run test
bun typecheck

- name: Publish beta
run: bun run release:http-recorder
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
202 changes: 187 additions & 15 deletions bun.lock

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,12 @@
"dev:console": "ulimit -n 10240 2>/dev/null; bun run --cwd packages/console/app dev",
"dev:stats": "bun sst shell --stage=production -- bun run --cwd packages/stats/app dev",
"dev:storybook": "bun --cwd packages/storybook storybook",
"changeset": "changeset",
"lint": "oxlint",
"release:http-recorder": "bun ./packages/http-recorder/script/publish.ts",
"typecheck": "bun turbo typecheck",
"upgrade-opentui": "bun run script/upgrade-opentui.ts",
"version:http-recorder": "bun ./packages/http-recorder/script/version.ts",
"postinstall": "bun run --cwd packages/core fix-node-pty",
"prepare": "husky",
"random": "echo 'Random script'",
Expand Down Expand Up @@ -94,6 +97,7 @@
},
"devDependencies": {
"@actions/artifact": "5.0.1",
"@changesets/cli": "2.31.0",
"@tsconfig/bun": "catalog:",
"@types/mime-types": "3.0.1",
"@typescript/native-preview": "catalog:",
Expand Down
43 changes: 43 additions & 0 deletions packages/http-recorder/CONTEXT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# HTTP Recorder

The HTTP recorder is a public testing library for recording real Effect transport traffic into deterministic cassettes and replaying it without contacting the upstream service.

## Language

**Library Version**:
The independently managed semantic version of `@opencode-ai/http-recorder` published to npm.
_Avoid_: OpenCode version

**OpenCode Release Version**:
The version applied to OpenCode applications and packages by the repository-wide release process. It does not determine the **Library Version**.

**Effect Compatibility Version**:
The exact Effect 4 beta against which the package's unstable HTTP and socket integrations are built and verified.

**Provider Conversation**:
A finite WebSocket connection in which either peer may send first, the client sends one or more JSON commands, server frames follow in causal order, and the application closes after a terminal event.

**Connection Identity**:
The explicit cassette name supplied to the WebSocket recorder. It identifies the recorded conversation during replay.

**Completed Conversation**:
A WebSocket socket run that opened and finished successfully after recording a valid finite transcript.

**Client Frame Match**:
The replay check that requires an outgoing application frame to equal the next recorded client frame after redaction. Text JSON ignores object-key order; other text and binary frames match exactly.

## Relationships

- The **Library Version** begins with the public beta at `0.1.0` and advances independently through Changesets.
- Repository-wide OpenCode release synchronization must not rewrite the **Library Version**.
- The initial **Effect Compatibility Version** is `4.0.0-beta.83`; compatibility with later Effect betas is not implied.
- Changing the **Effect Compatibility Version** requires a new **Library Version** and clean-consumer package verification.
- A **Provider Conversation** is the canonical WebSocket scenario for evaluating the public beta contract; arbitrary socket emulation is not implied.
- Application code owns WebSocket construction, including URL, protocols, timeout, authentication, and close policy; the recorder decorates the resulting Effect `Socket.Socket` service.
- **Connection Identity**, not the live WebSocket destination, selects and validates a replay. The beta does not validate URL or handshake configuration during replay.
- Only a **Completed Conversation** is committed to a cassette; failed, interrupted, unopened, or invalid runs do not produce a recording.
- Replay requires every recorded frame to be consumed before application close.
- Terminal close codes, close reasons, connection timing, and transport failures are not cassette events in the first public beta.
- Every outgoing replay frame must satisfy the **Client Frame Match** before later server frames are released.
- The first public beta does not expose a custom WebSocket frame matcher.
- Replay starts incoming frame handlers in recorded order and may run them concurrently; it waits for every handler before the socket run completes but does not guarantee handler completion order.
53 changes: 36 additions & 17 deletions packages/http-recorder/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ Use it for provider integrations, retries, polling, multi-step flows, and any te
## Install

```sh
bun add effect@4.0.0-beta.74
bun add effect@4.0.0-beta.83 @effect/platform-node@4.0.0-beta.83
bun add -d @opencode-ai/http-recorder@beta @effect/vitest vitest
```

The package supports Node.js 22+ and Bun. It is not intended for browsers, workers, or Deno.

Effect `4.0.0-beta.74` has a known declaration error (`SchemaErrorTypeId` is missing). Until that upstream declaration is fixed, TypeScript consumers need:
Effect `4.0.0-beta.83` currently contains unresolved symbols in its published declarations. Until those upstream declarations are fixed, TypeScript consumers need:

```json
{
Expand Down Expand Up @@ -89,41 +89,58 @@ That is the complete public API. `http` provides a fetch-backed recorded `HttpCl

## WebSockets

WebSocket cassettes preserve one ordered transcript of client and server text or binary frames. Replay follows that chronology: server frames are released until the next recorded client frame, then replay waits for the application to send the matching frame before continuing.
Effect models a WebSocket as a `Socket.Socket` service. A program obtains a scoped `writer` for outgoing frames and runs one receive loop for the lifetime of a connection. The application supplies the live URL-bound socket; the recorder decorates that service without owning its URL, protocols, authentication, timeout, or close policy.

The cassette name is the connection identity during replay. Replay does not validate the live URL or handshake configuration.

```ts
import { assert, it } from "@effect/vitest"
import { it } from "@effect/vitest"
import { NodeSocket } from "@effect/platform-node"
import { Effect, Layer } from "effect"
import { Socket } from "effect/unstable/socket"
import { HttpRecorder } from "@opencode-ai/http-recorder"

const echo = Effect.gen(function* () {
const conversation = Effect.gen(function* () {
const socket = yield* Socket.Socket
const write = yield* socket.writer

yield* socket.runString(
(message) =>
Effect.gen(function* () {
assert.strictEqual(message, "hello")
yield* write(new Socket.CloseEvent(1000))
}),
{ onOpen: write("hello") },
yield* socket.runString((message) =>
Effect.gen(function* () {
const event: unknown = JSON.parse(message)

if (typeof event !== "object" || event === null || !("type" in event)) return
if (event.type === "session.created") {
yield* write(JSON.stringify({ type: "response.create", prompt: "hello" }))
}
if (event.type === "response.completed") {
yield* write(new Socket.CloseEvent(1000, "done"))
}
}),
)
})

const recordedSocket = HttpRecorder.socket("echo/hello").pipe(
const recordedSocket = HttpRecorder.socket("provider/conversation").pipe(
Layer.provide(
NodeSocket.layerWebSocket("wss://ws.postman-echo.com/raw", {
NodeSocket.layerWebSocket("wss://provider.example/realtime", {
closeCodeIsError: (code) => code !== 1000,
}),
),
)

it.effect("exchanges WebSocket frames", () => echo.pipe(Effect.provide(recordedSocket)))
it.effect("completes a provider conversation", () => conversation.pipe(Effect.scoped, Effect.provide(recordedSocket)))
```

The application owns the WebSocket URL and protocols through normal Effect layer wiring. The recorder wraps that socket without duplicating its URL in recorder configuration. Provide separate socket layers for separate endpoints or concurrent connections.
`socket.runString` owns the receive loop and finishes when the connection closes or fails. Its optional `onOpen` effect is the safe place to send protocols whose client speaks first. The writer is scoped because sending is valid only while a connection run is active.

WebSocket cassettes preserve one ordered transcript of client and server text or binary frames. Replay releases recorded server frames until it reaches a client frame, waits for the application to write the matching frame, then continues. This preserves causal ordering without reproducing network timing.

Client text frames containing JSON compare canonically, so object-key order does not matter. Changed fields, extra fields, non-JSON text, and binary frames must match exactly after redaction. There is intentionally no custom WebSocket matcher in this beta.

Incoming frame handlers start in recorded order and may run concurrently, matching Effect's socket abstraction. Replay waits for all handlers before the socket run completes, but handler completion order is not guaranteed. Use Effect synchronization such as `Queue`, `Ref`, or `Deferred` instead of unsynchronized mutable state.

A cassette is written only after the live socket opened and its run completed successfully. Failed, interrupted, unopened, or invalid runs do not produce a recording. During replay, closing before every recorded frame is consumed fails the test.

The application owns the WebSocket URL and protocols through normal Effect layer wiring. Provide separate recorder and live socket layers for separate endpoints or concurrent connections. One recorder layer supports sequential reconnects, but rejects concurrent runs.

Text frames use the same JSON-field and body redaction as HTTP bodies. Binary frames are stored losslessly as base64. Client and server frame kinds must match during replay.

Expand Down Expand Up @@ -195,6 +212,8 @@ interface RecorderOptions {
readonly redact?: RedactOptions
readonly match?: RequestMatcher
}

type SocketRecorderOptions = Omit<RecorderOptions, "match">
```

`directory` defaults to `<cwd>/test/fixtures/recordings`.
Expand All @@ -207,7 +226,7 @@ Cassettes are readable JSON files intended to be committed with your tests. HTTP

- Responses are buffered while recording and replaying, so this beta is not suitable for tests that assert streaming timing, cancellation, or backpressure.
- WebSocket replay preserves frame chronology and content, not real network timing or backpressure.
- WebSocket V1 cassettes do not reproduce terminal close codes, close reasons, or transport failures. Failed and interrupted live runs are not recorded.
- WebSocket V1 cassettes do not reproduce terminal close codes, close reasons, handshake configuration, or transport failures. Failed and interrupted live runs are not recorded.
- WebSocket transcripts are retained in memory until the connection finishes; avoid using this beta for unbounded sessions.
- The package currently requires the exact Effect beta listed above.
- Cassette format version `1` has no migration tooling yet.
Expand Down
33 changes: 33 additions & 0 deletions packages/http-recorder/RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Release

`@opencode-ai/http-recorder` is versioned independently from OpenCode and published under the npm `beta` tag.

## Prepare A Release

1. Add a Changeset for every user-facing package change.
2. Run `bunx changeset status` and confirm that only `@opencode-ai/http-recorder` will be bumped.
3. Merge the package changes into `dev`.
4. From a branch based on the latest `dev`, run `bun run version:http-recorder`.
5. Review the generated version and `packages/http-recorder/CHANGELOG.md`, then open and merge the release PR.

The first minor Changeset advances the unpublished `0.0.0` package to `0.1.0`. OpenCode's repository-wide release script deliberately excludes this package from its version synchronization.

## Verify And Publish

Before merging the release PR, run these commands from `packages/http-recorder`:

```sh
bun run test
bun typecheck
bun run verify:package
```

After the release PR reaches `dev`, manually dispatch the `http-recorder release` workflow from the `dev` branch. The workflow repeats the focused tests, builds and verifies the exact tarball in a clean npm consumer, and publishes it with provenance under the `beta` tag.

The bootstrap release uses the repository's existing `NPM_TOKEN` secret because npm trusted publishing cannot be configured for a package that does not exist yet. After the package exists, configure its npm trusted publisher for repository `anomalyco/opencode` and workflow `http-recorder-release.yml`, then remove `NODE_AUTH_TOKEN` from the workflow so later releases authenticate through GitHub OIDC like the repository's other npm packages.

Verify the result:

```sh
npm view @opencode-ai/http-recorder version dist-tags --json
```
5 changes: 2 additions & 3 deletions packages/http-recorder/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json.schemastore.org/package.json",
"version": "1.17.9",
"version": "0.0.0",
"name": "@opencode-ai/http-recorder",
"description": "Record and replay Effect HTTP client traffic with deterministic cassettes",
"type": "module",
Expand Down Expand Up @@ -51,8 +51,7 @@
"typescript": "catalog:"
},
"dependencies": {
"@effect/platform-node": "4.0.0-beta.83",
"@effect/platform-node-shared": "4.0.0-beta.83"
"@effect/platform-node": "4.0.0-beta.83"
},
"peerDependencies": {
"effect": "4.0.0-beta.83"
Expand Down
34 changes: 34 additions & 0 deletions packages/http-recorder/script/publish.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/usr/bin/env bun
import { $ } from "bun"
import { fileURLToPath } from "node:url"
import { pack } from "./pack.js"
import { verifyPackage } from "./verify-package.js"

const dir = fileURLToPath(new URL("..", import.meta.url))
process.chdir(dir)

const published = async (name: string, version: string) => {
const result = await $`npm view ${name}@${version} version`.quiet().nothrow()
if (result.exitCode === 0) return true
const stderr = result.stderr.toString()
if (stderr.includes("E404")) return false
throw new Error(`Failed to check whether ${name}@${version} is published:\n${stderr}`)
}

// oxlint-disable-next-line typescript-eslint/no-unsafe-type-assertion -- package.json is validated by the package schema and build checks.
const pkg = JSON.parse(await Bun.file("package.json").text()) as { readonly name: string; readonly version: string }
if (pkg.version === "0.0.0") throw new Error("Version the HTTP recorder before publishing")
if (!(await Bun.file("CHANGELOG.md").exists()))
throw new Error("Generate the HTTP recorder changelog before publishing")

if (await published(pkg.name, pkg.version)) {
console.log(`already published ${pkg.name}@${pkg.version}`)
} else {
const archive = await pack()
try {
await verifyPackage(archive)
await $`npm publish ${archive} --tag beta --access public --provenance`
} finally {
await Bun.file(archive).delete()
}
}
9 changes: 6 additions & 3 deletions packages/http-recorder/script/verify-package.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,14 @@ import { Layer } from "effect"
import { HttpClient } from "effect/unstable/http"
import { Socket } from "effect/unstable/socket"

const options: HttpRecorder.RecorderOptions = { redact: { jsonFields: ["access_token"] } }
const options: HttpRecorder.RecorderOptions = { match: () => true, redact: { jsonFields: ["access_token"] } }
const socketOptions: HttpRecorder.SocketRecorderOptions = { redact: { jsonFields: ["access_token"] } }
HttpRecorder.http("consumer", options) satisfies Layer.Layer<HttpClient.HttpClient>
HttpRecorder.socket("consumer/socket", options).pipe(
HttpRecorder.socket("consumer/socket", socketOptions).pipe(
Layer.provide(NodeSocket.layerWebSocket("wss://example.test")),
) satisfies Layer.Layer<Socket.Socket>
// @ts-expect-error HTTP request matching does not apply to WebSocket frames.
HttpRecorder.socket("consumer/socket", { match: () => true })
`,
)
await writeFile(
Expand All @@ -41,7 +44,7 @@ HttpRecorder.socket("consumer/socket", options).pipe(
moduleResolution: "NodeNext",
strict: true,
noEmit: true,
// Required by effect@4.0.0-beta.74: its schema.d.ts references an undeclared SchemaErrorTypeId.
// Required by effect@4.0.0-beta.83: its declarations currently contain unresolved internal symbols.
skipLibCheck: true,
lib: ["ES2022", "DOM", "ESNext.Disposable"],
},
Expand Down
18 changes: 18 additions & 0 deletions packages/http-recorder/script/version.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env bun
import { $ } from "bun"
import { fileURLToPath } from "node:url"

const packages = await Array.fromAsync(
new Bun.Glob("packages/**/package.json").scan({
cwd: fileURLToPath(new URL("../../../", import.meta.url)),
absolute: true,
}),
)
const ignored = (await Promise.all(packages.map(async (file): Promise<unknown> => Bun.file(file).json())))
.map((pkg) =>
typeof pkg === "object" && pkg !== null && "name" in pkg && typeof pkg.name === "string" ? pkg.name : undefined,
)
.filter((name): name is string => name !== undefined && name !== "@opencode-ai/http-recorder")
.flatMap((name) => ["--ignore", name])

await $`bunx changeset version ${ignored}`
2 changes: 2 additions & 0 deletions packages/http-recorder/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,6 @@ export namespace HttpRecorder {
export type RequestMatcher = import("./types.js").RequestMatcher
/** The normalized HTTP request representation used for matching. */
export type RequestSnapshot = import("./types.js").RequestSnapshot
/** Recorder configuration for a provided Effect WebSocket service. */
export type SocketRecorderOptions = import("./types.js").SocketRecorderOptions
}
15 changes: 11 additions & 4 deletions packages/http-recorder/src/socket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { makeReplayState, resolveAutoMode } from "./recorder.js"
import { make, type Redactor } from "./redactor.js"
import { webSocketInteractions } from "./schema.js"
import type {
RecorderOptions,
SocketRecorderOptions,
WebSocketEvent,
WebSocketInteraction,
WebSocketRecorderOptions,
Expand Down Expand Up @@ -302,10 +302,17 @@ const recordingLayer = (
* Wraps a provided `Socket.Socket` with cassette recording and replay.
*
* Supply the ordinary URL-bound Effect socket layer beneath this decorator.
* The cassette name identifies the connection; recorder configuration does not
* duplicate the transport URL.
* The cassette name identifies the connection during replay; recorder
* configuration does not duplicate or validate the transport URL.
*
* A recording is committed only after the socket run completes successfully.
* Replay releases server frames in order and waits at each recorded client
* frame until the application writes a matching frame.
*/
export const socket = (name: string, options: RecorderOptions = {}): Layer.Layer<Socket.Socket, never, Socket.Socket> =>
export const socket = (
name: string,
options: SocketRecorderOptions = {},
): Layer.Layer<Socket.Socket, never, Socket.Socket> =>
provideCassette(recordingLayer(name, { url: "" }, { ...options, compareClientMessagesAsJson: true }), options)

/** @internal */
Expand Down
Loading
Loading