diff --git a/.claude/skills/composable-pipelines/SKILL.md b/.claude/skills/composable-pipelines/SKILL.md new file mode 100644 index 0000000..5fc1605 --- /dev/null +++ b/.claude/skills/composable-pipelines/SKILL.md @@ -0,0 +1,126 @@ +--- +name: composable-pipelines +description: Build AI pipelines with the ComposablePipelines Swift package — author a declarative Pipeline (result-builder DSL + @State), compile it to a graph, and run it through an Executor. Use when writing or debugging ComposablePipelines code: Pipeline/Model/Guardrail/While/ForEach/ClientTask, @State data flow, the Executor seam, reactive While/branching, tool-calling agent loops, or when a file imports ComposablePipelines / PipelineDSL / PipelineCompiler / ExecutionEngine. +--- + +# Composable Pipelines + +A runtime-agnostic stack for AI pipelines in Swift: a declarative **DSL** lowers to a `Codable` +**AST**, a **compiler** emits an execution graph, and an observable **walker** runs it — while an +`Executor` you supply decides what "run a model" means. + +## Quick start + +```swift +import ComposablePipelines + +struct Summary: Pipeline { + typealias Output = String + let document: String + + @State var keyPoints = "" + @State var summary = "" + + var body: some Pipeline { + $keyPoints.set { // step 1 → slot + Model().systemPrompt("Extract the 5 key points.").message(document) + } + $summary.set { // step 2 reads step 1 + Model().systemPrompt("Summarize from these points.").input { $keyPoints.get() } + } + $summary.get() // pipeline output + } +} + +// Run it (linear flow): compile → walk. +let graph = PipelineCompiler().compile(pipeline.loweredGraph()) +let result = try await PipelineWalker(executor: MyExecutor()).run(graph: graph) { event in print(event) } +let text = try JSONDecoder().decode(String.self, from: result) // ExecutionValue == Data (JSON) +``` + +## Authoring cheatsheet + +- **Pipeline**: `struct X: Pipeline { typealias Output = T; @State var … ; var body: some Pipeline { … } }`. + The graph is inferred from the `@State` slots each step reads/writes — never wired by hand. +- **State**: `@State var s = ""` → write with `$s.set { }` or `$s.set(value)`; read with + `$s.get()`. End `body` with the output slot's `.get()`. +- **Model** (output-only generic): `Model()` then chain `.systemPrompt(_)`, + `.input { $slot.get() }` **or** `.message(staticValue)`, and optionally `.tools([…])`, + `.temperature(_)`, `.maxTokens(_)`, `.requirements(ModelSelectionRequirements(traits: […]))`. +- **Primitives**: `Guardrail(input, rules:allowed:blocked:)`, `While(condition:) { … }`, + `Group { … }`, `ForEach(in: xs) { x in … }`, `Summarize(text: $slot, maxTokens:)`, + `ClientTask(input: $slot) { value in … }`, `From(provider, query: $slot)`, `Self.return(value)`. +- **Control flow**: use native `if` / `switch` on produced state (e.g. `if verdict == "trivial"`). + +## Running a pipeline + +- **Linear flow** (sequential, `ForEach`, retrieval, runtime `.get()`): compile once, then + `PipelineWalker(executor:).run(graph:)`. +- **Reactive flow** (a `While` loop, or `if`/`switch` that branch on a *produced* value): use + `PipelineRunner.run(pipeline, executor:)`. It re-lowers and recompiles after each committed write + so the condition/branch re-reads state. A single `walker.run` is **not** enough for these. + +## The Executor seam + +The walker never calls a model itself. Implement one method: + +```swift +struct MyExecutor: Executor { + func runModel(config: ModelConfig, arguments: ModelArguments, + onDelta: (@Sendable (String) -> Void)?) async throws -> ExecutionValue { + let system = arguments.systemPrompt // String + let user: String; if case .string(let s)? = arguments.message { user = s } else { user = "" } + // … call your model … then encode the output as JSON Data: + return try JSONEncoder().encode(reply) // or ModelTurn for tool turns + } +} +``` + +`MockExecutor` ships for tests; `OpenAIChatExecutor` (Foundation-only) talks to any +OpenAI-compatible endpoint (local or remote). + +## Tool-calling agents + +A model step whose output is `ModelTurn` can request tools; you dispatch them and loop: + +```swift +While(condition: { reply.isEmpty && turns < maxTurns }) { + $lastTurn.set { Model().tools(tools.map(\.descriptor)).systemPrompt(prompt).input { $transcript.get() } } + $transcript.set { + ClientTask(input: $lastTurn) { turn in + guard let calls = turn.toolCalls, !calls.isEmpty else { return transcript } + var t = transcript + for call in calls { + let out = try await ToolRegistry(tools).executeJSON(toolName: call.name, inputJSON: Data(call.arguments.utf8)) + t += "\n[\(call.name)] " + String(decoding: out, as: UTF8.self) + } + return t + } + } + $reply.set { ClientTask(input: $lastTurn) { $0.toolCalls?.isEmpty == false ? "" : ($0.reply ?? "") } } + $turns.set { ClientTask(input: $turns) { $0 + 1 } } +} +$reply.get() +``` + +Tools are `ModelTool`s (static `name`/`description`/`inputSchema` + `func call(_:) -> Output`), +registered in a `ToolRegistry`. Drive tool/agent loops with `PipelineRunner.run`. + +## Pitfalls (read before writing code) + +- **`Model` takes one generic — the output.** It's `Model().systemPrompt(…).input { … }`, + **not** `Model(instructions:input:)` (that API does not exist). +- **Never read `@State` inside a `ClientTask` closure.** Thread state in via the typed input: + `ClientTask(input: $slot) { value in … }`. Reading `@State` in the closure is disallowed. +- **`While` / value-dependent branching need `PipelineRunner.run`**, not a bare `walker.run` — they + require reactive re-lowering. +- **`ExecutionValue` is JSON `Data`.** Decode results (`JSONDecoder().decode(T.self, from:)`) and + encode executor outputs (`JSONEncoder().encode(_)`). +- **A turn can carry a preamble *and* tool calls.** In an agent loop, keep looping while there are + tool calls; only a tool-free turn is the final answer. + +## Reference + +Deep docs live in the package: [`docs/examples.md`](../../../docs/examples.md) (worked pipelines +with code), `docs/primitives.md`, `docs/executors.md`, `docs/architecture.md`, and the top-level +`README.md`. The runnable examples in `Examples/` are execution-tested — copy from them. diff --git a/.claude/skills/install.sh b/.claude/skills/install.sh new file mode 100755 index 0000000..ee6c78f --- /dev/null +++ b/.claude/skills/install.sh @@ -0,0 +1,55 @@ +#!/bin/sh +# +# Install the ComposablePipelines agent skills into the skill directories of major AI coding agents. +# +# Usage: +# curl -fsSL https://raw.githubusercontent.com/MacPaw/ComposablePipelines/main/.claude/skills/install.sh | sh +# +# Installs every skill bundled in this repo (each directory under .claude/skills/ with a SKILL.md), +# so new skills are picked up automatically. By default it installs for Claude Code and for any other +# supported agent whose config directory already exists. Override the destinations explicitly: +# SKILL_DIRS="$HOME/.claude/skills $HOME/.codex/skills" # space-separated +# Other env: CP_SKILLS_REPO (repo URL), CP_SKILLS_BRANCH (branch). +# +set -eu + +REPO="${CP_SKILLS_REPO:-https://github.com/MacPaw/ComposablePipelines}" +BRANCH="${CP_SKILLS_BRANCH:-main}" + +command -v git >/dev/null 2>&1 || { echo "error: git is required to install the skills" >&2; exit 1; } + +# Destinations: explicit override, or Claude Code + any other agent whose home dir is present. +if [ -n "${SKILL_DIRS:-}" ]; then + targets="$SKILL_DIRS" +else + targets="$HOME/.claude/skills" # Claude Code (always) + [ -d "$HOME/.codex" ] && targets="$targets $HOME/.codex/skills" # Codex CLI + [ -d "$HOME/.config/opencode" ] && targets="$targets $HOME/.config/opencode/skills" # opencode + [ -d "$HOME/.gemini" ] && targets="$targets $HOME/.gemini/skills" # Gemini CLI + [ -d "$HOME/.copilot" ] && targets="$targets $HOME/.copilot/skills" # Copilot CLI +fi + +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT + +echo "Fetching skills from $REPO ($BRANCH)…" +git clone --depth 1 --branch "$BRANCH" "$REPO" "$tmp/repo" >/dev/null 2>&1 +src="$tmp/repo/.claude/skills" + +total=0 +for dest in $targets; do + mkdir -p "$dest" + n=0 + for dir in "$src/"*/; do + [ -f "${dir}SKILL.md" ] || continue # only real skill directories + name="$(basename "$dir")" + rm -rf "$dest/$name" + cp -R "$dir" "$dest/$name" + n=$((n + 1)) + done + echo " $dest ($n skill(s))" + total=$((total + n)) +done + +[ "$total" -gt 0 ] || { echo "error: no skills found under .claude/skills/ in $REPO" >&2; exit 1; } +echo "Done. Restart your agent to pick up the skills." diff --git a/README.md b/README.md index 22f3db7..184ab26 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,11 @@ the data dependencies and emits an execution graph; and an observable interprete — runs that graph with incremental, epoch-based re-execution, delegating every model and tool step to a backend you supply through a single `Executor` interface. +The same primitives scale from a two-step summary to a full **tool-calling agent**: loops, tools, +and branching are ordinary pipeline constructs. See the +[coding-agent worked example](#worked-example-a-coding-agent-as-a-pipeline) — a real agent built +entirely on this DSL. + ```swift import ComposablePipelines @@ -296,6 +301,20 @@ struct CodingAgentPipeline: Pipeline { > are rejected). `bash` is an escape hatch — it runs with the directory as its cwd but is not > otherwise sandboxed, so point the agent at a scratch directory and a model you trust. +## Build pipelines with your AI agent + +An agent **skill** aggregates the API, patterns, and common pitfalls of building pipelines, so your +AI coding agent authors and debugs them correctly. Install it for the major agents: + +```bash +curl -fsSL https://raw.githubusercontent.com/MacPaw/ComposablePipelines/main/.claude/skills/install.sh | sh +``` + +The script installs every bundled skill into the skill directory of each supported agent it finds +(Claude Code, Codex, opencode, Gemini CLI, Copilot CLI); set `SKILL_DIRS` to choose destinations. +Each skill activates automatically when you work in a project that uses ComposablePipelines. Source: +[`.claude/skills/`](.claude/skills/). + ## Documentation - [Getting started](docs/getting-started.md) — install, your first pipeline, compile + walk.