Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 126 additions & 0 deletions .claude/skills/composable-pipelines/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
name: composable-pipelines
description: Build AI pipelines with the ComposablePipelines Swift package — author a declarative Pipeline (result-builder DSL + @State), compile it to a graph, and run it through an Executor. Use when writing or debugging ComposablePipelines code: Pipeline/Model/Guardrail/While/ForEach/ClientTask, @State data flow, the Executor seam, reactive While/branching, tool-calling agent loops, or when a file imports ComposablePipelines / PipelineDSL / PipelineCompiler / ExecutionEngine.
---

# Composable Pipelines

A runtime-agnostic stack for AI pipelines in Swift: a declarative **DSL** lowers to a `Codable`
**AST**, a **compiler** emits an execution graph, and an observable **walker** runs it — while an
`Executor` you supply decides what "run a model" means.

## Quick start

```swift
import ComposablePipelines

struct Summary: Pipeline {
typealias Output = String
let document: String

@State var keyPoints = ""
@State var summary = ""

var body: some Pipeline {
$keyPoints.set { // step 1 → slot
Model<String>().systemPrompt("Extract the 5 key points.").message(document)
}
$summary.set { // step 2 reads step 1
Model<String>().systemPrompt("Summarize from these points.").input { $keyPoints.get() }
}
$summary.get() // pipeline output
}
}

// Run it (linear flow): compile → walk.
let graph = PipelineCompiler().compile(pipeline.loweredGraph())
let result = try await PipelineWalker(executor: MyExecutor()).run(graph: graph) { event in print(event) }
let text = try JSONDecoder().decode(String.self, from: result) // ExecutionValue == Data (JSON)
```

## Authoring cheatsheet

- **Pipeline**: `struct X: Pipeline { typealias Output = T; @State var … ; var body: some Pipeline { … } }`.
The graph is inferred from the `@State` slots each step reads/writes — never wired by hand.
- **State**: `@State var s = ""` → write with `$s.set { <pipeline> }` or `$s.set(value)`; read with
`$s.get()`. End `body` with the output slot's `.get()`.
- **Model** (output-only generic): `Model<Output>()` then chain `.systemPrompt(_)`,
`.input { $slot.get() }` **or** `.message(staticValue)`, and optionally `.tools([…])`,
`.temperature(_)`, `.maxTokens(_)`, `.requirements(ModelSelectionRequirements(traits: […]))`.
- **Primitives**: `Guardrail(input, rules:allowed:blocked:)`, `While(condition:) { … }`,
`Group { … }`, `ForEach(in: xs) { x in … }`, `Summarize(text: $slot, maxTokens:)`,
`ClientTask(input: $slot) { value in … }`, `From(provider, query: $slot)`, `Self.return(value)`.
- **Control flow**: use native `if` / `switch` on produced state (e.g. `if verdict == "trivial"`).

## Running a pipeline

- **Linear flow** (sequential, `ForEach`, retrieval, runtime `.get()`): compile once, then
`PipelineWalker(executor:).run(graph:)`.
- **Reactive flow** (a `While` loop, or `if`/`switch` that branch on a *produced* value): use
`PipelineRunner.run(pipeline, executor:)`. It re-lowers and recompiles after each committed write
so the condition/branch re-reads state. A single `walker.run` is **not** enough for these.

## The Executor seam

The walker never calls a model itself. Implement one method:

```swift
struct MyExecutor: Executor {
func runModel(config: ModelConfig, arguments: ModelArguments,
onDelta: (@Sendable (String) -> Void)?) async throws -> ExecutionValue {
let system = arguments.systemPrompt // String
let user: String; if case .string(let s)? = arguments.message { user = s } else { user = "" }
// … call your model … then encode the output as JSON Data:
return try JSONEncoder().encode(reply) // or ModelTurn for tool turns
}
}
```

`MockExecutor` ships for tests; `OpenAIChatExecutor` (Foundation-only) talks to any
OpenAI-compatible endpoint (local or remote).

## Tool-calling agents

A model step whose output is `ModelTurn` can request tools; you dispatch them and loop:

```swift
While(condition: { reply.isEmpty && turns < maxTurns }) {
$lastTurn.set { Model<ModelTurn>().tools(tools.map(\.descriptor)).systemPrompt(prompt).input { $transcript.get() } }
$transcript.set {
ClientTask(input: $lastTurn) { turn in
guard let calls = turn.toolCalls, !calls.isEmpty else { return transcript }
var t = transcript
for call in calls {
let out = try await ToolRegistry(tools).executeJSON(toolName: call.name, inputJSON: Data(call.arguments.utf8))
t += "\n[\(call.name)] " + String(decoding: out, as: UTF8.self)
}
return t
}
}
$reply.set { ClientTask(input: $lastTurn) { $0.toolCalls?.isEmpty == false ? "" : ($0.reply ?? "") } }
$turns.set { ClientTask(input: $turns) { $0 + 1 } }
}
$reply.get()
```

Tools are `ModelTool`s (static `name`/`description`/`inputSchema` + `func call(_:) -> Output`),
registered in a `ToolRegistry`. Drive tool/agent loops with `PipelineRunner.run`.

## Pitfalls (read before writing code)

- **`Model` takes one generic — the output.** It's `Model<Output>().systemPrompt(…).input { … }`,
**not** `Model<Input, Output>(instructions:input:)` (that API does not exist).
- **Never read `@State` inside a `ClientTask` closure.** Thread state in via the typed input:
`ClientTask(input: $slot) { value in … }`. Reading `@State` in the closure is disallowed.
- **`While` / value-dependent branching need `PipelineRunner.run`**, not a bare `walker.run` — they
require reactive re-lowering.
- **`ExecutionValue` is JSON `Data`.** Decode results (`JSONDecoder().decode(T.self, from:)`) and
encode executor outputs (`JSONEncoder().encode(_)`).
- **A turn can carry a preamble *and* tool calls.** In an agent loop, keep looping while there are
tool calls; only a tool-free turn is the final answer.

## Reference

Deep docs live in the package: [`docs/examples.md`](../../../docs/examples.md) (worked pipelines
with code), `docs/primitives.md`, `docs/executors.md`, `docs/architecture.md`, and the top-level
`README.md`. The runnable examples in `Examples/` are execution-tested — copy from them.
55 changes: 55 additions & 0 deletions .claude/skills/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/sh
#
# Install the ComposablePipelines agent skills into the skill directories of major AI coding agents.
#
# Usage:
# curl -fsSL https://raw.githubusercontent.com/MacPaw/ComposablePipelines/main/.claude/skills/install.sh | sh
#
# Installs every skill bundled in this repo (each directory under .claude/skills/ with a SKILL.md),
# so new skills are picked up automatically. By default it installs for Claude Code and for any other
# supported agent whose config directory already exists. Override the destinations explicitly:
# SKILL_DIRS="$HOME/.claude/skills $HOME/.codex/skills" # space-separated
# Other env: CP_SKILLS_REPO (repo URL), CP_SKILLS_BRANCH (branch).
#
set -eu

REPO="${CP_SKILLS_REPO:-https://github.com/MacPaw/ComposablePipelines}"
BRANCH="${CP_SKILLS_BRANCH:-main}"

command -v git >/dev/null 2>&1 || { echo "error: git is required to install the skills" >&2; exit 1; }

# Destinations: explicit override, or Claude Code + any other agent whose home dir is present.
if [ -n "${SKILL_DIRS:-}" ]; then
targets="$SKILL_DIRS"
else
targets="$HOME/.claude/skills" # Claude Code (always)
[ -d "$HOME/.codex" ] && targets="$targets $HOME/.codex/skills" # Codex CLI
[ -d "$HOME/.config/opencode" ] && targets="$targets $HOME/.config/opencode/skills" # opencode
[ -d "$HOME/.gemini" ] && targets="$targets $HOME/.gemini/skills" # Gemini CLI
[ -d "$HOME/.copilot" ] && targets="$targets $HOME/.copilot/skills" # Copilot CLI
fi

tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT

echo "Fetching skills from $REPO ($BRANCH)…"
git clone --depth 1 --branch "$BRANCH" "$REPO" "$tmp/repo" >/dev/null 2>&1
src="$tmp/repo/.claude/skills"

total=0
for dest in $targets; do
mkdir -p "$dest"
n=0
for dir in "$src/"*/; do
[ -f "${dir}SKILL.md" ] || continue # only real skill directories
name="$(basename "$dir")"
rm -rf "$dest/$name"
cp -R "$dir" "$dest/$name"
n=$((n + 1))
done
echo " $dest ($n skill(s))"
total=$((total + n))
done

[ "$total" -gt 0 ] || { echo "error: no skills found under .claude/skills/ in $REPO" >&2; exit 1; }
echo "Done. Restart your agent to pick up the skills."
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ the data dependencies and emits an execution graph; and an observable interprete
— runs that graph with incremental, epoch-based re-execution, delegating every model and tool step
to a backend you supply through a single `Executor` interface.

The same primitives scale from a two-step summary to a full **tool-calling agent**: loops, tools,
and branching are ordinary pipeline constructs. See the
[coding-agent worked example](#worked-example-a-coding-agent-as-a-pipeline) — a real agent built
entirely on this DSL.

```swift
import ComposablePipelines

Expand Down Expand Up @@ -296,6 +301,20 @@ struct CodingAgentPipeline: Pipeline {
> are rejected). `bash` is an escape hatch — it runs with the directory as its cwd but is not
> otherwise sandboxed, so point the agent at a scratch directory and a model you trust.

## Build pipelines with your AI agent

An agent **skill** aggregates the API, patterns, and common pitfalls of building pipelines, so your
AI coding agent authors and debugs them correctly. Install it for the major agents:

```bash
curl -fsSL https://raw.githubusercontent.com/MacPaw/ComposablePipelines/main/.claude/skills/install.sh | sh
```

The script installs every bundled skill into the skill directory of each supported agent it finds
(Claude Code, Codex, opencode, Gemini CLI, Copilot CLI); set `SKILL_DIRS` to choose destinations.
Each skill activates automatically when you work in a project that uses ComposablePipelines. Source:
[`.claude/skills/`](.claude/skills/).

## Documentation

- [Getting started](docs/getting-started.md) — install, your first pipeline, compile + walk.
Expand Down
Loading