B: dreaming harness — Pi AgentSession for the daemon plan phase (A: inference cutover shipped via #949)

[NicholaiVogel]

Status (2026-07-22): Workstream A SHIPPED via #949 (merged to main as b36d3586f). This issue is REOPENED and now tracks only Workstream B (the dreaming harness).

• ✅ A — Inference provisioning cutover: DONE. provider.ts collapsed 4073→~1000 LOC; @earendil-works/pi-ai is the single direct-API backend, ACPX the single harness-subprocess backend; Claude Code / OpenCode / Codex / command-line providers deleted; one-time config migration; aggregate-recall separated as a pi-ai-only latency path. Verified: 8/8 operation kinds pass a live pi-ai gauntlet, 45/45 tests pass, 4 adversarial-review blockers fixed.
• 🔲 B — Dreaming harness: OPEN. No Pi AgentSession exists in the daemon yet; the plan phase still runs on legacy platform/daemon/src/pipeline/dreaming-worker.ts (the "broken third path" per Unify Background Memory around Source Aggregation and Dreaming #913, enabled: false). B consumes A's substrate (now shipped) and fills Unify Background Memory around Source Aggregation and Dreaming #913's plan phase. The full B spec — triggers, agent loop, and the custom dreaming-tool surface — is preserved below unchanged for whoever picks this up.
• Follow-up (not in scope): platform/daemon-rs parity for the A changes was waived for feat(daemon): replace inference provisioning with pi-ai (#947) #949 (owner-approved). That gap is tracked separately.

---

Summary

Replace Signet's hand-rolled inference provisioning with Pi's provider/auth/model stack as a hard cutover, and build the dreaming harness as a Pi agent session embedded in the daemon. This issue owns two separable workstreams:

• A — Inference provisioning (ships first, independently of Unify Background Memory around Source Aggregation and Dreaming #913). The 4,073-line platform/daemon/src/pipeline/provider.ts — with per-provider subprocess management for Claude Code, OpenCode, Codex, Ollama, llama.cpp, ACPX, and direct HTTP — is replaced by a single Pi-backed provider plus a retained ACPX backend. Hard cutover, no fallback path.
• B — Dreaming harness. The daemon's plan phase (per Unify Background Memory around Source Aggregation and Dreaming #913) runs as a Pi AgentSession with custom dreaming tools, sharing A's auth/model substrate.

A is a prerequisite for B and ships ahead of #913's cutover.

Current State

Our inference provisioning is a 4,073-line provider.ts (grown from the 3,448 noted when this issue opened) with ad-hoc, per-provider implementations:

• Claude Code, OpenCode, and Codex spawned as headless subprocesses (process-tree cleanup, fragile across every harness/distribution/runtime update)
• Anthropic, OpenAI-compatible, Ollama, llama.cpp as direct HTTP
• ACPX as a harness subprocess protocol
• A custom global concurrency semaphore, per-provider rate limiter, usage tracking, and a Claude Code circuit breaker

Each new provider means another block of subprocess/HTTP code. Using agent CLIs (Claude Code, OpenCode, Codex) as extraction providers is the wrong pattern, and carrying six hand-rolled implementations alongside Pi would be a parallel fallback path Signet should not maintain. The legacy pieces left over from Signet's initial development can't be allowed to block progress.

Pi as the Inference Backend (Workstream A)

Two backends, not six

After this work the inference layer has exactly two backends behind the existing LlmProvider interface (re-exported from @signet/core, consumed by 35 call sites — those call sites are unchanged):

1. Pi — every direct API call (Anthropic, OpenAI including Codex, Google, Bedrock, Mistral, Azure, Cloudflare, Copilot, and the rest of pi-ai's built-in providers).
2. ACPX — retained and promoted from a special case to a first-class peer endpoint in the agent.yaml routing registry, for cases that genuinely need a harness subprocess.

Claude Code, OpenCode, Codex, and the generic command-line providers are folded, not migrated: deleted outright. Their capability (running those models headlessly) is provided by Pi's API providers and, where a harness is genuinely needed, by ACPX. Signet stops owning per-harness subprocess complexity.

The real Pi API

This issue originally referenced ModelRuntime.create(). That symbol does not exist in the installed Pi SDK (v0.79.2) — verified against dist/core/sdk.d.ts and dist/index.d.ts. The actual substrate is:

• Model + getModel() from @earendil-works/pi-ai — the single LLM-call primitive (one-shot and streaming). This is what replaces each hand-rolled provider.
• AuthStorage with pluggable backends (FileAuthStorageBackend, InMemoryAuthStorageBackend) — unified auth including OAuth for Codex/Copilot/Claude Pro/Grok, shell-command credentials, env interpolation, literal keys.
• ModelRegistry — auto-refreshing model catalog with offline cache.
• createAgentSession() + defineTool / ToolDefinition — the agent loop, used by workstream B.

So the real shared substrate for A is AuthStorage + ModelRegistry + pi-ai Model, not one "runtime" object.

What is preserved (Pi does not replace these)

Four load-bearing pieces in provider.ts stay, wrapped around the Pi provider exactly as today's factories are:

• LlmConcurrencySemaphore / configureLlmConcurrency — the fix(pipeline): bound and cancel LLM inference #918 global daemon cap (default 2, bounded 1..16). The Pi provider acquires through it like every other provider; Pi's internal calls must not bypass the daemon-global cap.
• TokenBucketRateLimiter / withRateLimit — per-provider throttling.
• generateWithTracking — usage/diagnostics surface.
• Claude Code circuit breaker (ClaudeCodeCircuitOpenError, quota/billing cooldown) — reassessed once Pi's own retry/cooldown covers those states.

Dependency, credentials, and config

• Dependency. @earendil-works/pi-ai is added to platform/daemon/package.json as a real dependency. Users do not install Pi separately. (The full @earendil-works/pi-coding-agent is only needed for workstream B's AgentSession; A depends on pi-ai only, keeping the TUI out of the inference path.)
• Credentials. Pi's AuthStorage is backed by a Signet-native backend reading $SIGNET_WORKSPACE/.secrets/. Pi never owns the credential file.
• Config. Model objects are constructed explicitly from Signet's routing config in agent.yaml. Pi's SettingsManager and models.json are bypassed entirely — one config source, not two.
• Migration. A one-time silent migration rewrites old agent.yaml provider blocks to Pi/ACPX-backed target refs on daemon startup. This is a data transform (old config → canonical config in place), not perpetual compat: after it runs, runtime reads canonical config only. Anything it cannot map is reported; nothing is silently dropped.

Coverage note (spike must close)

@earendil-works/pi-ai ships native stream providers for anthropic, openai (responses/completions/codex), google, google-vertex, azure-openai, mistral, bedrock, cloudflare, and github-copilot (OAuth headers). Ollama and llama.cpp are not native pi-ai providers — both expose OpenAI-compatible endpoints and would route through openai-completions. That almost certainly works, but under hard cutover there is no fallback, so the spike proves it including local-model edge cases (custom base URLs, no auth, long timeouts).

Sequencing

1. Spike (gate, no production code). Clone pi-mono into references/ (sibling-harness inspection). Verify against real source: Ollama/llama.cpp via the OpenAI-compatible shim, auth resolution off Signet secrets, usage/abort/timeout/streaming parity, and that HTTP providers are subprocess-free. Hard cutover has no fallback, so the spike has veto power — if it finds a load-bearing gap Pi cannot close quickly, the approach reopens rather than shipping a regression.
2. Pi provider behind LlmProvider. New inference module: Pi adapter + retained/promoted ACPX backend + routing plumbing + the four preserved pieces. Feature-flagged off until parity-proven.
3. Hard cutover. Delete the six hand-rolled providers (createClaudeCodeProvider, createOpenAiCompatibleProvider, createOllamaProvider, createLlamaCppProvider, createCommandLineProvider, and the native OpenCode/Codex paths). Collapse provider.ts to the adapter + preserved broker (~750 LOC; net ~3,300 deleted).
4. AGENTS.md policy edits (same PR). Remove the platform/daemon-rs parity requirement and the "silent compat for old/malformed config keys" phrase from "Will Not Merge." The "runtime reads canonical config only" line stays — the one-time migration satisfies it.
5. daemon-rs parity waived for this PR (owner-approved exception); tracked as follow-up.

Testing / proof

• Unit parity tests with a mocked pi-ai Model (in CI) — content, usage, streaming events, abort/timeout, error shapes that downstream code pattern-matches on.
• Opt-in live tests (*.live.test.ts, env-gated, never in CI) — same pattern as reranker-llm.live.test.ts.
• For each deleted provider, a test that would fail on the old factory and pass on the Pi path.
• Runtime proof on the real daemon before the default-flip (installed-CLI/daemon change gate).

The Dreaming Harness (Workstream B)

Pi is also the dreaming harness. Instead of making a single LLM call in the plan phase (current design), the daemon creates a Pi AgentSession with custom dreaming tools. The agent:

1. Wakes up on a trigger (cron, token threshold, manual)
2. Reads from the same unified queue the pipeline uses (as described in Unify Background Memory around Source Aggregation and Dreaming #913)
3. Explores the memory graph using tool calls (graph inspection, memory recall)
4. Checks existing claims, dependencies, and evidence
5. Makes decisions about what to create, update, or supersede
6. Submits a DreamPlan through the shared validate/apply phase

The deterministic parts (dedup, embeddings, significance checks, write gating, durability gate) stay as library functions the agent calls internally. The daemon still owns the queue, leases, validation, and writes. Pi fills the reasoning slot with an autonomous agent instead of a single LLM call.

B reuses A's AuthStorage/ModelRegistry/Model substrate directly — same credentials, same model catalog, same concurrency cap — so the agent session and every other daemon LLM call share one inference backend.

Custom Tool Surface for the Dreaming Agent

The Pi agent session registers the following tool categories:

Graph inspection

• knowledge_tree / knowledge_get_entity: traverse entity aspect group claim outline
• knowledge_list_entities / entity_list: find entities by name/type/filter
• entity_aspects / entity_groups / entity_claims / entity_attributes: drill into graph paths
• knowledge_hygiene_report: duplicates, suspicious entities, orphans
• ontology_aliases / ontology_links / ontology_conflicts: structural reads

Memory recall

• memory_search / signet_recall: hybrid vector + keyword search
• memory_get: full memory by id
• signet_session_search: transcript search
• signet_source_search: source artifact search

Graph mutation

• ontology_stream_apply: batch operations (set_claim_value, create_entity, etc.)
• ontology_entity_merge: deduplicate entities with impact preview
• ontology_assertion_create / ontology_assertion_import: epistemic assertions

Memory write (source-backed only, never the raw remember endpoint)

• memory_store with provenance fields
• memory_modify: status changes, supersede
• memory_forget: remove stale memories

Dreaming management

• dream_status: worker state, last pass info
• runbook_read / runbook_write: dreaming pass log
• queue_status: pending/leased/failed/dead counts

Pipeline and ingest

• ingest_lease: claim next queue item
• ingest_apply: submit DreamPlan
• ingest_release / ingest_fail

Direct graph transactions

• tx_persist_entities: entities, aspects, relations, mentions
• tx_dependency_upsert: entity dependency rows
• tx_mention_link: link mentions to memories

Dependency extraction

• structural_dependency_analyze
• structural_dependency_status

Prospective indexing

• prospective_hint_enqueue
• prospective_hint_status

Relation to Issue #913

This supports #913 by providing both:

1. The in-process inference layer for the daemon dreaming path (Pi's pi-ai Model/AuthStorage/ModelRegistry replaces provider.ts)
2. The agentic reasoning loop for the plan phase (Pi AgentSession with custom dreaming tools replaces a single LLM call)

A ships first and independently; B consumes A's substrate and fills #913's plan phase.

Related

• Unify Background Memory around Source Aggregation and Dreaming #913 — the unified dreaming refactor this is the B-track for.
• Inference settings ignore pi-ai auth: no OAuth sign-in (Claude Max/Codex/Copilot), OpenRouter models don't load, keys saved as Anthropic #966 — pi-ai OAuth/auth integration needed before AgentSession can authenticate with provider-harness connections.
• Replace the Signet dashboard with TypeScript + shad-cn and build proper UX #948 — dashboard rewrite will surface AgentSession lifecycle state.

[NicholaiVogel]

Reopening to track Workstream B (the dreaming harness) — it was part of the original issue scope and was not built in #949.

• A (inference cutover): shipped via feat(daemon): replace inference provisioning with pi-ai (#947) #949, merged to main (b36d3586f). Verified end-to-end against live pi-ai.
• B (dreaming harness): still open. The daemon plan phase still runs on the legacy dreaming-worker.ts (the Unify Background Memory around Source Aggregation and Dreaming #913 "broken third path"). B consumes A's now-shipped substrate (AuthStorage / ModelRegistry / pi-ai Model) and replaces that worker with a Pi AgentSession + the custom dreaming-tool surface spec'd in the body.

Title and body updated to reflect this. The full B spec below is unchanged. If B should be a separate issue instead of continuing under #947, say the word and I'll split it.