You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Browse filesBrowse the repository at this point in the historyBrowse files
t
committed
docs: accuracy fixes + clear-writing pass on README and docs/
- Correct overclaims surfaced while writing the site: holotype is a
(specialized) backup, not 'not a backup'; deposits are automatic after
setup; installing as a skill is optional, not required.
- Fix stale principle 6 to reflect consent-driven auto-push (v2.4.0).
- Add 'deliberately overkill', 'built Claude Code first', set-and-forget
install model, and non-programmer-audience framing.
- Remove all em dashes across README and docs/ (clear-writing pass).
Copy file name to clipboardExpand all lines: docs/ADDING_A_SOURCE.md
+16-16Lines changed: 16 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
You are reading this because you are an LLM invoked through an agent CLI that holotype does not yet recognize. Your CLI stores session transcripts somewhere, in some format, and the user wants you to archive them with holotype's forensic guarantees alongside their other sessions.
4
4
5
-
This document is your contract. If you follow it, the LLM who reads the archive in 5 years can verify, search, and cite your transcripts the same way they verify, search, and cite Claude Code or Codex transcripts. The rest of holotype does not need to change to accommodate your CLI — only one new file in `holotype/sources/` does.
5
+
This document is your contract. If you follow it, the LLM who reads the archive in 5 years can verify, search, and cite your transcripts the same way they verify, search, and cite Claude Code or Codex transcripts. The rest of holotype does not need to change to accommodate your CLI: only one new file in `holotype/sources/` does.
# Yield one DepositCandidate per transcript file under source_root.
25
25
# MUST refuse to traverse outside source_root (e.g. symlinks
26
-
# that resolve elsewhere — see CodexSource.discover for the
26
+
# that resolve elsewhere, see CodexSource.discover for the
27
27
# pattern).
28
28
...
29
29
@@ -50,28 +50,28 @@ Don't guess from documentation. Inspect actual files on the user's machine. The
50
50
51
51
1.**Find where the CLI stores transcripts.** Look in `~/.<cli-name>/`, `~/Library/Application Support/<CLI>/`, `~/.config/<cli-name>/`. Look for files whose names include UUIDs, timestamps, "session", "conversation", "thread", "rollout". Run `ls -la` and read what's there. Confirm with the user before assuming.
52
52
53
-
2.**Categorize the sensitivity of nearby files.** If `auth.json`, `credentials`, `*.key`, `*.token`, or `config.toml` with API keys live near the transcripts, your `default_source_paths()` MUST point at the deepest dir that contains only transcripts. holotype enforces no-walk-up at the ingest level — make that boundary explicit.
53
+
2.**Categorize the sensitivity of nearby files.** If `auth.json`, `credentials`, `*.key`, `*.token`, or `config.toml` with API keys live near the transcripts, your `default_source_paths()` MUST point at the deepest dir that contains only transcripts. holotype enforces no-walk-up at the ingest level: make that boundary explicit.
54
54
55
55
3.**Pick the largest transcript file you can find** (`find <dir> -name '*.jsonl' -o -name '*.json' | xargs wc -l | sort -rn | head -5`) and read its first ~20 lines. Most agent CLIs use JSONL where each line is one event, but some use a single JSON document with a `messages` array, and a few use SQLite. Figure out which.
56
56
57
57
4.**Map your format's roles to the `MessageInfo` fields:**
58
-
-`role` —`"user"` / `"assistant"` / `"tool_result"` / etc. Whatever your CLI calls these.
59
-
-`timestamp` — ISO-8601 string preferred. If your CLI uses Unix epoch, convert.
60
-
-`model` — the model ID for assistant turns. None for user turns. Don't put the *provider* name here (e.g. "openai", "anthropic") — those aren't model IDs and they pollute `manifest.models`.
61
-
-`has_tool_use` — True if this turn contains a tool call.
62
-
-`has_thinking` — True if this turn contains a reasoning / thinking block.
63
-
-`fts_content` — flat text of EVERYTHING in the turn that's searchable. **Do not filter for "noise".** Include tool inputs, tool outputs, thinking blocks, system reminders. Holotype's pitch is forensic completeness; a search index that drops content violates it.
64
-
-`flags` — set of source-specific markers. `{"compaction"}` if this line is a context-compaction summary. `{"header"}` if it's a header / control / telemetry pseudo-record that shouldn't count toward `message_count` (the timestamps on header-flagged lines still update `first_timestamp`/`last_timestamp` and `session_metadata`/`usage` on them are still harvested — header just means "not a user-visible message").
65
-
-`usage` —*Optional, manifest v4.* When the host CLI exposes a per-turn token usage block (Claude Code's `message.usage`, Codex's `event_msg.token_count`), put the raw dict here. The manifest scanner aggregates `input_tokens`, `output_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens` / `cached_input_tokens` automatically. Leave `None` if your CLI doesn't report usage.
66
-
-`token_count_kind` —*Optional, manifest v4.*`"cumulative"` if the `usage` dict is a running total (Codex's `total_token_usage`), `"delta"` if it's per-turn billing (Claude Code's per-message `usage`), or `None` if no usage was attached. The aggregator prefers a final cumulative reading over summed deltas.
67
-
-`session_metadata` — *Optional, manifest v4.* For a header-flagged line that carries session-start metadata (Codex's `session_meta` carries the git block at session-start), put a dict here. Recognized keys: `cwd`, `cli_version`, `originator`, `model_provider`, `git_state: {commit, commit_short, branch, dirty, remote, captured_at}`. When `git_state.captured_at == "session-start"`, the manifest will prefer your header-derived state over the deposit-time probe — more accurate, because it was captured by the host CLI at the moment the session ran.
58
+
-`role`:`"user"` / `"assistant"` / `"tool_result"` / etc. Whatever your CLI calls these.
59
+
-`timestamp`: ISO-8601 string preferred. If your CLI uses Unix epoch, convert.
60
+
-`model`: the model ID for assistant turns. None for user turns. Don't put the *provider* name here (e.g. "openai", "anthropic"): those aren't model IDs and they pollute `manifest.models`.
61
+
-`has_tool_use`: True if this turn contains a tool call.
62
+
-`has_thinking`: True if this turn contains a reasoning / thinking block.
63
+
-`fts_content`: flat text of EVERYTHING in the turn that's searchable. **Do not filter for "noise".** Include tool inputs, tool outputs, thinking blocks, system reminders. Holotype's pitch is forensic completeness; a search index that drops content violates it.
64
+
-`flags`: set of source-specific markers. `{"compaction"}` if this line is a context-compaction summary. `{"header"}` if it's a header / control / telemetry pseudo-record that shouldn't count toward `message_count` (the timestamps on header-flagged lines still update `first_timestamp`/`last_timestamp` and `session_metadata`/`usage` on them are still harvested: header just means "not a user-visible message").
65
+
-`usage`:*Optional, manifest v4.* When the host CLI exposes a per-turn token usage block (Claude Code's `message.usage`, Codex's `event_msg.token_count`), put the raw dict here. The manifest scanner aggregates `input_tokens`, `output_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens` / `cached_input_tokens` automatically. Leave `None` if your CLI doesn't report usage.
66
+
-`token_count_kind`:*Optional, manifest v4.*`"cumulative"` if the `usage` dict is a running total (Codex's `total_token_usage`), `"delta"` if it's per-turn billing (Claude Code's per-message `usage`), or `None` if no usage was attached. The aggregator prefers a final cumulative reading over summed deltas.
67
+
-`session_metadata`: *Optional, manifest v4.* For a header-flagged line that carries session-start metadata (Codex's `session_meta` carries the git block at session-start), put a dict here. Recognized keys: `cwd`, `cli_version`, `originator`, `model_provider`, `git_state: {commit, commit_short, branch, dirty, remote, captured_at}`. When `git_state.captured_at == "session-start"`, the manifest will prefer your header-derived state over the deposit-time probe: more accurate, because it was captured by the host CLI at the moment the session ran.
68
68
69
69
5.**Decide your archive layout** in `DepositCandidate.archive_subpath`. The convention is `<source-name>/<project-or-date>/<session-id>/`. Examples:
Whatever's natural for your CLI's storage. The shape doesn't matter for correctness — `holotype/archive.py`'s `iter_all_sessions` walks arbitrary depth — but a layout that mirrors how the user thinks about their sessions makes manual inspection easier.
74
+
Whatever's natural for your CLI's storage. The shape doesn't matter for correctness (`holotype/archive.py`'s `iter_all_sessions` walks arbitrary depth) but a layout that mirrors how the user thinks about their sessions makes manual inspection easier.
75
75
76
76
## How to verify your implementation works
77
77
@@ -93,8 +93,8 @@ Before declaring done:
93
93
94
94
## What NOT to do
95
95
96
-
-**Do not** filter content for "noise" in `parse_line`. Tool calls, system reminders, hook outputs — all of it. The whole point of holotype is preserving what other tools throw away.
97
-
-**Do not** modify transcripts in place. Read-only with stable-mtime checks (see `read_with_stable_check` in `scripts/ingest.py` for the pattern). If your CLI's transcripts can be modified between reads, your source is automatically incompatible with holotype's hash-chain — flag it to the user and stop.
96
+
-**Do not** filter content for "noise" in `parse_line`. Tool calls, system reminders, hook outputs, all of it. The whole point of holotype is preserving what other tools throw away.
97
+
-**Do not** modify transcripts in place. Read-only with stable-mtime checks (see `read_with_stable_check` in `scripts/ingest.py` for the pattern). If your CLI's transcripts can be modified between reads, your source is automatically incompatible with holotype's hash-chain: flag it to the user and stop.
98
98
-**Do not** invent the on-disk schema. If your CLI doesn't document its transcript format and you can't infer it confidently from a real file, **defer**. A wrong source class will deposit garbage that passes hash checks but is unparseable to a reviewer 5 years from now.
99
99
-**Do not** widen `default_source_paths` to include directories with credentials. Codex's `~/.codex/` contains `auth.json` next to `sessions/`; we point at `sessions/` specifically. Do the analogous thing for your CLI.
100
100
-**Do not** push the archive to a remote without explicit user consent on this session, *every time*. The `--no-auto-push` discipline applies regardless of how the Source was added.
Copy file name to clipboardExpand all lines: docs/LINUX_SYSTEMD.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Linux: background ingest via user systemd
2
2
3
-
The macOS launchd background tick (`scripts/install-launchd.py`) catches long-running agent-CLI sessions whose `Stop` hook never fires. Linux users get the same behavior via a user-level systemd timer + service unit. This document is a reference template — not yet installed by a script.
3
+
The macOS launchd background tick (`scripts/install-launchd.py`) catches long-running agent-CLI sessions whose `Stop` hook never fires. Linux users get the same behavior via a user-level systemd timer + service unit. This document is a reference template, not yet installed by a script.
4
4
5
5
## Why a user-level service
6
6
@@ -21,7 +21,7 @@ Type=oneshot
21
21
# (e.g. /home/you/Git/holotype) and <ARCHIVE> with the absolute path
22
22
# returned by `cat ~/.config/holotype/archive-path`.
# Hold standard limits — the ingest is mostly I/O bound and should
24
+
# Hold standard limits: the ingest is mostly I/O bound and should
25
25
# never exceed normal user resource caps.
26
26
Nice=10
27
27
IOSchedulingClass=best-effort
@@ -87,7 +87,7 @@ systemctl --user daemon-reload
87
87
-**User must be logged in for the timer to fire** unless lingering is enabled (`loginctl enable-linger <user>`). For headless servers or auto-suspending laptops, enable lingering or accept that the timer pauses when you're logged out.
88
88
-**Concurrency safety is handled inside ingest.py** via `flock` at `<archive>/.holotype/.lock`. If you accidentally run two ingest cycles back-to-back, the second exits cleanly with "another ingest is running, exiting."
89
89
-**GPG-signed commits** (`--sign-commits`) need `GPG_TTY` available. For a systemd-driven invocation, that usually means using `gpg-agent` with `allow-loopback-pinentry` and an unlocked key. Test interactively first.
90
-
- This template doesn't push to a remote — same as the macOS launchd tick. Pushing is always an explicit user action.
90
+
- This template doesn't push to a remote, same as the macOS launchd tick. Pushing is always an explicit user action.
0 commit comments