From 9cf9e3883e8209193d4b82e765f436ad86d2011f Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Sun, 23 Aug 2026 21:05:34 -0700 Subject: [PATCH 01/43] fix(tooling): pyrightconfig stops hardcoding what the venv already knows. The site-packages extraPath spelled out .venv/lib/python3.12/... - a path that dies silently the day the venv rebuilds on another Python, same species as setup.sh's hand-written bootstrap list: a fact someone must remember to re-type. Replaced with venvPath+venv, which makes pyright resolve site-packages from the venv itself at whatever version it actually is. Also deleted the src/aipass/memory/.venv extraPath outright - that directory does not exist; a dead path in a config is a claim nobody is checking. Found tonight because Patrick's editor showed chromadb as a type error in a per-branch window: the root config never loads below the root, so the fix for HIM is open-at-repo-root or select the repo venv interpreter - but while proving the code was innocent (CLI pyright: 0 errors, chromadb 1.5.9 installed WITH py.typed) these two config lies surfaced. Verified after the change: memory storage handlers, devpulse apps, api modules - 0 errors, 0 warnings each. Selective commit on purpose: the tree carries @memory's live anchored-match dispatch and an unexplained TELEGRAM_PORT_MAP.md move, neither of which is this commit's to sweep. --- pyrightconfig.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pyrightconfig.json b/pyrightconfig.json index 706fb3320..3caedbfd4 100644 --- a/pyrightconfig.json +++ b/pyrightconfig.json @@ -1,9 +1,9 @@ { "extraPaths": [ - "src", - ".venv/lib/python3.12/site-packages", - "src/aipass/memory/.venv/lib/python3.12/site-packages" + "src" ], + "venvPath": ".", + "venv": ".venv", "pythonVersion": "3.10", "reportMissingImports": "error", "reportAttributeAccessIssue": "error", From e56d5de9f8e86b34d7fb19c0e2dccce18253b22e Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Sun, 23 Aug 2026 21:22:45 -0700 Subject: [PATCH 02/43] fix(memory): plan-ID matching anchors to a boundary, and the fourth site was a data-loss path (@memory's work, committed by devpulse). The symptom was a ranking oddity - DPLAN-0012 Hook Management pinning TDPLAN-0012 to the top at 100% - and the defect underneath was worse in exactly the way that matters: plan_label in source_file is an unanchored substring test, and DPLAN-0012 is a substring of TDPLAN-0012. FOUR sites carried it, not the three scoped. _check_plan and _get_by_source (both fetch), _pin_plan_id_matches (transitively through _get_by_source - its own _PLAN_ID_RE was already correct, the extraction was never broken, only the fetch), and the unnamed fourth: _delete_by_source, the IDENTICAL one-line predicate with DELETE behind it. delete_by_source DPLAN-0012 would have wiped TDPLAN-0012's vectors outright - no in-repo caller reaches it (JSON operation only), which is precisely why it would have been found by data loss and not by review. Fixed with its siblings; one site past the brief, flagged rather than smuggled. THE LIVE PROOF, against the real DB: DPLAN-0012 does not exist in the archive, and before the fix drone @memory verify DPLAN-0012 answered Vectorized (27 chunks) - all 27 borrowed from TDPLAN-0012. A verification door confirming a nonexistent plan on another plan's chunks means the coming 5000-plan sweep's failure mode was FALSE CONFIRMATIONS, not misses: the sweep would have under-reported the missing, which is the dangerous direction. After: DPLAN-0012 NOT vectorized, TDPLAN-0012 keeps its 27. The anchor is leading-only BY DESIGN: char before the label must not be alphanumeric, no trailing anchor because DPLAN-0012_watch is a legitimate prefix shape; the latent trailing variant needs 5-digit plan numbers, which do not exist below 10000. And it keys off the BOUNDARY, not the prefix list - so Patrick's pending TDPLAN/TPLAN naming ruling (todo 179) cannot re-open it either way; what would re-open it is a future prefix that is a suffix of another prefix, named here so nobody mints one. Behavior changes, stated not buried: empty pattern now matches NOTHING (was: every row - on the delete path that was a wipe-the-collection trigger); the two source-match operations are boundary match, not generic substring, and their docstrings now say so. Bar: 23 red-first tests (18 red before the fix, 2 deliberate regression guards), 5 mutations each caught by the right test including the rescan loop (a rejected first hit must not stop the scan - TDPLAN-0012_supersedes_DPLAN-0012 is a real shape), seedgo @memory 100%, branch suite 1109 passed / 5 skipped. chroma_subprocess.py 1.2.0 -> 1.3.0. Selective commit: the TELEGRAM_PORT_MAP.md move stays in the tree unexplained and unswept, awaiting Patrick. --- .../handlers/storage/chroma_subprocess.py | 43 ++- .../memory/tests/test_chroma_source_match.py | 260 ++++++++++++++++++ 2 files changed, 296 insertions(+), 7 deletions(-) create mode 100644 src/aipass/memory/tests/test_chroma_source_match.py diff --git a/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py b/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py index 9efd166ed..05e9f380a 100755 --- a/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py +++ b/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py @@ -1,9 +1,9 @@ # =================== AIPass ==================== # Name: chroma_subprocess.py # Description: ChromaDB Subprocess Handler -# Version: 1.2.0 +# Version: 1.3.0 # Created: 2025-11-27 -# Modified: 2026-03-12 +# Modified: 2026-08-23 # ============================================= """ @@ -113,6 +113,35 @@ def _list_collections(db_path=None): return {"success": True, "collections": names, "count": len(names)} +def _source_matches(source, pattern): + """Whether pattern occurs in source at a non-alphanumeric boundary. + + A plain `pattern in source` test lets a label match inside a longer one -- + DPLAN-0012 answers to a TDPLAN-0012 filename, so verification miscounts and + the search pin promotes the wrong plan. A hit only counts when the label + starts the string or follows a non-alphanumeric character. + + Args: + source: The source_file value from a chunk's metadata + pattern: The label to look for (e.g. "DPLAN-0012") + + Returns: + True if pattern occurs at a boundary in source + """ + if not source or not pattern: + return False + + idx = source.find(pattern) + while idx != -1: + # Keep scanning past a rejected hit: the same source may carry the + # label again at a real boundary ("TDPLAN-0012_supersedes_DPLAN-0012"). + if idx == 0 or not source[idx - 1].isalnum(): + return True + idx = source.find(pattern, idx + 1) + + return False + + def _check_plan(plan_label, db_path=None): """Check if a plan has been vectorized in ChromaDB. @@ -145,7 +174,7 @@ def _check_plan(plan_label, db_path=None): match_count = 0 for metadata in metadatas: source_file = metadata.get("source_file", "") - if plan_label in source_file: + if _source_matches(source_file, plan_label): match_count += 1 matching_files.add(source_file) @@ -157,7 +186,7 @@ def _get_by_source(collection_name, source_pattern, n_results=5, db_path=None): Args: collection_name: Name of the ChromaDB collection - source_pattern: Substring to match in source_file metadata + source_pattern: Label to match at a boundary in source_file metadata n_results: Maximum number of results to return db_path: Optional path to Chroma database @@ -176,7 +205,7 @@ def _get_by_source(collection_name, source_pattern, n_results=5, db_path=None): matches = [] for i, meta in enumerate(result.get("metadatas", [])): source = meta.get("source_file", "") - if source_pattern in source: + if _source_matches(source, source_pattern): matches.append( { "collection": collection_name, @@ -197,7 +226,7 @@ def _delete_by_source(collection_name, source_pattern, db_path=None): Args: collection_name: Name of the ChromaDB collection - source_pattern: Substring to match in source_file metadata + source_pattern: Label to match at a boundary in source_file metadata db_path: Optional path to Chroma database Returns: @@ -215,7 +244,7 @@ def _delete_by_source(collection_name, source_pattern, db_path=None): ids_to_delete = [] for i, meta in enumerate(result.get("metadatas", [])): source = meta.get("source_file", "") - if source_pattern in source: + if _source_matches(source, source_pattern): ids_to_delete.append(result["ids"][i]) if not ids_to_delete: diff --git a/src/aipass/memory/tests/test_chroma_source_match.py b/src/aipass/memory/tests/test_chroma_source_match.py new file mode 100644 index 000000000..bfb595214 --- /dev/null +++ b/src/aipass/memory/tests/test_chroma_source_match.py @@ -0,0 +1,260 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: tests/test_chroma_source_match.py +# Date: 2026-08-23 +# Version: 1.0.0 +# Category: memory/tests +# ============================================= + +"""Tests for source_file matching in the ChromaDB subprocess handler. + +Covers: _source_matches, _check_plan, _get_by_source, _delete_by_source + +The defect these pin: `plan_label in source_file` is an unanchored substring +test, so DPLAN-0012 matches a TDPLAN-0012 filename and the exact-match pin +promotes the wrong plan. A label only counts when the character before it is +not alphanumeric. + +All tests use a fake collection -- no live ChromaDB. +""" + +import pytest + +from aipass.memory.apps.handlers.storage import chroma_subprocess + + +# --------------------------------------------------------------------------- +# Fake ChromaDB collection +# --------------------------------------------------------------------------- + + +class _FakeCollection: + """Minimal stand-in for a Chroma collection backed by source_file names.""" + + def __init__(self, source_files): + self._sources = list(source_files) + self.deleted = [] + + def get(self, include=None): + return { + "metadatas": [{"source_file": s} for s in self._sources], + "documents": [f"body of {s}" for s in self._sources], + "ids": [f"id_{i}" for i in range(len(self._sources))], + } + + def delete(self, ids=None): + self.deleted.extend(ids or []) + + +class _FakeClient: + def __init__(self, collection): + self._collection = collection + + def get_collection(self, name, embedding_function=None): + return self._collection + + +@pytest.fixture +def fake_collection(monkeypatch): + """Install a fake client and return a factory for seeding source files.""" + + def _install(source_files): + collection = _FakeCollection(source_files) + monkeypatch.setattr(chroma_subprocess, "_get_client", lambda db_path=None: _FakeClient(collection)) + return collection + + return _install + + +# --------------------------------------------------------------------------- +# 1. _source_matches() -- the anchored predicate +# --------------------------------------------------------------------------- + + +class TestSourceMatches: + """The boundary rule: the character before the label must not be alphanumeric.""" + + def test_label_at_start_of_filename_matches(self): + assert chroma_subprocess._source_matches("DPLAN-0012_hook_management.md", "DPLAN-0012") + + @pytest.mark.parametrize("prefix", ["_", "-", " ", "/", "."]) + def test_non_alphanumeric_predecessor_matches(self, prefix): + source = f"archive{prefix}DPLAN-0012_hook.md" + assert chroma_subprocess._source_matches(source, "DPLAN-0012") + + def test_alphabetic_predecessor_is_rejected(self): + """The reported collision: TDPLAN-0012 must not answer to DPLAN-0012.""" + assert not chroma_subprocess._source_matches("TDPLAN-0012_hook_management.md", "DPLAN-0012") + + def test_digit_predecessor_is_rejected(self): + assert not chroma_subprocess._source_matches("9DPLAN-0012_hook.md", "DPLAN-0012") + + def test_later_anchored_occurrence_still_matches(self): + """A rejected first hit must not stop the scan -- keep looking.""" + source = "TDPLAN-0012_supersedes_DPLAN-0012.md" + assert chroma_subprocess._source_matches(source, "DPLAN-0012") + + def test_absent_label_does_not_match(self): + assert not chroma_subprocess._source_matches("FPLAN-0449_watchdog.md", "DPLAN-0012") + + def test_empty_pattern_matches_nothing(self): + """Guard the destructive path: an empty pattern must not select every row.""" + assert not chroma_subprocess._source_matches("DPLAN-0012_hook.md", "") + + def test_missing_source_does_not_match(self): + assert not chroma_subprocess._source_matches("", "DPLAN-0012") + + +# --------------------------------------------------------------------------- +# 2. _check_plan() -- vectorization verification +# --------------------------------------------------------------------------- + + +class TestCheckPlanAnchoring: + """is_plan_vectorized() must not count another plan's chunks as its own.""" + + def test_does_not_count_longer_prefix_family(self, fake_collection): + fake_collection(["TDPLAN-0012_hook.md", "TDPLAN-0012_hook.md"]) + + result = chroma_subprocess._check_plan("DPLAN-0012") + + assert result["success"] is True + assert result["found"] is False + assert result["count"] == 0 + assert result["source_files"] == [] + + def test_counts_only_its_own_chunks_in_a_mixed_collection(self, fake_collection): + fake_collection( + [ + "DPLAN-0012_hook_management.md", + "TDPLAN-0012_hook_management.md", + "DPLAN-0012_hook_management.md", + ] + ) + + result = chroma_subprocess._check_plan("DPLAN-0012") + + assert result["found"] is True + assert result["count"] == 2 + assert result["source_files"] == ["DPLAN-0012_hook_management.md"] + + def test_the_longer_label_still_finds_itself(self, fake_collection): + fake_collection(["TDPLAN-0012_hook.md", "DPLAN-0012_hook.md"]) + + result = chroma_subprocess._check_plan("TDPLAN-0012") + + assert result["found"] is True + assert result["count"] == 1 + assert result["source_files"] == ["TDPLAN-0012_hook.md"] + + +# --------------------------------------------------------------------------- +# 3. _get_by_source() -- the exact-match pin's data source +# --------------------------------------------------------------------------- + + +class TestGetBySourceAnchoring: + """The search pin fetches through this -- a wrong row here is pinned at 100%.""" + + def test_skips_longer_prefix_family(self, fake_collection): + fake_collection(["TDPLAN-0012_hook.md"]) + + result = chroma_subprocess._get_by_source("flow_plans", "DPLAN-0012") + + assert result["success"] is True + assert result["count"] == 0 + assert result["results"] == [] + + def test_returns_only_the_requested_plan(self, fake_collection): + fake_collection(["TDPLAN-0012_hook.md", "DPLAN-0012_hook.md"]) + + result = chroma_subprocess._get_by_source("flow_plans", "DPLAN-0012") + + assert result["count"] == 1 + assert result["results"][0]["metadata"]["source_file"] == "DPLAN-0012_hook.md" + + def test_n_results_still_caps_matches(self, fake_collection): + fake_collection(["DPLAN-0012_a.md", "DPLAN-0012_b.md", "DPLAN-0012_c.md"]) + + result = chroma_subprocess._get_by_source("flow_plans", "DPLAN-0012", n_results=2) + + assert result["count"] == 2 + + +# --------------------------------------------------------------------------- +# 4. _delete_by_source() -- same predicate, destructive +# --------------------------------------------------------------------------- + + +class TestDeleteBySourceAnchoring: + """Unanchored matching here deletes another plan's vectors outright.""" + + def test_does_not_delete_longer_prefix_family(self, fake_collection): + collection = fake_collection(["TDPLAN-0012_hook.md"]) + + result = chroma_subprocess._delete_by_source("flow_plans", "DPLAN-0012") + + assert result["success"] is True + assert result["deleted"] == 0 + assert collection.deleted == [] + + def test_deletes_only_the_requested_plan(self, fake_collection): + collection = fake_collection(["TDPLAN-0012_hook.md", "DPLAN-0012_hook.md"]) + + result = chroma_subprocess._delete_by_source("flow_plans", "DPLAN-0012") + + assert result["deleted"] == 1 + assert collection.deleted == ["id_1"] + + +# --------------------------------------------------------------------------- +# 5. The search pin -- both layers composed +# --------------------------------------------------------------------------- + + +class TestPinComposition: + """The pin has two layers: extract the label, then fetch by that label. + + The extractor was already correct (\\b keeps TDPLAN whole); the fetch was + not. These pin the pair so a regression in either surfaces here. + """ + + def _query_executor(self): + from aipass.memory.apps.handlers.search import query_executor + + return query_executor + + def test_extractor_keeps_the_longer_prefix_whole(self): + qe = self._query_executor() + assert qe._extract_plan_id("TDPLAN-0012 Hook Management") == "TDPLAN-0012" + assert qe._extract_plan_id("DPLAN-0012 Hook Management") == "DPLAN-0012" + + def test_pin_passes_the_label_through_unaltered(self, monkeypatch): + """Whatever the extractor produced is what the fetch anchors on.""" + qe = self._query_executor() + seen = {} + + def _capture(plan_id, n_results): + seen["plan_id"] = plan_id + return [] + + monkeypatch.setattr(qe, "_fetch_plan_by_metadata", _capture) + qe._pin_plan_id_matches("DPLAN-0012 Hook Management", [], 5) + + assert seen["plan_id"] == "DPLAN-0012" + + def test_wrong_family_plan_is_not_pinned(self, monkeypatch): + """End to end: a TDPLAN-only collection yields nothing to pin for DPLAN.""" + qe = self._query_executor() + collection = _FakeCollection(["TDPLAN-0012_hook.md"]) + monkeypatch.setattr(chroma_subprocess, "_get_client", lambda db_path=None: _FakeClient(collection)) + + def _through_handler(plan_id, n_results): + return chroma_subprocess._get_by_source("flow_plans", plan_id, n_results)["results"] + + monkeypatch.setattr(qe, "_fetch_plan_by_metadata", _through_handler) + existing = [{"id": "real", "similarity": 0.86}] + pinned = qe._pin_plan_id_matches("DPLAN-0012 Hook Management", existing, 5) + + assert pinned == existing + assert not any(r.get("similarity") == 1.0 for r in pinned) From 660ab6928709c32c601d980ecb74bc0775a02853 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 08:49:22 -0700 Subject: [PATCH 03/43] fix(ai_mail+memory): the purge seam stops eating mail - exit 0 was never evidence of archive. Two halves of ONE bug, committed together because each is meaningless without the other. THE LOSS PATH: purge's _vectorize_emails shells out to @memory's chroma_subprocess with operation vectorize_and_store - an operation that DID NOT EXIST. The handler answers a bad request honestly on stdout (success:false, error: Unknown operation) and exits 0 BY DESIGN - the subprocess ran fine, it was the request that was wrong. purge tested only returncode != 0, so the refusal sailed past as success and _purge_files unlinked the originals it never archived. Four months of purged fleet mail, unrecoverable. ai_mail's half: parse the reply - unreadable stdout is not evidence of success (fail carrying the raw fragment), success:false is a refusal (fail carrying the handler's reason), and nothing is deleted on either. memory's half: chroma_subprocess.py 1.3.0 -> 1.4.0 grows the missing vectorize_and_store operation - text-in vectorization, callers hand texts+metadatas straight in. Bar: test_purge.py +4 red-first tests (TestVectorizationFailureIsNotSuccess: unknown-op-at-exit-0 is failure, nothing deleted on refusal, unparseable stdout is failure, a real success still succeeds), test_chroma_vectorize.py 10 tests on the new op - 25 passed together this morning. ai_mail README documents the seam. Rides along, explained not smuggled: TELEGRAM_PORT_MAP.md moves devpulse -> skills/lib/telegram (Patrick's move, 08-24 - the port map lives beside the skill it maps). Both fixes are their senders' work (@ai_mail, @memory), reviewed green by them, committed by devpulse. --- src/aipass/ai_mail/README.md | 4 +- .../ai_mail/apps/handlers/email/purge.py | 33 ++++ src/aipass/ai_mail/tests/test_purge.py | 100 ++++++++++ .../handlers/storage/chroma_subprocess.py | 92 ++++++++- .../memory/tests/test_chroma_vectorize.py | 176 ++++++++++++++++++ .../lib/telegram}/TELEGRAM_PORT_MAP.md | 0 6 files changed, 402 insertions(+), 3 deletions(-) create mode 100644 src/aipass/memory/tests/test_chroma_vectorize.py rename src/aipass/{devpulse => skills/lib/telegram}/TELEGRAM_PORT_MAP.md (100%) diff --git a/src/aipass/ai_mail/README.md b/src/aipass/ai_mail/README.md index 6482b7396..71d9433a8 100644 --- a/src/aipass/ai_mail/README.md +++ b/src/aipass/ai_mail/README.md @@ -9,7 +9,7 @@ --- -**Status:** Operational | **Seedgo:** 100% (99% with every bypass rule off) | **Tests:** 1319 pass (1315 + 4 live-hygiene skips on a fresh checkout) | **Battle Tested:** S62 +**Status:** Operational | **Seedgo:** 100% (99% with every bypass rule off) | **Tests:** 1323 pass (1319 + 4 live-hygiene skips on a fresh checkout) | **Battle Tested:** S62 ## Quick Start @@ -754,7 +754,7 @@ ai_mail/ │ ├── paths.py # Shared find_repo_root() utility │ ├── notify.py # Notification feed writer (JSONL, BAUD reads) │ └── central_writer.py # Central inbox stats aggregation -└── tests/ # 1319 tests across 46 test files +└── tests/ # 1323 tests across 46 test files ├── conftest.py # Shared fixtures (mock_logger, mock_json_handler) ├── test_daemon.py # Daemon config, state, kill switch, dispatch check ├── test_dispatch_monitor.py # Monitor safety features, env stripping diff --git a/src/aipass/ai_mail/apps/handlers/email/purge.py b/src/aipass/ai_mail/apps/handlers/email/purge.py index 39f3ace2c..ae5790805 100644 --- a/src/aipass/ai_mail/apps/handlers/email/purge.py +++ b/src/aipass/ai_mail/apps/handlers/email/purge.py @@ -226,6 +226,17 @@ def _vectorize_emails(emails: List[Dict[str, Any]], folder_type: str) -> Dict[st ) # Call @memory vectorization via subprocess (handler independence) + # KNOWN GAP, LEFT VISIBLE ON PURPOSE — DO NOT "FIX" THIS BY RENAMING IT. + # @memory's handler has no `vectorize_and_store` operation and there is + # no evidence it ever did, so this call has always been refused. Renaming + # to `store_vectors` does NOT make it work: that operation wants + # `embeddings` (already-encoded vectors) plus `documents`, and what is + # sent here is raw `texts`. Encoding them first would mean THIS branch + # picking an embedding model, and a collection whose vectors come from + # two different models is silently unsearchable — that choice belongs to + # the branch that owns the store. Requested from @memory as a text-in + # operation on their surface; until it lands, the check above makes this + # fail LOUDLY and preserve the mail instead of deleting it. input_data = { "operation": "vectorize_and_store", "branch": "AI_MAIL", @@ -245,6 +256,28 @@ def _vectorize_emails(emails: List[Dict[str, Any]], folder_type: str) -> Dict[st if result.returncode != 0: return {"success": False, "error": result.stderr or "Storage failed"} + # EXIT 0 MEANS THE PROCESS RAN, NOT THAT THE REQUEST SUCCEEDED. @memory's + # handler answers a bad request on stdout — {"success": false, "error": + # ...} — and exits 0, correctly: the subprocess did its job, it was the + # ASK that was wrong. Testing only returncode read that refusal as a + # store, and _purge_files then unlinked the originals under the comment + # "data is safely in @memory". It was not. Found by @memory (9da1ba52, + # 2026-08-23) from their side of the wire; the database agrees — not one + # email collection exists among 37. + try: + reply = json.loads(result.stdout) + except (json.JSONDecodeError, TypeError, ValueError) as e: + # Unreadable output is not evidence of success. Guessing "probably + # fine" here is the original defect wearing a different hat, and the + # cost of guessing wrong is deleted mail. + logger.warning("[purge] Unreadable vectorization reply for %s: %s", folder_type, e) + return {"success": False, "error": f"Unreadable reply from @memory: {result.stdout[:200]!r}"} + + if not isinstance(reply, dict) or not reply.get("success"): + error = (reply or {}).get("error") if isinstance(reply, dict) else None + logger.warning("[purge] @memory refused vectorization for %s: %s", folder_type, error) + return {"success": False, "error": str(error or "Vectorization refused without a reason")} + return {"success": True, "count": len(texts)} except subprocess.TimeoutExpired as e: diff --git a/src/aipass/ai_mail/tests/test_purge.py b/src/aipass/ai_mail/tests/test_purge.py index b2df30449..3ef9d49fb 100644 --- a/src/aipass/ai_mail/tests/test_purge.py +++ b/src/aipass/ai_mail/tests/test_purge.py @@ -200,3 +200,103 @@ def test_run_purge_failure_propagates(tmp_path, monkeypatch): assert result["success"] is False assert result["sent"]["success"] is False + + +# ---- The subprocess seam -------------------------------------- + + +class TestVectorizationFailureIsNotSuccess: + """Reported by @memory (9da1ba52, 2026-08-23) and verified here before fixing. + + purge sent ``"operation": "vectorize_and_store"``. @memory's chroma handler + accepts six operations and that is not one of them; there is no evidence it + ever was. Their handler answers an unknown operation on STDOUT with + ``{"success": false, ...}`` and EXITS 0 — a deliberate choice, since the + subprocess ran fine, it was the request that was wrong. purge tested only + ``returncode != 0``, so the refusal sailed past and it returned success. + + THAT IS NOT A COSMETIC BUG. ``_purge_files`` gates deletion on this result + and then calls ``file_path.unlink()`` under the comment "data is safely in + @memory". It never was. Reproduced live against the real handler: + + $ echo '{"operation":"vectorize_and_store",...}' | python3 chroma_subprocess.py + {"success": false, "error": "Unknown operation: vectorize_and_store"} + EXIT CODE: 0 + + And confirmed from the other side: of 37 live collections, ai_mail_observations + and ai_mail_local exist (the .trinity rollover, working), and no email + collection exists at all. + + WHY EVERY EXISTING TEST IN THIS FILE MISSED IT: they all monkeypatch + ``_vectorize_emails`` wholesale, so the seam between purge and the actual + subprocess had no coverage at any point. The bug lived exactly where the + mock began — the same shape as the dispatch-register phantoms this branch + hit on 08-22, where a writer and its first consumer were built separately + and neither suite covered the join. + """ + + @staticmethod + def _handler_reply(stdout: str, returncode: int = 0): + """Stand in for subprocess.run with a real handler response.""" + + class _Result: + def __init__(self): + self.returncode = returncode + self.stdout = stdout + self.stderr = "" + + return lambda *a, **k: _Result() + + def test_an_unknown_operation_is_a_failure_even_though_exit_is_zero(self, monkeypatch): + """The exact live response, byte for byte.""" + monkeypatch.setattr( + purge_mod.subprocess, + "run", + self._handler_reply('{"success": false, "error": "Unknown operation: vectorize_and_store"}'), + ) + + result = purge_mod._vectorize_emails([{"subject": "s", "message": "m"}], "sent") + + assert result["success"] is False, "exit 0 with success:false is a REFUSAL, not a store" + assert "Unknown operation" in str(result.get("error", "")), ( + f"the handler's own reason must survive to the caller. Got: {result}" + ) + + def test_nothing_is_deleted_when_the_handler_refuses(self, tmp_path, monkeypatch): + """The consequence that matters: originals must outlive a failed archive.""" + folder = tmp_path / "sent" + folder.mkdir() + _populate_folder(folder, 15) + before = len(list(folder.glob("*.json"))) + + monkeypatch.setattr( + purge_mod.subprocess, + "run", + self._handler_reply('{"success": false, "error": "Unknown operation: vectorize_and_store"}'), + ) + + result = purge_sent_folder(tmp_path) + + assert result["success"] is False + assert len(list(folder.glob("*.json"))) == before, "purge deleted originals it never archived" + + def test_unparseable_stdout_is_not_treated_as_success(self, monkeypatch): + """A handler that returns garbage has not stored anything either. + + Guessing "probably fine" from unreadable output is how the original + defect would come back wearing a different hat. + """ + monkeypatch.setattr(purge_mod.subprocess, "run", self._handler_reply("not json at all")) + + result = purge_mod._vectorize_emails([{"subject": "s", "message": "m"}], "sent") + + assert result["success"] is False + + def test_a_real_success_still_succeeds(self, monkeypatch): + """The fix must not make a working store look broken.""" + monkeypatch.setattr(purge_mod.subprocess, "run", self._handler_reply('{"success": true, "stored": 1}')) + + result = purge_mod._vectorize_emails([{"subject": "s", "message": "m"}], "sent") + + assert result["success"] is True + assert result["count"] == 1 diff --git a/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py b/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py index 05e9f380a..0a6b0a93e 100755 --- a/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py +++ b/src/aipass/memory/apps/handlers/storage/chroma_subprocess.py @@ -1,7 +1,7 @@ # =================== AIPass ==================== # Name: chroma_subprocess.py # Description: ChromaDB Subprocess Handler -# Version: 1.3.0 +# Version: 1.4.0 # Created: 2025-11-27 # Modified: 2026-08-23 # ============================================= @@ -20,6 +20,7 @@ import sys import json +import subprocess import logging import hashlib from pathlib import Path @@ -35,6 +36,12 @@ _MEMORY_ROOT = Path(__file__).resolve().parents[3] _DEFAULT_DB_PATH = _MEMORY_ROOT / ".chroma" +# Sibling embedder script -- same venv, invoked by path. Encoding lives behind +# this handler on purpose: a caller that picks its own embedding model can put +# vectors from two models in one collection, which fails silently rather than +# loudly (no error, just wrong neighbours). The store owns the model choice. +_EMBED_SCRIPT = Path(__file__).resolve().parent.parent / "vector" / "embed_subprocess.py" + # Singleton clients per path _clients = {} @@ -105,6 +112,81 @@ def _store_vectors(branch, memory_type, embeddings, documents, metadatas, db_pat } +def _vectorize_and_store(branch, memory_type, texts, metadatas, db_path=None): + """Encode raw texts and store them -- the text-in entry point. + + Callers send text and get a verdict; they never pick an embedding model. + Every failure returns success=False with a reason: a caller that deletes its + originals on the strength of this answer must be able to trust it. + + Args: + branch: Owning branch name (e.g. "AI_MAIL") + memory_type: Collection suffix (e.g. "email_sent") + texts: Raw strings to encode and store + metadatas: One metadata dict per text, same order + db_path: Optional path to Chroma database + + Returns: + Dict with success, collection, count -- or success=False and an error + """ + if not branch or not memory_type: + return {"success": False, "error": "branch and memory_type are required"} + + texts = texts or [] + metadatas = metadatas or [] + + if not texts: + return {"success": True, "count": 0, "message": "No texts to store"} + + # zip() in the store path would truncate a mismatch silently and hand a row + # someone else's provenance. Refuse instead. + if len(metadatas) != len(texts): + return {"success": False, "error": f"metadatas ({len(metadatas)}) does not match texts ({len(texts)})"} + + timeout = max(30, len(texts) * 3) + try: + completed = subprocess.run( + [sys.executable, str(_EMBED_SCRIPT)], + input=json.dumps({"texts": texts}), + capture_output=True, + text=True, + timeout=timeout, + ) + except subprocess.TimeoutExpired: + logger.warning(f"[chroma_subprocess] Embedding timed out after {timeout}s for {len(texts)} texts") + return {"success": False, "error": f"Embedding timed out after {timeout}s"} + except Exception as e: + logger.warning(f"[chroma_subprocess] Embed subprocess failed: {e}") + return {"success": False, "error": f"Embedding subprocess failed: {e}"} + + if completed.returncode != 0: + return {"success": False, "error": completed.stderr or "Embedding failed"} + + try: + embed_reply = json.loads(completed.stdout) + except (json.JSONDecodeError, TypeError, ValueError) as e: + # Unreadable output is not evidence of success. + logger.warning(f"[chroma_subprocess] Unreadable embedder reply: {e}") + return {"success": False, "error": f"Unreadable embedder reply: {e}"} + + if not isinstance(embed_reply, dict) or not embed_reply.get("success"): + error = embed_reply.get("error") if isinstance(embed_reply, dict) else None + return {"success": False, "error": str(error or "Embedding refused without a reason")} + + embeddings = embed_reply.get("embeddings") or [] + if len(embeddings) != len(texts): + return {"success": False, "error": f"embeddings ({len(embeddings)}) does not match texts ({len(texts)})"} + + return _store_vectors( + branch=branch, + memory_type=memory_type, + embeddings=embeddings, + documents=texts, + metadatas=metadatas, + db_path=db_path, + ) + + def _list_collections(db_path=None): """List all collections.""" client = _get_client(db_path) @@ -322,6 +404,14 @@ def main(): metadatas=input_data.get("metadatas"), db_path=input_data.get("db_path"), ) + elif operation == "vectorize_and_store": + result = _vectorize_and_store( + branch=input_data.get("branch"), + memory_type=input_data.get("memory_type"), + texts=input_data.get("texts"), + metadatas=input_data.get("metadatas"), + db_path=input_data.get("db_path"), + ) elif operation == "list_collections": result = _list_collections(db_path=input_data.get("db_path")) elif operation == "search_vectors": diff --git a/src/aipass/memory/tests/test_chroma_vectorize.py b/src/aipass/memory/tests/test_chroma_vectorize.py new file mode 100644 index 000000000..543ba7fb0 --- /dev/null +++ b/src/aipass/memory/tests/test_chroma_vectorize.py @@ -0,0 +1,176 @@ +# ===================AIPASS==================== +# META DATA HEADER +# Name: tests/test_chroma_vectorize.py +# Date: 2026-08-23 +# Version: 1.0.0 +# Category: memory/tests +# ============================================= + +"""Tests for the text-in vectorize_and_store operation. + +Covers: _vectorize_and_store + +The gap this closes: callers had to pre-encode their own texts, which meant each +caller picked an embedding model. Two callers picking differently put vectors +from two models in one collection, which is silently unsearchable. The model +choice belongs to the branch that owns the store. + +All tests stub the embedder subprocess -- no live model load or ChromaDB. +""" + +import json + +from aipass.memory.apps.handlers.storage import chroma_subprocess + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +class _FakeCompleted: + def __init__(self, stdout, returncode=0, stderr=""): + self.stdout = stdout + self.returncode = returncode + self.stderr = stderr + + +def _stub_embedder(monkeypatch, payload, returncode=0, stderr=""): + """Stand in for the embed subprocess; capture what it was asked to encode.""" + seen = {} + + def _run(cmd, input=None, capture_output=None, text=None, timeout=None): + seen["texts"] = json.loads(input)["texts"] + seen["timeout"] = timeout + return _FakeCompleted(json.dumps(payload) if payload is not None else "", returncode, stderr) + + monkeypatch.setattr(chroma_subprocess.subprocess, "run", _run) + return seen + + +def _stub_store(monkeypatch): + """Capture what reached the storage layer.""" + seen = {} + + def _store(branch, memory_type, embeddings, documents, metadatas, db_path=None): + seen.update( + branch=branch, memory_type=memory_type, embeddings=embeddings, documents=documents, metadatas=metadatas + ) + return {"success": True, "collection": f"{branch.lower()}_{memory_type.lower()}", "count": len(documents)} + + monkeypatch.setattr(chroma_subprocess, "_store_vectors", _store) + return seen + + +# --------------------------------------------------------------------------- +# 1. The happy path +# --------------------------------------------------------------------------- + + +class TestVectorizeAndStore: + def test_encodes_then_stores(self, monkeypatch): + embed = _stub_embedder(monkeypatch, {"success": True, "embeddings": [[0.1, 0.2], [0.3, 0.4]]}) + store = _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store( + branch="AI_MAIL", + memory_type="email_sent", + texts=["first mail", "second mail"], + metadatas=[{"subject": "a"}, {"subject": "b"}], + ) + + assert result["success"] is True + assert embed["texts"] == ["first mail", "second mail"] + assert store["embeddings"] == [[0.1, 0.2], [0.3, 0.4]] + assert store["documents"] == ["first mail", "second mail"] + assert store["branch"] == "AI_MAIL" + + def test_empty_texts_is_a_no_op_success(self, monkeypatch): + _stub_store(monkeypatch) + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", [], []) + assert result["success"] is True + assert result["count"] == 0 + + def test_embedder_timeout_scales_with_volume(self, monkeypatch): + embed = _stub_embedder(monkeypatch, {"success": True, "embeddings": [[0.1]] * 50}) + _stub_store(monkeypatch) + + chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["t"] * 50, [{}] * 50) + + assert embed["timeout"] >= 50 + + +# --------------------------------------------------------------------------- +# 2. Failing loud -- the whole point of the operation +# --------------------------------------------------------------------------- + + +class TestVectorizeFailsLoud: + """Every failure returns success:false with a reason. Nothing is guessed.""" + + def test_embedder_nonzero_exit_is_reported(self, monkeypatch): + _stub_embedder(monkeypatch, None, returncode=1, stderr="fastembed exploded") + _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["x"], [{}]) + + assert result["success"] is False + assert "fastembed exploded" in result["error"] + + def test_embedder_refusal_is_reported(self, monkeypatch): + _stub_embedder(monkeypatch, {"success": False, "error": "model missing"}) + _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["x"], [{}]) + + assert result["success"] is False + assert "model missing" in result["error"] + + def test_unreadable_embedder_output_is_not_success(self, monkeypatch): + _stub_embedder(monkeypatch, None) + _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["x"], [{}]) + + assert result["success"] is False + + def test_embedding_count_mismatch_is_refused(self, monkeypatch): + """Two texts in, one vector out -- storing that would misalign every row.""" + _stub_embedder(monkeypatch, {"success": True, "embeddings": [[0.1]]}) + store = _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["a", "b"], [{}, {}]) + + assert result["success"] is False + assert store == {} + + def test_metadata_count_mismatch_is_refused(self, monkeypatch): + """zip() would truncate silently and drop a row's provenance.""" + _stub_embedder(monkeypatch, {"success": True, "embeddings": [[0.1], [0.2]]}) + store = _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("AI_MAIL", "email_sent", ["a", "b"], [{}]) + + assert result["success"] is False + assert store == {} + + def test_missing_branch_is_refused(self, monkeypatch): + _stub_embedder(monkeypatch, {"success": True, "embeddings": [[0.1]]}) + _stub_store(monkeypatch) + + result = chroma_subprocess._vectorize_and_store("", "email_sent", ["a"], [{}]) + + assert result["success"] is False + + +# --------------------------------------------------------------------------- +# 3. Reachable as an operation -- the wire ai_mail actually calls +# --------------------------------------------------------------------------- + + +class TestOperationIsRouted: + def test_vectorize_and_store_is_a_known_operation(self): + source = chroma_subprocess.__file__ + with open(source, encoding="utf-8") as fh: + body = fh.read() + assert 'operation == "vectorize_and_store"' in body diff --git a/src/aipass/devpulse/TELEGRAM_PORT_MAP.md b/src/aipass/skills/lib/telegram/TELEGRAM_PORT_MAP.md similarity index 100% rename from src/aipass/devpulse/TELEGRAM_PORT_MAP.md rename to src/aipass/skills/lib/telegram/TELEGRAM_PORT_MAP.md From d0ce2ce45eab3004e8cde748b92a6ab9f209f71f Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 17:39:55 -0700 Subject: [PATCH 04/43] feat(spawn): the passport number becomes real - citizenship.citizen_id, stamped fleet-wide and minted once at birth (Patrick's ruling 08-24, @spawn's build, committed by devpulse). THE RULING: a passport's number must be UNIQUE per citizen. What passports carried was citizenship.registry_id - the id of the REGISTRY holding the citizen (AIPASS_REGISTRY metadata.id 7087bb93 for all 18 core, each project registry's own id for its citizens), correct as the branch-registry lock but shared BY DESIGN - and BAUD rendered it as Passport no., which is how a fleet audit briefly read as 18 duplicate passports. The unique per-citizen id existed all along in every registry's branches[] entries and simply never reached the passport file. THE BUILD: citizen_id stamped into all 23 live passports (passports are untracked - this commit carries the MACHINERY), copied from each citizen's own branches[] row, verified byte-identical elsewhere, idempotent re-run reports 23 correct / 0 to write. THE REAL WORK was where the id is MINTED: registries minted the citizen UUID in add_to_registry at create step 8 while the passport writes at step 1 - stamping from separate mints would issue TWO uuids per citizen, a passport number matching no registry row. The mint moved into _spawn_agent, one value feeds both writers; add_to_registry takes an optional citizen_id and mints only for adoption. Templates: both classes now stamp citizen_id: {{CITIZEN_ID}} at birth, template registries regenerated. Speakeasy skipped on principle - its registry has zero branches[] entries, so a stamp would fabricate provenance; Patrick ruled it stays legacy, TBD. Known and NOT fixed here, spawn greenlit separately: a brand-new external project's first citizen gets AIPass's registry credential because load_registry's default schema mints no id. Bar: tests/test_citizen_id.py new, 7 tests incl. mutation guards (aliasing CITIZEN_ID to REGISTRY_ID fails 5 by name), 495 passed / 1 skipped repo-root, seedgo @spawn 100%. Spawn's prompt file-counts corrected (46/18 -> 50/17). BAUD renders the pair as Passport no. / Branch reg no. in its own repo. --- .../spawn/.aipass/aipass_local_prompt.md | 4 +- src/aipass/spawn/README.md | 11 +- .../spawn/apps/handlers/placeholders.py | 10 +- src/aipass/spawn/apps/handlers/registry.py | 19 ++- src/aipass/spawn/apps/modules/core.py | 9 ++ .../.spawn/.template_registry.json | 4 +- .../aipass_framework/.trinity/passport.json | 7 +- .../.spawn/.template_registry.json | 4 +- .../project_agent/.trinity/passport.json | 7 +- src/aipass/spawn/tests/test_citizen_id.py | 128 ++++++++++++++++++ 10 files changed, 185 insertions(+), 18 deletions(-) create mode 100644 src/aipass/spawn/tests/test_citizen_id.py diff --git a/src/aipass/spawn/.aipass/aipass_local_prompt.md b/src/aipass/spawn/.aipass/aipass_local_prompt.md index d4631da8f..a6f35f5f7 100644 --- a/src/aipass/spawn/.aipass/aipass_local_prompt.md +++ b/src/aipass/spawn/.aipass/aipass_local_prompt.md @@ -63,7 +63,9 @@ apps/ - Py files NEVER auto-overwritten during updates (design) - JSON files deep-merged (preserve existing values, add new template keys) - Update uses Phase 0 workflow: snapshot old tracking → detect changes → execute → refresh metadata -- Two citizen classes: aipass_framework (full 3-layer scaffold, 46 files), project_agent (18 files) +- Two citizen classes: aipass_framework (full 3-layer scaffold, 50 files), project_agent (17 files) +- Birth stamps TWO ids: citizenship.citizen_id (this citizen's own UID, == its branches[] registry_id) + and citizenship.registry_id (the REGISTRY's id, shared project-wide). Minted once in core, used twice - Mint verifies completeness: a template that ships fewer files than its manifest declares REFUSES, never half-registers ## Known Gotchas diff --git a/src/aipass/spawn/README.md b/src/aipass/spawn/README.md index 521bb85c5..9f7484763 100644 --- a/src/aipass/spawn/README.md +++ b/src/aipass/spawn/README.md @@ -213,10 +213,11 @@ spawn/ 3. **Copy** — Recursive copy of class template to target (skips `__pycache__`) 4. **Rename** — Replace `{{BRANCH}}` in directory and file names 5. **Replace** — Substitute all `{{PLACEHOLDER}}` patterns in file contents, including `{{CITIZEN_CLASS}}` (sourced from the create call, not a baked literal) -6. **Meta** — Generate `.branch_meta.json` (meta tabs load from `@memory` when available, degrading gracefully to empty when it's not) -7. **Verify** — Compare the minted tree against the template's own manifest (`.spawn/.template_registry.json`) and its on-disk contents. A file the template claims but the mint never produced REFUSES the create, names every missing path, and never reaches the registry — a gitignored template file used to mint a citizen with an empty `artifacts/` and no `inbox.json` while printing "Agent created" (2026-08-17). Custom `--template ` trees carry no manifest and are verified against their own contents only -8. **Registry** — Register in the target project's own `AIPASS_REGISTRY.json` -9. **Validate** — Scan for any remaining `{{...}}` patterns +6. **Identity ids** — Mint the citizen's own UUID ONCE and use it twice: stamped into the passport as `citizenship.citizen_id` (the citizen's unique id, rendered by faces as the passport number) and written as the `registry_id` of its `branches[]` registry entry. Minting it at registration time instead would be too late — the passport is written earlier in this list, so the two copies would be different UUIDs for one citizen. Distinct from `citizenship.registry_id`, which is the id of the REGISTRY holding the citizen and is shared by every citizen in a project (Patrick's ruling, 2026-08-24) +7. **Meta** — Generate `.branch_meta.json` (meta tabs load from `@memory` when available, degrading gracefully to empty when it's not) +8. **Verify** — Compare the minted tree against the template's own manifest (`.spawn/.template_registry.json`) and its on-disk contents. A file the template claims but the mint never produced REFUSES the create, names every missing path, and never reaches the registry — a gitignored template file used to mint a citizen with an empty `artifacts/` and no `inbox.json` while printing "Agent created" (2026-08-17). Custom `--template ` trees carry no manifest and are verified against their own contents only +9. **Registry** — Register in the target project's own `AIPASS_REGISTRY.json` +10. **Validate** — Scan for any remaining `{{...}}` patterns ### Update (class-aware, Phase 0) @@ -318,6 +319,6 @@ mandate. --- -*Last Updated: 2026-08-23* +*Last Updated: 2026-08-24* [← Back to AIPass](../../../README.md) diff --git a/src/aipass/spawn/apps/handlers/placeholders.py b/src/aipass/spawn/apps/handlers/placeholders.py index ec83eece0..f08954032 100644 --- a/src/aipass/spawn/apps/handlers/placeholders.py +++ b/src/aipass/spawn/apps/handlers/placeholders.py @@ -33,7 +33,7 @@ def build_replacements_dict(target_dir, branch_name, **overrides): target_dir: Path to target directory branch_name: Raw folder name **overrides: Optional overrides for ROLE, TRAITS, PURPOSE_BRIEF, PROFILE, - CITIZEN_NUMBER, MODULE, etc. + CITIZEN_NUMBER, CITIZEN_ID, MODULE, etc. Returns: Dict mapping placeholder names to replacement values @@ -42,6 +42,13 @@ def build_replacements_dict(target_dir, branch_name, **overrides): lower = branch_name.lower().replace("-", "_") now = datetime.now() + # CITIZEN_ID is the citizen's OWN unique id, stamped into the passport as + # citizenship.citizen_id and rendered by faces as the passport number. + # REGISTRY_ID below is a different fact: the id of the REGISTRY that holds + # the citizen (shared by every citizen in a project). One name apart, two + # meanings — see the citizenship block in the template passports. + citizen_id = overrides.get("citizen_id", "") + registry_id = overrides.get("registry_id", "") if not registry_id: registry_path = find_registry(start_path=Path(target_dir).parent) @@ -69,6 +76,7 @@ def build_replacements_dict(target_dir, branch_name, **overrides): "CITIZEN_NUMBER": str(overrides.get("citizen_number", 0)), "CITIZEN_CLASS": overrides.get("citizen_class", "aipass_framework"), "REGISTRY_ID": registry_id, + "CITIZEN_ID": citizen_id, "KEY_CAPABILITIES": "", "DEPENDS_ON": "", "PROVIDES_TO": "", diff --git a/src/aipass/spawn/apps/handlers/registry.py b/src/aipass/spawn/apps/handlers/registry.py index 3bf4a58d9..234455e6d 100644 --- a/src/aipass/spawn/apps/handlers/registry.py +++ b/src/aipass/spawn/apps/handlers/registry.py @@ -220,12 +220,18 @@ def _validate_path_containment(branch_path, registry_path): return False -def add_to_registry(registry_path, branch_name, branch_path, profile, email, purpose=""): +def add_to_registry(registry_path, branch_name, branch_path, profile, email, purpose="", citizen_id=""): """Add a new branch entry to the registry. - Always mints a fresh per-citizen UUID for the entry's ``registry_id`` - (the citizen UID). This is NOT the project credential — that lives - in ``metadata.id`` and is copied into passports separately. + Mints a fresh per-citizen UUID for the entry's ``registry_id`` (the citizen + UID) unless the caller supplies one. This is NOT the project credential — + that lives in ``metadata.id`` and is copied into passports separately. + + ``citizen_id`` exists so a caller that has ALREADY stamped the id into the + citizen's passport can hand the same value here: the passport's + ``citizenship.citizen_id`` and this entry's ``registry_id`` are two copies + of one fact, and minting twice would silently produce two different ids for + the same citizen. Callers with no passport to match (adoption) omit it. Uses file locking around the entire read-modify-write cycle to prevent corruption from concurrent spawns. Skips locking on Windows. @@ -237,6 +243,9 @@ def add_to_registry(registry_path, branch_name, branch_path, profile, email, pur profile: Profile string (e.g. "AIPass Workshop") email: Branch email (e.g. "@my_agent") purpose: Optional purpose description + citizen_id: Optional pre-minted per-citizen UUID. Empty string mints a + fresh one, which is the correct behaviour when no passport carries + the id yet. Returns: True if added, False if already exists or error @@ -280,7 +289,7 @@ def add_to_registry(registry_path, branch_name, branch_path, profile, email, pur "status": "active", "created": today, "last_active": today, - "registry_id": str(uuid.uuid4()), + "registry_id": citizen_id or str(uuid.uuid4()), } if isinstance(branches, dict): diff --git a/src/aipass/spawn/apps/modules/core.py b/src/aipass/spawn/apps/modules/core.py index 58a44e9fe..7bbdfe68f 100644 --- a/src/aipass/spawn/apps/modules/core.py +++ b/src/aipass/spawn/apps/modules/core.py @@ -19,6 +19,7 @@ 7. Validate no unreplaced placeholders remain """ +import uuid from pathlib import Path from typing import List @@ -277,6 +278,12 @@ def _spawn_agent( if reg_data: resolved_registry_id = reg_data.get("metadata", {}).get("id", "") + # Mint the citizen's own unique id ONCE, here, so the passport and the + # registry entry carry the same value. Minting it inside add_to_registry + # (step 4) would be too late: the passport is written at step 1, so the two + # facts would be two different UUIDs for one citizen. + citizen_id = str(uuid.uuid4()) + # Build placeholder replacements meta_tabs = _load_meta_tabs() replacements = build_replacements_dict( @@ -290,6 +297,7 @@ def _spawn_agent( citizen_class=citizen_class, meta_tabs=meta_tabs, registry_id=resolved_registry_id, + citizen_id=citizen_id, ) # Step 1: Copy template with placeholder replacement in content @@ -350,6 +358,7 @@ def _spawn_agent( detected_profile, f"@{branch_lower}", purpose or "New agent - purpose TBD", + citizen_id=citizen_id, ) # Step 5: Ensure at least one agent in the project is the owner diff --git a/src/aipass/spawn/templates/aipass_framework/.spawn/.template_registry.json b/src/aipass/spawn/templates/aipass_framework/.spawn/.template_registry.json index 188ab1a5d..4ff59ebf7 100644 --- a/src/aipass/spawn/templates/aipass_framework/.spawn/.template_registry.json +++ b/src/aipass/spawn/templates/aipass_framework/.spawn/.template_registry.json @@ -201,7 +201,7 @@ "path": ".trinity/observations.json" }, "f014": { - "content_hash": "98f1e33dace0", + "content_hash": "50f84bdf948a", "has_branch_placeholder": false, "name": "passport.json", "path": ".trinity/passport.json" @@ -425,7 +425,7 @@ }, "metadata": { "description": "Template file tracking registry for ID-based updates", - "last_updated": "2026-08-23", + "last_updated": "2026-08-24", "version": "1.0.0" } } diff --git a/src/aipass/spawn/templates/aipass_framework/.trinity/passport.json b/src/aipass/spawn/templates/aipass_framework/.trinity/passport.json index 572979315..b5fca3734 100644 --- a/src/aipass/spawn/templates/aipass_framework/.trinity/passport.json +++ b/src/aipass/spawn/templates/aipass_framework/.trinity/passport.json @@ -7,7 +7,11 @@ "created": "{{DATE}}", "last_updated": "{{DATE}}", "managed_by": "{{BRANCHNAME}}", - "tags": ["identity", "passport", "branch_profile"] + "tags": [ + "identity", + "passport", + "branch_profile" + ] }, "branch_info": { "branch_name": "{{BRANCHNAME}}", @@ -34,6 +38,7 @@ "citizenship": { "registered": true, "registry_id": "{{REGISTRY_ID}}", + "citizen_id": "{{CITIZEN_ID}}", "communications": true, "memory": true } diff --git a/src/aipass/spawn/templates/project_agent/.spawn/.template_registry.json b/src/aipass/spawn/templates/project_agent/.spawn/.template_registry.json index fd4d9ceee..01706dd51 100644 --- a/src/aipass/spawn/templates/project_agent/.spawn/.template_registry.json +++ b/src/aipass/spawn/templates/project_agent/.spawn/.template_registry.json @@ -78,7 +78,7 @@ "path": ".trinity/observations.json" }, "f006": { - "content_hash": "c94e263d2b37", + "content_hash": "8e6a1d1a40a5", "has_branch_placeholder": false, "name": "passport.json", "path": ".trinity/passport.json" @@ -152,7 +152,7 @@ }, "metadata": { "description": "Template file tracking registry for ID-based updates", - "last_updated": "2026-08-23", + "last_updated": "2026-08-24", "version": "1.0.0" } } diff --git a/src/aipass/spawn/templates/project_agent/.trinity/passport.json b/src/aipass/spawn/templates/project_agent/.trinity/passport.json index a56a4cee9..3c75224b0 100644 --- a/src/aipass/spawn/templates/project_agent/.trinity/passport.json +++ b/src/aipass/spawn/templates/project_agent/.trinity/passport.json @@ -7,7 +7,11 @@ "created": "{{DATE}}", "last_updated": "{{DATE}}", "managed_by": "{{BRANCHNAME}}", - "tags": ["identity", "passport", "branch_profile"] + "tags": [ + "identity", + "passport", + "branch_profile" + ] }, "branch_info": { "branch_name": "{{BRANCHNAME}}", @@ -34,6 +38,7 @@ "citizenship": { "registered": true, "registry_id": "{{REGISTRY_ID}}", + "citizen_id": "{{CITIZEN_ID}}", "communications": true, "memory": true } diff --git a/src/aipass/spawn/tests/test_citizen_id.py b/src/aipass/spawn/tests/test_citizen_id.py new file mode 100644 index 000000000..c1424e955 --- /dev/null +++ b/src/aipass/spawn/tests/test_citizen_id.py @@ -0,0 +1,128 @@ +# =================== AIPass ==================== +# Name: test_citizen_id.py +# Description: citizenship.citizen_id — the per-citizen UID stamped at birth +# Version: 1.0.0 +# Created: 2026-08-24 +# Modified: 2026-08-24 +# ============================================= + +"""Tests for the citizen_id contract (Patrick's ruling, 2026-08-24). + +Two ids live near each other and mean different things: + - ``citizenship.registry_id`` — the id of the REGISTRY holding the citizen. + Shared by every citizen in a project. Rendered as "Branch reg no.". + - ``citizenship.citizen_id`` — the citizen's OWN unique id, the same value + the registry keeps in its ``branches[]`` entry. Rendered as "Passport no.". + +The load-bearing property is that those two copies of the citizen's own id are +minted ONCE and therefore always agree. The mint used to happen inside +add_to_registry, which runs after the passport is written — so this file pins +the ordering, not just the presence of a field. +""" + +import json +import uuid + +from aipass.spawn.apps.handlers.placeholders import build_replacements_dict +from aipass.spawn.apps.handlers.registry import add_to_registry + + +def _fresh_registry(path): + """Write a minimal registry the add path will accept.""" + path.write_text( + json.dumps({"metadata": {"id": str(uuid.uuid4()), "version": "1.0.0", "total_branches": 0}, "branches": []}), + encoding="utf-8", + ) + return path + + +# ============================================================================= +# PLACEHOLDER SURFACE +# ============================================================================= + + +def test_citizen_id_override_reaches_the_placeholder_map(tmp_path): + """A supplied citizen_id is offered to the template as {{CITIZEN_ID}}.""" + given = str(uuid.uuid4()) + + result = build_replacements_dict(tmp_path / "widget", "widget", citizen_id=given) + + assert result["CITIZEN_ID"] == given + + +def test_citizen_id_defaults_to_empty_not_missing(tmp_path): + """The key always exists — a missing key would leave {{CITIZEN_ID}} unrendered.""" + result = build_replacements_dict(tmp_path / "widget", "widget") + + assert result["CITIZEN_ID"] == "" + + +def test_citizen_id_is_not_the_registry_id(tmp_path): + """The two ids are distinct facts and must not be aliased to one value.""" + result = build_replacements_dict( + tmp_path / "widget", "widget", citizen_id="aaaa-citizen", registry_id="bbbb-registry" + ) + + assert result["CITIZEN_ID"] == "aaaa-citizen" + assert result["REGISTRY_ID"] == "bbbb-registry" + + +# ============================================================================= +# REGISTRY ENTRY +# ============================================================================= + + +def test_add_to_registry_uses_the_supplied_citizen_id(tmp_path): + """A caller that already stamped the passport hands the SAME id here.""" + reg = _fresh_registry(tmp_path / "AIPASS_REGISTRY.json") + given = str(uuid.uuid4()) + branch = tmp_path / "widget" + branch.mkdir() + + assert add_to_registry(reg, "WIDGET", str(branch), "p", "@widget", citizen_id=given) is True + + entry = json.loads(reg.read_text(encoding="utf-8"))["branches"][0] + assert entry["registry_id"] == given + + +def test_add_to_registry_mints_when_no_citizen_id_supplied(tmp_path): + """Adoption has no passport id to match, so the entry still gets a real UUID.""" + reg = _fresh_registry(tmp_path / "AIPASS_REGISTRY.json") + branch = tmp_path / "widget" + branch.mkdir() + + add_to_registry(reg, "WIDGET", str(branch), "p", "@widget") + + entry = json.loads(reg.read_text(encoding="utf-8"))["branches"][0] + assert uuid.UUID(entry["registry_id"]) # parses => a real UUID, not "" or None + + +def test_supplied_citizen_id_is_not_overwritten_by_a_fresh_mint(tmp_path): + """Regression guard: the mint must not run when the caller supplied a value.""" + reg = _fresh_registry(tmp_path / "AIPASS_REGISTRY.json") + given = "11111111-2222-3333-4444-555555555555" + branch = tmp_path / "widget" + branch.mkdir() + + add_to_registry(reg, "WIDGET", str(branch), "p", "@widget", citizen_id=given) + + entry = json.loads(reg.read_text(encoding="utf-8"))["branches"][0] + assert entry["registry_id"] == given + + +# ============================================================================= +# TEMPLATE CONTRACT +# ============================================================================= + + +def test_both_templates_declare_citizen_id(): + """Every class must stamp the field, or that class's births render no number.""" + from pathlib import Path + + templates = Path(__file__).resolve().parents[1] / "templates" + for citizen_class in ("aipass_framework", "project_agent"): + passport = templates / citizen_class / ".trinity" / "passport.json" + citizenship = json.loads(passport.read_text(encoding="utf-8"))["citizenship"] + + assert citizenship["citizen_id"] == "{{CITIZEN_ID}}", f"{citizen_class} does not stamp citizen_id" + assert citizenship["registry_id"] == "{{REGISTRY_ID}}", f"{citizen_class} lost its registry_id" From 90d6723ff6cd30652b78c5faffa13bc9d91d527f Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 20:50:02 -0700 Subject: [PATCH 05/43] fix(ai_mail): registry rows leave the reader absolute - a projects/* citizen's mailbox stops resolving into a phantom dir in OUR tree (@ai_mail's build, committed by devpulse). THE DEFECT: _lookup_branch_by_name correctly falls back to the caller's project registry for citizens absent from AIPASS_REGISTRY, but returned the RAW ROW - a path relative to the registry that holds it, with no memory of which registry answered. Every consumer joined it to the AIPass repo root, so BAUD (projects/baud, row src/baud/baud) resolved to /src/baud/baud. Both measured symptoms were that one path: inbox printed empty against a store holding 4, reply refused an id read straight out of the file under his feet. THE BAD PART: it did not fail loudly, it FABRICATED - the wrong path is inside the AIPass tree, so writes silently succeeded. BAUD's answer to Patrick's continuity probe (papa26, dispatch 6f0a7109) was swallowed into the phantom at 20:13, status refused, seen by nobody. Reply is the ONLY sanctioned cross-project return lane, so the defect forced exactly the silent completion the house forbids - his answer only reached us because devpulse's feedback channel is a separate path. THE FIX, at the source not the nine call sites: _rooted() absolutises a row against its own registry in both lanes of _lookup_branch_by_name and both of get_branch_info_from_registry - rows leave the reader absolute, so the join-to-wrong-root question cannot be asked downstream. Rows already absolute pass through untouched. Three red-first tests, including a guard that an AIPass citizen with a relative row is unaffected (six of eighteen AIPass rows are relative - fixing the project lane by breaking the main one would trade a rare failure for the common one). ACCEPTANCE, live from BAUD's own seat, Patrick's bar: inbox lists all 5 including both timestamp formats (BAUD's own mixed-format suspect thereby refuted by evidence - the formats were real and were not the fault), the once-missing id resolves, and his reply arrived through ai_mail itself - the dead command, alive. Timestamp side note: devpulse's feedback module stops writing UTC ISO-T into ai_mail stores in the sibling commit; the existing ISO rows stay - ai_mail reads them fine, a rewrite of another citizen's store would be a migration, not a tidy. The phantom src/baud/ was removed via drone rm after content was preserved on the live lane. RIDER, explained not smuggled: purge.py's known-gap comment said @memory's vectorize_and_store DOES NOT EXIST - @memory shipped it this morning (chroma_subprocess 1.4.0, commit 660ab692), the seam is verified live from this side, and the comment now records the four-month history instead of asserting a gap that closed. README test counts 1323 -> 1326. Bar: 1326 passed from branch root (re-run independently by devpulse), fresh-checkout mimic green, winhome_sim green, seedgo 100%. NOT touched, flagged to Patrick: @daemon's inbox_sweep discovers candidates from AIPASS_REGISTRY only, so the fresh-wake self-healing that mints pointers for all 18 core citizens has never reached a projects/* citizen - whether they join the fleet sweep is a ruling, not a repair. --- src/aipass/ai_mail/README.md | 4 +- .../ai_mail/apps/handlers/email/purge.py | 24 ++-- .../apps/handlers/users/branch_detection.py | 41 ++++++- .../ai_mail/tests/test_identity_fence.py | 113 ++++++++++++++++++ 4 files changed, 165 insertions(+), 17 deletions(-) diff --git a/src/aipass/ai_mail/README.md b/src/aipass/ai_mail/README.md index 71d9433a8..00599301a 100644 --- a/src/aipass/ai_mail/README.md +++ b/src/aipass/ai_mail/README.md @@ -9,7 +9,7 @@ --- -**Status:** Operational | **Seedgo:** 100% (99% with every bypass rule off) | **Tests:** 1323 pass (1319 + 4 live-hygiene skips on a fresh checkout) | **Battle Tested:** S62 +**Status:** Operational | **Seedgo:** 100% (99% with every bypass rule off) | **Tests:** 1326 pass (1322 + 4 live-hygiene skips on a fresh checkout) | **Battle Tested:** S62 ## Quick Start @@ -754,7 +754,7 @@ ai_mail/ │ ├── paths.py # Shared find_repo_root() utility │ ├── notify.py # Notification feed writer (JSONL, BAUD reads) │ └── central_writer.py # Central inbox stats aggregation -└── tests/ # 1323 tests across 46 test files +└── tests/ # 1326 tests across 46 test files ├── conftest.py # Shared fixtures (mock_logger, mock_json_handler) ├── test_daemon.py # Daemon config, state, kill switch, dispatch check ├── test_dispatch_monitor.py # Monitor safety features, env stripping diff --git a/src/aipass/ai_mail/apps/handlers/email/purge.py b/src/aipass/ai_mail/apps/handlers/email/purge.py index ae5790805..1fe88aea9 100644 --- a/src/aipass/ai_mail/apps/handlers/email/purge.py +++ b/src/aipass/ai_mail/apps/handlers/email/purge.py @@ -226,17 +226,19 @@ def _vectorize_emails(emails: List[Dict[str, Any]], folder_type: str) -> Dict[st ) # Call @memory vectorization via subprocess (handler independence) - # KNOWN GAP, LEFT VISIBLE ON PURPOSE — DO NOT "FIX" THIS BY RENAMING IT. - # @memory's handler has no `vectorize_and_store` operation and there is - # no evidence it ever did, so this call has always been refused. Renaming - # to `store_vectors` does NOT make it work: that operation wants - # `embeddings` (already-encoded vectors) plus `documents`, and what is - # sent here is raw `texts`. Encoding them first would mean THIS branch - # picking an embedding model, and a collection whose vectors come from - # two different models is silently unsearchable — that choice belongs to - # the branch that owns the store. Requested from @memory as a text-in - # operation on their surface; until it lands, the check above makes this - # fail LOUDLY and preserve the mail instead of deleting it. + # THE GAP IS CLOSED — this operation is real now. It was NOT, for about + # four months: @memory's handler had no `vectorize_and_store`, answered + # this call with success:false at exit 0, and purge read that refusal as + # a store and deleted the originals. @memory shipped the operation in + # chroma_subprocess 1.4.0 (2026-08-24) and the seam is verified live from + # this side: a probe returns {'success': True} and both + # ai_mail_email_sent and ai_mail_email_deleted now exist. + # + # Kept as a TEXT-IN call deliberately. `store_vectors` wants `embeddings` + # plus `documents`; sending raw `texts` and letting @memory encode them + # means the branch that OWNS the store picks the embedding model. If each + # caller encoded its own, two callers could put vectors from two models + # in one collection, which is silently unsearchable rather than an error. input_data = { "operation": "vectorize_and_store", "branch": "AI_MAIL", diff --git a/src/aipass/ai_mail/apps/handlers/users/branch_detection.py b/src/aipass/ai_mail/apps/handlers/users/branch_detection.py index 9b0f28d88..3443dcce8 100644 --- a/src/aipass/ai_mail/apps/handlers/users/branch_detection.py +++ b/src/aipass/ai_mail/apps/handlers/users/branch_detection.py @@ -346,6 +346,39 @@ def detect_branch_from_pwd() -> Optional[Dict]: return None +def _rooted(branch: Dict, registry_path: Path) -> Dict: + """Return *branch* with its ``path`` made absolute against its OWN registry. + + A registry row's ``path`` is relative to THE REGISTRY THAT HOLDS IT, and a + row handed back raw carries no memory of which registry answered. Every + consumer then joins it to the AIPass repo root — right for AIPass citizens + by coincidence, wrong for every project citizen. + + Found live 2026-08-24 (@devpulse 10400b9b, measured by @baud): a projects/* + citizen read "Inbox is empty" with four unread messages in the file under + his feet, and `reply ` answered "Message not found" for an id read out + of that same file. ``projects/baud`` + ``src/baud/baud`` had been resolved + as ``/src/baud/baud``. + + IT FABRICATED RATHER THAN FAILING. The wrong path sits inside the AIPass + tree, so the mail lane CREATED it — a phantom .ai_mail.local/sent/ holding + the reply he believed he had sent, in a directory belonging to no citizen. + A refusal would have been loud; a confident wrong address was not. + + Absolutising HERE, at the one place a registry is read, rather than at the + nine call sites that join a registry path: a consumer cannot re-derive a + root it was never given, and nine copies of that join is how they drift. + Rows that are already absolute are returned untouched. + """ + path = str(branch.get("path", "")) + if not path or Path(path).is_absolute(): + return branch + + rooted = dict(branch) + rooted["path"] = str((registry_path.parent / path).resolve()) + return rooted + + def _lookup_branch_by_name(branch_name: str) -> Optional[Dict]: """ Look up branch in the registry by name (case-insensitive). @@ -368,7 +401,7 @@ def _lookup_branch_by_name(branch_name: str) -> Optional[Dict]: registry = json.load(f) for branch in _get_branches_list(registry): if branch.get("name", "").lower() == name_lower: - return branch + return _rooted(branch, BRANCH_REGISTRY_PATH) except Exception as e: logger.warning("[identity] _lookup_branch_by_name(%s) failed: %s", branch_name, e) @@ -380,7 +413,7 @@ def _lookup_branch_by_name(branch_name: str) -> Optional[Dict]: registry = json.load(f) for branch in _get_branches_list(registry): if branch.get("name", "").lower() == name_lower: - return branch + return _rooted(branch, caller_registry) except Exception as e: logger.warning( "[identity] _lookup_branch_by_name(%s) caller registry %s failed: %s", branch_name, caller_registry, e @@ -443,7 +476,7 @@ def get_branch_info_from_registry(branch_path: Path) -> Optional[Dict]: else: reg_path = reg_path.resolve() if reg_path == branch_path_resolved: - return branch + return _rooted(branch, BRANCH_REGISTRY_PATH) except Exception as e: logger.warning("[identity] get_branch_info_from_registry(%s) failed: %s", branch_path, e) @@ -461,7 +494,7 @@ def get_branch_info_from_registry(branch_path: Path) -> Optional[Dict]: else: reg_path = reg_path.resolve() if reg_path == branch_path_resolved: - return branch + return _rooted(branch, caller_registry) except Exception as e: logger.warning("[identity] get_branch_info_from_registry(%s) caller registry failed: %s", branch_path, e) diff --git a/src/aipass/ai_mail/tests/test_identity_fence.py b/src/aipass/ai_mail/tests/test_identity_fence.py index 697af9dcb..b245044d3 100644 --- a/src/aipass/ai_mail/tests/test_identity_fence.py +++ b/src/aipass/ai_mail/tests/test_identity_fence.py @@ -371,3 +371,116 @@ def test_contacts_still_serve_a_name_the_registry_does_not_know(self, monkeypatc assert resolved is not None assert Path(resolved["path"]).name == "vera" + + +class TestAProjectCitizenResolvesAgainstItsOwnRegistry: + """Found live 2026-08-24 (@devpulse 10400b9b, measured first by @baud). + + A projects/* citizen could not read or reply to his own mail: `inbox` printed + "Inbox is empty" while four unread messages sat in the file under his feet, + and `reply ` answered "Message not found" for an id read straight out of + that file. Both symptoms are one cause, and the identity log named it in a + single row - the contrast with the row logged seconds later is the defect: + + BAUD strategy caller_branch:registry resolved_path src/baud/baud + DEVPULSE strategy caller_branch:registry resolved_path /home/.../devpulse + + One relative, one absolute. A registry row's ``path`` is relative to THE + REGISTRY IT CAME FROM. ``_lookup_branch_by_name`` falls back to the caller's + own project registry for citizens absent from the AIPass one - correctly - + but returns the raw row, which carries no memory of which registry answered. + Every consumer then joins it to the AIPass repo root, so + ``projects/baud`` + ``src/baud/baud`` was read as ``/src/baud/baud``. + + IT DID NOT FAIL LOUDLY, IT FABRICATED. The wrong path is inside the AIPass + tree, so the mail lane CREATED it: src/baud/baud/.ai_mail.local/sent/ exists, + timestamped inside his session, holding the reply he believed he had sent. + Reading found nothing there (hence "empty"), and writing made a phantom + mailbox belonging to no citizen. Same species as the contacts-cache leak of + 08-23 and the register-in-the-wrong-place risk of 08-22: not a refusal, a + confident wrong address. + + Fixed at the SOURCE rather than at the nine call sites that join a registry + path - the row is absolutised by the function that reads the registry, so no + consumer can ever be handed a relative path without the root that explains it. + """ + + @staticmethod + def _project(tmp_path): + """A project citizen laid out exactly like projects/baud.""" + project_root = tmp_path / "projects" / "baud" + citizen = project_root / "src" / "baud" / "baud" + (citizen / ".ai_mail.local").mkdir(parents=True) + # A real seat: the fence refuses a caller_cwd that stands in no branch. + (citizen / ".trinity").mkdir() + (citizen / ".trinity" / "passport.json").write_text( + json.dumps({"branch_info": {"branch_name": "baud", "email": "@baud"}}), encoding="utf-8" + ) + (project_root / "BAUD_REGISTRY.json").write_text( + json.dumps({"branches": [{"name": "BAUD", "email": "@baud", "path": "src/baud/baud"}]}), + encoding="utf-8", + ) + return project_root, citizen + + def test_the_relative_path_is_rooted_at_its_own_registry(self, monkeypatch, tmp_path): + """The regression test: a project row must not be joined to the AIPass root.""" + project_root, citizen = self._project(tmp_path) + + aipass_registry = tmp_path / "AIPASS_REGISTRY.json" + aipass_registry.write_text(json.dumps({"branches": []}), encoding="utf-8") + monkeypatch.setattr(bd, "BRANCH_REGISTRY_PATH", aipass_registry) + monkeypatch.setenv("AIPASS_CALLER_CWD", str(citizen)) + monkeypatch.setenv("AIPASS_CALLER_BRANCH", "baud") + + resolved = bd.detect_branch_from_pwd() + + assert resolved is not None, "premise: the project citizen must still resolve" + assert Path(resolved["path"]).is_absolute(), ( + f"a row leaving the registry reader must carry its root. Got: {resolved['path']}" + ) + assert Path(resolved["path"]).resolve() == citizen.resolve(), ( + f"resolved to the wrong tree: {resolved['path']} (his mailbox is at {citizen})" + ) + + def test_an_aipass_citizen_with_a_relative_row_is_unaffected(self, monkeypatch, tmp_path): + """Six of the eighteen AIPass rows are relative — this must not move them. + + Fixing the project lane by breaking the main one would trade a rare + failure for the common one. + """ + repo = tmp_path / "aipass" + citizen = repo / "src" / "aipass" / "ai_mail" + (citizen / ".trinity").mkdir(parents=True) + (citizen / ".trinity" / "passport.json").write_text( + json.dumps({"branch_info": {"branch_name": "ai_mail", "email": "@ai_mail"}}), encoding="utf-8" + ) + registry = repo / "AIPASS_REGISTRY.json" + registry.write_text( + json.dumps({"branches": [{"name": "AI_MAIL", "email": "@ai_mail", "path": "src/aipass/ai_mail"}]}), + encoding="utf-8", + ) + monkeypatch.setattr(bd, "BRANCH_REGISTRY_PATH", registry) + monkeypatch.delenv("AIPASS_CALLER_CWD", raising=False) + monkeypatch.setenv("AIPASS_CALLER_BRANCH", "ai_mail") + + resolved = bd.detect_branch_from_pwd() + + assert resolved is not None + assert Path(resolved["path"]).resolve() == citizen.resolve() + + def test_the_mailbox_follows_the_corrected_root(self, monkeypatch, tmp_path): + """The consequence the citizen actually feels: inbox and reply find his mail.""" + from aipass.ai_mail.apps.handlers.users.user import get_current_user + + project_root, citizen = self._project(tmp_path) + aipass_registry = tmp_path / "AIPASS_REGISTRY.json" + aipass_registry.write_text(json.dumps({"branches": []}), encoding="utf-8") + monkeypatch.setattr(bd, "BRANCH_REGISTRY_PATH", aipass_registry) + monkeypatch.setenv("AIPASS_CALLER_CWD", str(citizen)) + monkeypatch.setenv("AIPASS_CALLER_BRANCH", "baud") + + user = get_current_user() + + assert Path(user["mailbox_path"]).resolve() == (citizen / ".ai_mail.local").resolve(), ( + f"his commands would read {user['mailbox_path']} instead of his own mailbox" + ) From 78180e6d17a4908da6f15533fa644174bc11a651 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 20:50:22 -0700 Subject: [PATCH 06/43] feat(spawn): a new external project is born WITH its credential - load_registry mints metadata.id for a missing registry, and the mint-once ordering reaches the passport (@spawn's build, greenlit 08-24, committed by devpulse). THE DEFECT, named in d0ce2ce4 and fixed here: load_registry's default schema minted no id, so a brand-new external project's first citizen fell back to whatever registry discovery found next - AIPass's own credential on a passport from a project it was never part of. THE FIX in registry.py: _default_registry_schema(credential) builds the empty document, and load_registry splits three ways - file MISSING means a NEW PROJECT and is born with str(uuid.uuid4()); file READS means the file, untouched; file UNREADABLE means an id-less schema plus a logger.warning naming the file. The asymmetry is deliberate and four tests hold it: a file that exists but will not parse is not a new project, it is a live project whose credential we failed to READ - minting a replacement would re-credential it and orphan every passport carrying the real id. Missing means regenerate; unreadable means never clobber. THE SECOND FILE was not scope creep, it was the fix reaching the passport: the passport writes at create step 1 and the registry at step 4, so resolving the credential at registration time hands the passport a value not yet minted - the live probe still leaked 7087bb93 after the registry.py change alone. _spawn_agent now resolves the credential at step 1 via load_registry and hands it to add_to_registry, which adopts it ONLY for a registry it is CREATING - keyed off registry_path.exists() captured BEFORE the load, not off id-already-set, because load_registry now always returns an id and the caller's would otherwise be shadowed into a two-mint disagreement (@spawn built that bug, caught it by minting a real agent, and took it out - passport 53b58cb2 vs registry 35ed868d, one project two credentials, never shipped). PROOF, live mint in a clean probe: project registry metadata.id == passport registry_id, citizen_id == its own registry row, AIPass 7087bb93 leaked = False; probe removed via drone rm. Additive as scoped - no existing registry is read, written, or touched differently; AIPASS_REGISTRY still carries 7087bb93 and all 23 live passports still match, 23/23. Bar: tests/test_registry_credential.py new, 11 red-first tests with three mutations each caught by the right test by name; 506 passed / 1 skipped (re-run independently by devpulse); seedgo @spawn 100%. FLAGGED NOT FIXED, awaiting Patrick's GO: add_to_registry against an EXISTS-but-unreadable registry receives the id-less empty schema, adds one branch, and would write that over the real file - every existing branch entry gone. Pre-existing, now precisely nameable: the empty schema is a legitimate return for two different situations and only one is safe to write back. The guard is a small separate fix with its own red-first tests. --- src/aipass/spawn/apps/handlers/registry.py | 77 +++++++-- src/aipass/spawn/apps/modules/core.py | 18 ++- .../spawn/tests/test_registry_credential.py | 151 ++++++++++++++++++ 3 files changed, 223 insertions(+), 23 deletions(-) create mode 100644 src/aipass/spawn/tests/test_registry_credential.py diff --git a/src/aipass/spawn/apps/handlers/registry.py b/src/aipass/spawn/apps/handlers/registry.py index 234455e6d..a9ace5146 100644 --- a/src/aipass/spawn/apps/handlers/registry.py +++ b/src/aipass/spawn/apps/handlers/registry.py @@ -134,10 +134,44 @@ def find_registry(start_path=None): return _common_find(start_path=start_path) +def _default_registry_schema(credential=""): + """Return an empty registry document. + + Args: + credential: The project's ``metadata.id``. Empty string omits the key + entirely — see load_registry for why that is not the same as + minting a fresh one. + + Returns: + Dict with metadata and an empty branches list. + """ + metadata = { + "version": "1.0.0", + "last_updated": datetime.now().strftime("%Y-%m-%d"), + "total_branches": 0, + } + if credential: + metadata["id"] = credential + return {"metadata": metadata, "branches": []} + + def load_registry(registry_path): """ Load registry from JSON file. Returns empty schema if missing. + A registry that does not exist yet is a NEW PROJECT, and it is born with a + freshly minted ``metadata.id`` — the project credential every passport in + that project carries as ``citizenship.registry_id`` (rendered as + "Branch reg no."). Without it the project's first citizen falls back to + whatever registry discovery finds next, which is AIPass's own id: a + brand-new agent displaying a number from a project it was never part of. + + The unreadable case deliberately does NOT mint one. A file that exists but + cannot be parsed is not a new project — it is a project whose credential we + failed to read, and inventing a replacement would re-credential a live + project and orphan every passport already carrying the real id. Missing + means regenerate; unreadable means do not clobber. + Args: registry_path: Path to AIPASS_REGISTRY.json @@ -146,26 +180,18 @@ def load_registry(registry_path): """ registry_path = Path(registry_path) if not registry_path.exists(): - return { - "metadata": { - "version": "1.0.0", - "last_updated": datetime.now().strftime("%Y-%m-%d"), - "total_branches": 0, - }, - "branches": [], - } + return _default_registry_schema(credential=str(uuid.uuid4())) data = json_handler.read_json(registry_path) if data is not None: return data - return { - "metadata": { - "version": "1.0.0", - "last_updated": datetime.now().strftime("%Y-%m-%d"), - "total_branches": 0, - }, - "branches": [], - } + + logger.warning( + "[registry] %s exists but could not be read — returning an empty schema WITHOUT a credential " + "so a live project is never re-credentialled. Its branches are not in this result.", + registry_path.name, + ) + return _default_registry_schema() def save_registry(registry_path, data): @@ -220,7 +246,7 @@ def _validate_path_containment(branch_path, registry_path): return False -def add_to_registry(registry_path, branch_name, branch_path, profile, email, purpose="", citizen_id=""): +def add_to_registry(registry_path, branch_name, branch_path, profile, email, purpose="", citizen_id="", credential=""): """Add a new branch entry to the registry. Mints a fresh per-citizen UUID for the entry's ``registry_id`` (the citizen @@ -246,6 +272,10 @@ def add_to_registry(registry_path, branch_name, branch_path, profile, email, pur citizen_id: Optional pre-minted per-citizen UUID. Empty string mints a fresh one, which is the correct behaviour when no passport carries the id yet. + credential: Optional project credential (``metadata.id``) to write when + the registry does not carry one yet. Supplied by a caller that has + ALREADY stamped it into a passport, so the file and the passport + agree. NEVER overwrites an id the registry already has. Returns: True if added, False if already exists or error @@ -268,7 +298,20 @@ def add_to_registry(registry_path, branch_name, branch_path, profile, email, pur fcntl.flock(lock_fd, fcntl.LOCK_EX) try: + # Whether the FILE existed decides who owns the credential, not whether + # the loaded dict has an id: load_registry mints one for a missing file, + # so a freshly minted id would otherwise look "already set" and shadow + # the caller's — leaving the passport claiming one credential and the + # registry carrying another. Two mints, one project, no agreement. + registry_existed = registry_path.exists() registry = load_registry(registry_path) + + # A registry the caller is creating adopts the caller's credential (the + # one already stamped into the passport). An EXISTING registry's id is + # the project's lock and is never touched here. + if credential and not registry_existed: + registry.setdefault("metadata", {})["id"] = credential + branches = registry.get("branches", []) if isinstance(branches, dict): diff --git a/src/aipass/spawn/apps/modules/core.py b/src/aipass/spawn/apps/modules/core.py index 7bbdfe68f..719da3cb3 100644 --- a/src/aipass/spawn/apps/modules/core.py +++ b/src/aipass/spawn/apps/modules/core.py @@ -44,6 +44,7 @@ from aipass.spawn.apps.handlers.meta_ops import load_template_registry, generate_branch_meta, save_branch_meta from aipass.spawn.apps.handlers.mint_verify import verify_mint from aipass.spawn.apps.handlers.registry import ( + load_registry, find_registry, add_to_registry, get_next_citizen_number, @@ -271,12 +272,16 @@ def _spawn_agent( reg_path = _find_project_registry(target) citizen_number = get_next_citizen_number(reg_path) - # Read registry_id from the resolved registry for credential linkage - resolved_registry_id = "" - if reg_path.exists(): - reg_data = json_handler.read_json(reg_path) - if reg_data: - resolved_registry_id = reg_data.get("metadata", {}).get("id", "") + # Resolve the PROJECT credential (the registry's own metadata.id) for the + # passport's citizenship.registry_id. load_registry mints one when the + # registry does not exist yet, which is what a brand-new external project + # is — resolving it HERE rather than at registration time is the whole + # point: the passport is written at step 1 and the registry at step 4, so + # reading it later would stamp the passport with a credential that had not + # been minted yet and fall back to AIPass's own id. Same mint-once ordering + # as citizen_id below; the value is handed to add_to_registry so the file + # that eventually lands carries the id the passport already claims. + resolved_registry_id = load_registry(reg_path).get("metadata", {}).get("id", "") # Mint the citizen's own unique id ONCE, here, so the passport and the # registry entry carry the same value. Minting it inside add_to_registry @@ -359,6 +364,7 @@ def _spawn_agent( f"@{branch_lower}", purpose or "New agent - purpose TBD", citizen_id=citizen_id, + credential=resolved_registry_id, ) # Step 5: Ensure at least one agent in the project is the owner diff --git a/src/aipass/spawn/tests/test_registry_credential.py b/src/aipass/spawn/tests/test_registry_credential.py new file mode 100644 index 000000000..d5304d303 --- /dev/null +++ b/src/aipass/spawn/tests/test_registry_credential.py @@ -0,0 +1,151 @@ +# =================== AIPass ==================== +# Name: test_registry_credential.py +# Description: metadata.id — the project credential a new registry is born with +# Version: 1.0.0 +# Created: 2026-08-24 +# Modified: 2026-08-24 +# ============================================= + +"""Tests for the registry credential (metadata.id). + +A project registry's ``metadata.id`` is the branch-registry lock: every passport +in that project carries it as ``citizenship.registry_id``, and BAUD renders it +as "Branch reg no.". A registry born WITHOUT one is not merely incomplete — its +first citizen's passport falls back to whatever registry discovery finds next, +which in practice is AIPass's own id, so a brand-new project's agent displays a +number belonging to a project it has never been part of. + +The two default-schema paths in load_registry are deliberately NOT symmetric, +and these tests pin that asymmetry: + + - registry file ABSENT -> a genuinely new project -> mint a credential. + - registry file PRESENT but unreadable -> the project ALREADY HAS a + credential we simply cannot read. Minting a fresh one here would + re-credential a live project and orphan every existing passport, so this + path must NOT invent one. +""" + +import json +import uuid + +import pytest + +from aipass.spawn.apps.handlers.registry import load_registry + + +# ============================================================================= +# ABSENT REGISTRY — a new project earns a credential +# ============================================================================= + + +def test_missing_registry_is_born_with_a_credential(tmp_path): + """A registry that does not exist yet gets a real minted id.""" + result = load_registry(tmp_path / "NEW_REGISTRY.json") + + assert uuid.UUID(result["metadata"]["id"]) # parses => real UUID, not "" or None + + +def test_minted_credential_is_unique_per_registry(tmp_path): + """Two new projects must not share a credential — it is their lock.""" + first = load_registry(tmp_path / "ONE_REGISTRY.json")["metadata"]["id"] + second = load_registry(tmp_path / "TWO_REGISTRY.json")["metadata"]["id"] + + assert first != second + + +def test_minted_default_keeps_the_rest_of_the_schema(tmp_path): + """Adding the credential must not disturb the fields callers already read.""" + result = load_registry(tmp_path / "NEW_REGISTRY.json") + + assert result["metadata"]["version"] == "1.0.0" + assert result["metadata"]["total_branches"] == 0 + assert result["branches"] == [] + assert "last_updated" in result["metadata"] + + +def test_new_project_credential_is_not_aipass_own_id(tmp_path): + """The regression this exists to prevent: inheriting AIPass's credential.""" + result = load_registry(tmp_path / "NEW_REGISTRY.json") + + assert result["metadata"]["id"] != "7087bb93-570f-4b9a-b035-4fd7f570200e" + + +# ============================================================================= +# EXISTING REGISTRY — never re-credential what we merely failed to read +# ============================================================================= + + +def test_real_registry_credential_is_never_replaced(tmp_path): + """A readable registry hands back its OWN id, untouched.""" + path = tmp_path / "REAL_REGISTRY.json" + path.write_text( + json.dumps({"metadata": {"id": "keep-me", "version": "1.0.0", "total_branches": 0}, "branches": []}), + encoding="utf-8", + ) + + assert load_registry(path)["metadata"]["id"] == "keep-me" + + +def test_unreadable_registry_does_not_mint_a_replacement_credential(tmp_path): + """An unreadable file is not a new project — inventing an id here would + silently re-credential a live project and orphan every passport in it.""" + corrupt = tmp_path / "CORRUPT_REGISTRY.json" + corrupt.write_text("{not valid json", encoding="utf-8") + + result = load_registry(corrupt) + + assert result["metadata"].get("id") in (None, ""), ( + "load_registry minted a fresh credential for a registry that already exists — " + "saving that would replace a live project's id" + ) + + +@pytest.mark.parametrize("content", ["", " ", "{malformed"]) +def test_unreadable_variants_all_withhold_a_credential(tmp_path, content): + """Empty and malformed both mean 'cannot read', never 'does not exist'.""" + path = tmp_path / f"X{len(content)}_REGISTRY.json" + path.write_text(content, encoding="utf-8") + + assert load_registry(path)["metadata"].get("id") in (None, "") + + +# ============================================================================= +# CALLER-SUPPLIED CREDENTIAL — one project, one id +# ============================================================================= + + +def test_new_registry_adopts_the_callers_credential(tmp_path): + """The credential already stamped into the passport is the one that lands. + + Regression guard for a double mint: load_registry mints an id for a missing + file, so an "is the id already set?" check would see that fresh mint and + silently discard the caller's — leaving the passport claiming one credential + and the registry file carrying a different one. + """ + from aipass.spawn.apps.handlers.registry import add_to_registry + + reg = tmp_path / "NEW_REGISTRY.json" + branch = tmp_path / "widget" + branch.mkdir() + stamped = str(uuid.uuid4()) + + add_to_registry(reg, "WIDGET", str(branch), "p", "@widget", credential=stamped) + + assert json.loads(reg.read_text(encoding="utf-8"))["metadata"]["id"] == stamped + + +def test_existing_registry_credential_survives_a_new_citizen(tmp_path): + """Registering into a live project never re-credentials it.""" + from aipass.spawn.apps.handlers.registry import add_to_registry + + reg = tmp_path / "REAL_REGISTRY.json" + reg.write_text( + json.dumps({"metadata": {"id": "the-real-lock", "version": "1.0.0", "total_branches": 0}, "branches": []}), + encoding="utf-8", + ) + branch = tmp_path / "widget" + branch.mkdir() + + add_to_registry(reg, "WIDGET", str(branch), "p", "@widget", credential=str(uuid.uuid4())) + + assert json.loads(reg.read_text(encoding="utf-8"))["metadata"]["id"] == "the-real-lock" From 2b7e6bcc0f612df06e4cc68624c49b70e69842f4 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 20:50:35 -0700 Subject: [PATCH 07/43] fix(drone): owner-tier git refusals carry their species, and the warn-mode rollback stops lifting a refusal outside its scope (@drone's build, committed by devpulse). THE PAGE THAT STARTED IT was not a fault: BAUD ran pr in projects/baud, cleared all four owner-tier authority checks, and hit the untranslated-verb wall - pr encodes our dev->PR->main flow and is not translated for external repos, so refusing is right. But verify_git_access logged EVERY owner-tier refusal at ERROR, designed ones included: the whole auth.log read 77 lines, 74 designed WARNINGs, 3 ERRORs, all three designed refusals - ERROR in that file had a 0% true-fault rate, which trains everyone to ignore the page. THE HOLE UNDERNEATH, severity medium and the reason this was more than a log-level nudge: AIPASS_GIT_AUTH_MODE=warn is the rollback for the AUTHORITY migration (F59 6.1), tested once against every refusal species - so it ALSO lifted the untranslated-verb refusal. Proven live before the fix: under warn mode, pr in BAUD's repo returned ALLOWED. A rollback scoped to ownership checks was silently re-arming a half-run of our merge flow inside someone else's repository; it requires a deliberate env var, but an ambient credential-shaped env var inherited by a process is exactly how that bites, and nobody would find it by reading the flag's name. THE FIX: authority refusals (not this repo's proven owner) stay ERROR and warn-mode lifts them - that is the switch's job; capability refusals (IS the proven owner, verb untranslated for this repo) log WARNING and warn-mode does NOT touch them. Message honesty corrected with it: a proven owner now reads 'cannot run pr in this repo', not 'is not authorized' - the old wording sent a manager with a clean passport off to audit a passport that was never the problem. Bar: 6 tests, 3 mutations each biting only its own test, 1200 passed / 5 skipped (re-run independently by devpulse), seedgo 34/34 files, branch audit 100%, 0 type errors, auth.py 1.0.1 -> 1.1.0. Deliberately NOT touched, policy not repair: which verbs get translated for external repos, and whether a non-manager's refusal keeps paging - both queued into the single untrack + external-repo-lane design session awaiting Patrick's ruling, tonight being the third incident in 48h of that one family. --- .../drone/apps/plugins/devpulse_ops/auth.py | 110 +++++++++++++----- src/aipass/drone/tests/test_git_access.py | 41 +++++++ 2 files changed, 125 insertions(+), 26 deletions(-) diff --git a/src/aipass/drone/apps/plugins/devpulse_ops/auth.py b/src/aipass/drone/apps/plugins/devpulse_ops/auth.py index bb8854444..ccf60dc88 100644 --- a/src/aipass/drone/apps/plugins/devpulse_ops/auth.py +++ b/src/aipass/drone/apps/plugins/devpulse_ops/auth.py @@ -1,9 +1,9 @@ # =================== AIPass ==================== # Name: auth.py # Description: Passport-based authorization for devpulse operations -# Version: 1.0.1 +# Version: 1.1.0 # Created: 2026-03-30 -# Modified: 2026-08-11 +# Modified: 2026-08-24 # ============================================= """Passport-based authorization for git operations. @@ -98,6 +98,19 @@ # hits one of the above learns what they can use instead of guessing. _TRANSLATED_VERBS = "'commit', 'sync' and 'tag'" +# Owner-tier refuses for two reasons that are NOT interchangeable, and conflating +# them cost us both a false page and a real hole: +# authority — the caller is not, or cannot be shown to be, this repo's owner. +# Fault-shaped, so it logs ERROR, and warn-mode DOES lift it — +# rolling back the authority migration is exactly that switch's job. +# capability — the caller IS the proven owner; the verb simply is not translated +# for this repo yet (_AIPASS_FLOW_VERBS). Not a fault, so it logs +# WARNING for the same reason _resolve_caller does, and warn-mode +# must NOT lift it: this refusal predates that migration and exists +# to stop a half-run in someone else's repo, which no rollback wants. +_AUTHORITY = "authority" +_CAPABILITY = "capability" + # Real git verbs drone deliberately does not expose — staging and remote work are # folded into higher-level commands. Without a pointer the refusal is a dead end. @@ -128,6 +141,18 @@ class Caller(NamedTuple): passport: dict +class Refusal(NamedTuple): + """Why owner-tier was refused, and which species of refusal it is. + + ``kind`` is load-bearing, not decoration: it selects the log severity and + decides whether the warn-mode rollback may lift the refusal. See _AUTHORITY + and _CAPABILITY above. + """ + + reason: str + kind: str + + def _resolve_caller() -> Caller: """Walk up from CWD to find passport.json and return the caller's identity. @@ -228,18 +253,25 @@ def _recorded_home(entry: dict, repo_root: Path) -> Path | None: return None -def _owner_tier_refusal(command: str, caller: Caller) -> str | None: +def _owner_tier_refusal(command: str, caller: Caller) -> Refusal | None: """Return why owner-tier is refused for this caller, or None if authorized. Every branch names the check that refused, so a passport/registry drift is diagnosable from a single log line instead of a bisect. Fails CLOSED: any check that cannot be completed is a refusal, never a silent pass. + + Each refusal also carries its species (_AUTHORITY vs _CAPABILITY), because + the caller has to know whether it is looking at a fault or at an untranslated + verb — they differ in severity and in whether warn-mode may lift them. """ citizen_class = _citizen_class(caller.passport) if citizen_class != _MANAGER_CLASS: - return ( - f"caller '{caller.name}' is citizen_class '{citizen_class or 'unset'}' — " - f"owner-tier requires '{_MANAGER_CLASS}'" + return Refusal( + ( + f"caller '{caller.name}' is citizen_class '{citizen_class or 'unset'}' — " + f"owner-tier requires '{_MANAGER_CLASS}'" + ), + _AUTHORITY, ) try: @@ -251,48 +283,64 @@ def _owner_tier_refusal(command: str, caller: Caller) -> str | None: # Same verdict, named the same way — a refusal must not depend on which # layer happened to notice first. logger.warning("owner-tier refused for '%s': registry credential mismatch: %s", caller.name, exc) - return f"caller '{caller.name}' does not hold citizenship in this project's registry ({exc})" + return Refusal( + f"caller '{caller.name}' does not hold citizenship in this project's registry ({exc})", _AUTHORITY + ) except Exception as exc: logger.warning("owner-tier refused for '%s': registry unreadable: %s", caller.name, exc) - return f"the project registry could not be read ({exc}) — cannot verify ownership" + return Refusal(f"the project registry could not be read ({exc}) — cannot verify ownership", _AUTHORITY) registry_id = registry_data.get("metadata", {}).get("id") passport_id = caller.passport.get("citizenship", {}).get("registry_id") if not registry_id: - return f"registry {registry_path.name} declares no metadata.id — cannot verify tenancy" + return Refusal(f"registry {registry_path.name} declares no metadata.id — cannot verify tenancy", _AUTHORITY) if not passport_id: - return ( - f"caller '{caller.name}' passport has no citizenship.registry_id — " - "needs a registry backfill before it can hold owner-tier" + return Refusal( + ( + f"caller '{caller.name}' passport has no citizenship.registry_id — " + "needs a registry backfill before it can hold owner-tier" + ), + _AUTHORITY, ) if passport_id != registry_id: - return ( - f"caller '{caller.name}' belongs to registry {passport_id}, but this repo is " - f"{registry_id} — a manager of one project holds nothing in another" + return Refusal( + ( + f"caller '{caller.name}' belongs to registry {passport_id}, but this repo is " + f"{registry_id} — a manager of one project holds nothing in another" + ), + _AUTHORITY, ) entry = _registry_entry(registry_data, caller.name) if entry is None: - return f"caller '{caller.name}' is not listed in {registry_path.name}" + return Refusal(f"caller '{caller.name}' is not listed in {registry_path.name}", _AUTHORITY) if entry.get("owner") is not True: - return f"caller '{caller.name}' is listed in {registry_path.name} without owner: true" + return Refusal(f"caller '{caller.name}' is listed in {registry_path.name} without owner: true", _AUTHORITY) # The registry file's own directory is the repo root by construction, which # keeps path-binding anchored to the SAME registry the checks above used. repo_root = registry_path.parent.resolve() recorded = _recorded_home(entry, repo_root) if recorded is None: - return f"registry entry for '{caller.name}' records no path — cannot bind authority to a location" + return Refusal( + f"registry entry for '{caller.name}' records no path — cannot bind authority to a location", _AUTHORITY + ) if caller.home != recorded and recorded not in caller.home.parents: - return ( - f"caller '{caller.name}' presented a passport from {caller.home}, but the registry " - f"binds that name to {recorded} — a passport outside its recorded home proves nothing" + return Refusal( + ( + f"caller '{caller.name}' presented a passport from {caller.home}, but the registry " + f"binds that name to {recorded} — a passport outside its recorded home proves nothing" + ), + _AUTHORITY, ) + # Reached only once all four authority checks have PASSED, so the caller is a + # proven owner and this is purely a missing translation — hence _CAPABILITY. if registry_path.name != AIPASS_REGISTRY_NAME and command in _AIPASS_FLOW_VERBS: - return ( + return Refusal( f"'{command}' encodes AIPass's own dev→PR→main flow and is not translated for " - f"external repos yet — {_TRANSLATED_VERBS} work here today (DPLAN-0281 P2, DPLAN-0290)" + f"external repos yet — {_TRANSLATED_VERBS} work here today (DPLAN-0281 P2, DPLAN-0290)", + _CAPABILITY, ) return None @@ -329,8 +377,18 @@ def verify_git_access(command: str) -> str: caller_info = _resolve_caller() refusal = _owner_tier_refusal(command, caller_info) warn_only = os.environ.get(_AUTH_MODE_ENV, "").strip().lower() == "warn" + if refusal and refusal.kind == _CAPABILITY: + # A proven owner meeting an untranslated verb. Not a fault, so it warns + # rather than pages — and warn_only is deliberately not consulted: the + # authority rollback has no business re-arming a half-run in someone + # else's repo. Says "cannot run", not "not authorized", because this + # caller IS authorized and sending them to audit their passport would + # be a false trail. + msg = f"Branch '{caller_info.name}' cannot run '{command}' in this repo: {refusal.reason}." + logger.warning(msg) + raise PermissionError(msg) if refusal and not warn_only: - msg = f"Branch '{caller_info.name}' is not authorized for '{command}': {refusal}." + msg = f"Branch '{caller_info.name}' is not authorized for '{command}': {refusal.reason}." logger.error(msg) raise PermissionError(msg) if refusal: @@ -340,7 +398,7 @@ def verify_git_access(command: str) -> str: "git auth warn-mode: '%s' for '%s' would be denied under enforcement: %s", command, caller_info.name, - refusal, + refusal.reason, ) json_handler.log_operation( "git_access_verify", @@ -349,7 +407,7 @@ def verify_git_access(command: str) -> str: "command": command, "tier": "owner", "mode": "warn" if warn_only else "enforce", - "would_refuse": refusal, + "would_refuse": refusal.reason if refusal else None, }, ) return caller_info.name diff --git a/src/aipass/drone/tests/test_git_access.py b/src/aipass/drone/tests/test_git_access.py index d4c6fc4af..6e863068a 100644 --- a/src/aipass/drone/tests/test_git_access.py +++ b/src/aipass/drone/tests/test_git_access.py @@ -493,6 +493,32 @@ def test_same_verbs_work_in_aipass(self, tmp_path: Path, monkeypatch: pytest.Mon monkeypatch.chdir(tmp_path) assert verify_git_access(command) == "devpulse" + @pytest.mark.parametrize("command", ["dev-pr", "pr", "merge", "fix"]) + def test_warn_mode_does_not_lift_an_untranslated_verb( + self, vera_home: Path, monkeypatch: pytest.MonkeyPatch, command: str + ) -> None: + """Warn-mode rolls back the AUTHORITY migration (F59 6.1), never this refusal. + + The two were entangled: warn-mode tested one flag for every refusal, so the + authority rollback also re-armed a half-run of OUR dev→PR→main flow inside + someone else's repo — the exact mess _AIPASS_FLOW_VERBS exists to prevent. + Third scope limit on warn-mode, beside identification. + """ + monkeypatch.setenv("AIPASS_GIT_AUTH_MODE", "warn") + with pytest.raises(PermissionError, match="not translated for external repos"): + verify_git_access(command) + + def test_untranslated_refusal_does_not_call_a_proven_owner_unauthorized(self, vera_home: Path) -> None: + """VERA cleared all four authority checks — 'not authorized' would be a false trail. + + The wording decides where the reader goes next: audit a passport that is + fine, or wait for the verb to be translated. + """ + with pytest.raises(PermissionError) as exc_info: + verify_git_access("pr") + assert "not authorized" not in str(exc_info.value) + assert "cannot run 'pr'" in str(exc_info.value) + class TestGitAuthWarnMode: """One env var is the rollback (F59 6.1): warn logs the refusal and allows.""" @@ -588,6 +614,21 @@ def test_owner_tier_denial_still_logs_error(self, seedgo_dir: Path) -> None: verify_git_access("commit") mock_logger.error.assert_called_once() + def test_untranslated_verb_warns_instead_of_paging(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """A proven owner meeting an untranslated verb is a capability gap, not a fault. + + Live incident (@trigger error 38439475, 2026-08-24): BAUD ran `pr` in its own + repo, cleared every authority check, hit the untranslated-verb wall — and the + refusal logged ERROR, paging @drone about a gate working exactly as designed. + """ + home = make_owner_project(tmp_path, branch="BAUD", registry_name="BAUD_REGISTRY.json") + monkeypatch.chdir(home) + with patch("aipass.drone.apps.plugins.devpulse_ops.auth.logger") as mock_logger: + with pytest.raises(PermissionError, match="not translated for external repos"): + verify_git_access("pr") + mock_logger.warning.assert_called_once() + mock_logger.error.assert_not_called() + def test_git_module_does_not_duplicate_denial_error(self, repo_dir: Path) -> None: """git_module re-logs the denial at WARNING — auth.py already logged it authoritatively.""" with ( From 6cf01d649ea3fb878454f8511f0bbaa399f2b4f1 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Mon, 24 Aug 2026 20:50:59 -0700 Subject: [PATCH 08/43] fix(devpulse)+docs(changelog): feedback's cross-store writes speak the host's dialect, and the changelog train section rides with the train it documents (devpulse's own lane). THE FIX: compose.py 1.3.1 - _deliver_to_ai_mail stamps ai_mail's canonical local format (%Y-%m-%d %H:%M:%S, their create.py and per-user timestamp_format) into recipients' inbox.json instead of UTC ISO-T. Found through BAUD's bug report (feedback e8c235f8): his store held two timestamp shapes, ai_mail's own beside this module's. Measured honestly on both sides before fixing: the mixed formats were NOT the cause of his dead inbox - that was ai_mail's relative-row resolution defect, fixed in 90d6723f, and BAUD's post-fix listing returning all five messages with both formats intact refuted the parser theory by direct evidence. Real pollution regardless: a guest writing into another module's store writes in the HOST's format, not its own. The module's OWN feedback store keeps ISO - internal, unshared, its display reads it raw. Red-first test pins both properties: parses in the canonical shape (raises on ISO-T) and reads as local wall-clock, not a UTC stamp wearing local clothes - a UTC value in that format sits a whole offset from now, which is the quiet variant of the same lie. 70 feedback tests green. Existing ISO rows in baud's inbox.json left as-is: ai_mail reads them fine, and rewriting another citizen's store is a migration to be asked for, not a tidy to slip in. DOCS: CHANGELOG gains the post-v2.7.19 in-progress section carrying this train's five entries (ai_mail rooted rows, spawn credential mint, drone severity split, this fix, and the header restore), and the v2.7.17 section header is restored at its splice point - VERA's find (feedback 028e951f): the v2.7.18 merge glued the header's title mid-line onto a README paragraph, leaving GitHub tag v2.7.17 with no matching section, a released version vanished from the record. Kept the deliberate retitle (CI green campaign) over the 08-19 original (first green board) since the reword and the loss arrived in the same merge and only the loss was the accident. Header sequence verified 2.7.19/18/17/16. CONTEXT FOR THE TRAIN, one line each: the night started as Patrick's BAUD resume probe (cold resume PASSED - code word answered from continued context), detoured through the lane defect that had swallowed BAUD's first answer, and closed with the fix live-verified from BAUD's own seat before anything was committed - resolved and tested before commit, per Patrick's ruling. --- CHANGELOG.md | 55 ++++++++++++++++++- .../apps/handlers/feedback/compose.py | 11 +++- .../devpulse/tests/test_feedback_compose.py | 29 ++++++++++ 3 files changed, 91 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ec4de85eb..074f82d1f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,57 @@ PyPI version — not the changelog header. --- +## [2026-08-24] — post-v2.7.19 train (in progress) + +**fix(ai_mail)** — registry rows leave the reader ABSOLUTE, rooted against the +registry that answered (`_rooted()` in both lanes of `_lookup_branch_by_name` +and `get_branch_info_from_registry`). The defect: a projects/* citizen's +relative registry row was joined to the AIPass repo root, so BAUD's mailbox +resolved to a phantom dir inside our tree — inbox read empty against a full +store, reply refused an id sitting in the file under his feet, and the lane +FABRICATED the phantom on write: his answer to Patrick's continuity probe was +silently swallowed into `src/baud/` (recovered, then removed via drone rm once +re-sent on the live lane). Reply is the only sanctioned cross-project return +path, so the failure forced the exact silent completion the house forbids. +Three red-first tests including a guard for the six relative AIPass rows; +verified live from BAUD's seat — inbox lists all messages, the id resolves, +and his reply arrived through ai_mail itself. 1326 green. Rider: purge.py's +gap comment updated — @memory shipped `vectorize_and_store` (1.4.0), the seam +is verified live, and the four-month mail-loss loop is closed in practice. + +**feat(spawn)** — a brand-new external project's registry is born WITH its +credential: `load_registry` mints `metadata.id` for a missing file (a registry +that does not exist is a new project) and deliberately does NOT mint for an +unreadable one (a live project whose credential failed to read must never be +re-credentialled — four tests hold the asymmetry). Mint-once ordering moved +into `_spawn_agent`: the passport writes at step 1, the registry at step 4, so +the credential resolves at step 1 and `add_to_registry` adopts it only for a +registry it is CREATING, keyed off file-existed-before-load rather than +id-already-set. Live probe: fresh project mints its own credential, passport +and registry agree, zero AIPass leak. 11 red-first tests, 506 green. Known and +flagged, not fixed here: `add_to_registry` against an unreadable registry +would write the empty schema over the real file — guard awaits its own GO. + +**fix(drone)** — owner-tier git refusals carry their species: authority (not +this repo's owner) stays ERROR and warn-mode lifts it; capability (proven +owner, verb untranslated for external repos) logs WARNING and warn-mode does +NOT lift it — `AIPASS_GIT_AUTH_MODE=warn`, the AUTHORITY-migration rollback, +was proven live re-arming `pr` inside BAUD's repo, a half-run of our flow in +someone else's tree. Message honesty: a proven owner reads "cannot run pr in +this repo", not "is not authorized". 6 tests, 1200 green, auth.py 1.1.0. + +**fix(devpulse)** — feedback's cross-store writes speak the host's dialect: +`compose.py` 1.3.1 stamps ai_mail's canonical local format into recipients' +inbox.json instead of UTC ISO-T (two timestamp shapes in one store, BAUD's +report — measured NOT the cause of his dead inbox, but real pollution). +Red-first test pins format and wall-clock, 70 feedback tests green. + +**docs(changelog)** — the v2.7.17 section header is restored at its splice +point (VERA's find: the v2.7.18 merge glued the header's title onto a README +paragraph mid-line, leaving tag v2.7.17 with no matching section). Kept the +deliberate retitle over the 08-19 original; all released tags have sections +again. + ## [2026-08-23] — v2.7.19: the merge playbook grows teeth **docs(flow)** — README truth check and site parity become hard checkboxes in @@ -258,7 +309,9 @@ to Linux/macOS/Windows (Git Bash or WSL) — both extra platforms run the full suite in CI on every PR. Verified clean and left alone: every badge and link (zero dead, LICENSE genuinely MIT, codecov live at 78%), and the whole Subscriptions & Compliance section — the subprocess claim is backed by code, -zero credential handling in the tree.: one-brain enforced, CI green campaign, fleet perf night, phone file explorer +zero credential handling in the tree. + +## [2026-08-19] — v2.7.17: one-brain enforced, CI green campaign, fleet perf night, phone file explorer **fix(ci)** — the PR #734 green campaign: six owner rounds took the PR's own board from red to 21/21. Main was never red — the merge gate forbids it; these diff --git a/src/aipass/devpulse/apps/handlers/feedback/compose.py b/src/aipass/devpulse/apps/handlers/feedback/compose.py index 5c456fb4a..e792cbaee 100644 --- a/src/aipass/devpulse/apps/handlers/feedback/compose.py +++ b/src/aipass/devpulse/apps/handlers/feedback/compose.py @@ -1,9 +1,9 @@ # =================== AIPass ==================== # Name: compose.py # Description: Compose operations — send feedback and reply to messages -# Version: 1.3.0 +# Version: 1.3.1 # Created: 2026-04-11 -# Modified: 2026-08-07 +# Modified: 2026-08-24 # ============================================= """ @@ -224,7 +224,12 @@ def _deliver_to_ai_mail( logger.warning(f"[FEEDBACK] Failed to read {to_branch} ai_mail inbox: {e}") return False, reason - now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S") + # ai_mail's canonical store format — local wall-clock, space-separated + # (their create.py / user timestamp_format). This module wrote UTC ISO-T + # into the same store, leaving two timestamp shapes in one inbox.json + # (BAUD report e8c235f8, 2026-08-24). A guest writing into another + # module's store writes in the HOST's format, not its own. + now = datetime.now().strftime("%Y-%m-%d %H:%M:%S") mail_id = generate_id() # ai_mail v2 message schema (see ai_mail email/delivery.py) — the viewer diff --git a/src/aipass/devpulse/tests/test_feedback_compose.py b/src/aipass/devpulse/tests/test_feedback_compose.py index c0113cc1f..1c5b5c936 100644 --- a/src/aipass/devpulse/tests/test_feedback_compose.py +++ b/src/aipass/devpulse/tests/test_feedback_compose.py @@ -6,6 +6,7 @@ """Tests for feedback compose — send, reply, ai_mail delivery.""" import json +from datetime import datetime from unittest.mock import patch import pytest @@ -214,6 +215,34 @@ def test_delivers_to_ai_mail(self, inbox_with_message, mock_aipass_root): assert mail_msg["metadata"]["source"] == "feedback" assert mail_msg["metadata"]["thread_id"] == "aaa11111" + def test_delivered_timestamp_matches_ai_mail_store_format(self, inbox_with_message, mock_aipass_root): + """Delivered timestamp must use ai_mail's canonical store format. + + ai_mail stamps its store with local '%Y-%m-%d %H:%M:%S' (create.py, + reply.py). This module wrote UTC ISO-T into that same store, leaving + two timestamp shapes in one file (BAUD report e8c235f8, 2026-08-24). + One store, one format — and local wall-clock, not UTC wearing it. + """ + ai_mail_dir = mock_aipass_root / "seedgo" / ".ai_mail.local" + ai_mail_dir.mkdir(parents=True) + ai_mail_inbox = ai_mail_dir / "inbox.json" + with open(ai_mail_inbox, "w", encoding="utf-8") as f: + json.dump( + {"mailbox": "inbox", "total_messages": 0, "unread_count": 0, "messages": []}, + f, + ) + + compose.reply_to("aaa11111", "format check") + + with open(ai_mail_inbox, encoding="utf-8") as f: + ts = json.load(f)["messages"][0]["timestamp"] + + # Parses in ai_mail's canonical shape (raises on ISO-T)... + parsed = datetime.strptime(ts, "%Y-%m-%d %H:%M:%S") + # ...and reads as local wall-clock — a UTC stamp in local clothes + # would sit a whole UTC-offset away from now. + assert abs((parsed - datetime.now()).total_seconds()) < 60 + def test_skips_delivery_when_no_ai_mail(self, inbox_with_message, mock_aipass_root): """Should log warning and skip when sender has no ai_mail inbox.""" # Don't create sender's ai_mail directory From ac721b2709f3f551b19f8dd6872eaeee3ba622b1 Mon Sep 17 00:00:00 2001 From: AIOSAI Date: Tue, 25 Aug 2026 02:54:34 -0700 Subject: [PATCH 09/43] docs(fleet): the README truth campaign - 18 branch READMEs verified by their own citizens against the code as it exists, the 08-25 night shift (Patrick's GO before bed, executed solo on wake-backs). THE RULE: no guessing - view the actual code, run your real commands, and every number written tonight is a number measured tonight; what could not be verified is marked unverified IN the README rather than standing green. THE VERDICT: Patrick's distrust was justified everywhere - roughly 120 wrong claim families across 20 passes (16 core seats + devpulse self + the four resident projects in their own repos, uncommitted there for his review). HEADLINES: commons' README documented a Reward Drops mechanic (10% artifact chance on posting) that never existed in any code - the purest overclaim specimen; baud's README carried a false safety claim (26 commands listed as the complete backend surface while generate_handler! held 29 - three write-capable memory commands invisible to an attack-surface audit); aipass' own Quick Start told new users to run bare init, which prints help (the real verb is init run - the front door stranded its reader at step 1); and the spawn project_agent template ships contradictions to every child it mints: a --version its generated code never implements, a phantom system_logger name x3, an identical silent except-ImportError-continue at entry-point line 35 x3. THE PATTERN, held across all 20: docs rot BOTH directions - dead features documented live AND real features plus long-fixed debt still listed open. Both poison a post-reset agent equally, which is why this campaign was the precondition for DPLAN-0318's fleet memory reset: the announcement email will say current state + code = truth, and tonight makes that sentence true. DISCIPLINE HELD: docs-only - every code defect found (watchdog killed-or-True, warning-exit-0 family, commons help identity chain, git gate matching command TEXT inside heredoc content, the template trio) was flagged to its owner and queued, never smuggled into a docs commit; each branch edited only its own README + .trinity; every pass re-audited 100% after edits; seedgo readme gates green where a lane exists. The campaign log with all 20 reports, the code-fix queue, and the rulings Patrick owes rides in devpulse dropbox/readme_night_shift.md. Deliberately NOT in this commit: memory/templates/*.template.json (DPLAN-0318 live WIP with Patrick) and every project-repo file. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 18 ++++ src/aipass/ai_mail/README.md | 135 +++++++++++++++++++++++----- src/aipass/aipass/README.md | 87 +++++++++++------- src/aipass/api/README.md | 25 +++--- src/aipass/backup/README.md | 38 ++++++-- src/aipass/canary/README.md | 43 +++++++-- src/aipass/cli/README.md | 22 +++-- src/aipass/commons/README.md | 124 +++++++++++++++----------- src/aipass/daemon/README.md | 57 ++++++++---- src/aipass/devpulse/README.md | 43 ++++++--- src/aipass/drone/README.md | 67 ++++++++++---- src/aipass/flow/README.md | 129 +++++++++++++++++++-------- src/aipass/hooks/README.md | 57 ++++++++---- src/aipass/memory/README.md | 129 +++++++++++++++++++++------ src/aipass/prax/README.md | 68 +++++++++++---- src/aipass/seedgo/README.md | 160 +++++++++++++++++++++------------- src/aipass/skills/README.md | 32 +++++-- src/aipass/spawn/README.md | 142 +++++++++++++++++++----------- src/aipass/trigger/README.md | 120 +++++++++++++++++++------ 19 files changed, 1074 insertions(+), 422 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 074f82d1f..13d8d15d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,24 @@ PyPI version — not the changelog header. ## [2026-08-24] — post-v2.7.19 train (in progress) +**docs(fleet)** — README truth campaign, the 08-25 night shift (Patrick's GO): +every citizen verified its OWN README against the code as it exists, waves of +two, each pass re-audited to 100% after editing — 18 branch READMEs corrected +in this repo (~120 claim families), plus the four resident projects in their +own repos. The rule was measured-or-marked: every number rewritten tonight was +counted tonight, and what couldn't be verified is now labeled unverified in the +README itself instead of standing green. Headlines: commons documented a +Reward-Drops feature that never existed in any code; baud's command-registry +safety claim was false (26 listed, 29 registered — three write-capable commands +invisible to an audit); aipass's own Quick Start pointed new users at bare +`init`, which prints help; two project front doors misrepresented finished +products as empty templates. Fleet pattern, both directions: dead features +documented live AND real features/fixed debt undocumented — either would +poison a post-reset agent, which is why this campaign was the precondition for +the DPLAN-0318 fleet memory reset. Code defects found were flagged to owners, +never smuggled into docs-only edits; the queue rides in devpulse's campaign +log (dropbox/readme_night_shift.md) for the morning read. + **fix(ai_mail)** — registry rows leave the reader ABSOLUTE, rooted against the registry that answered (`_rooted()` in both lanes of `_lookup_branch_by_name` and `get_branch_info_from_registry`). The defect: a projects/* citizen's diff --git a/src/aipass/ai_mail/README.md b/src/aipass/ai_mail/README.md index 00599301a..c79dbf9a4 100644 --- a/src/aipass/ai_mail/README.md +++ b/src/aipass/ai_mail/README.md @@ -5,11 +5,11 @@ **Purpose:** Inter-agent messaging for AIPass. File-based email system that lets agents send, receive, and process messages using `@branch` addresses. No SMTP, no external services — just JSON files and symbolic routing. **Module:** `aipass.ai_mail` **Created:** 2025-11-08 -**Last Updated:** 2026-08-18 +**Last Updated:** 2026-08-25 --- -**Status:** Operational | **Seedgo:** 100% (99% with every bypass rule off) | **Tests:** 1326 pass (1322 + 4 live-hygiene skips on a fresh checkout) | **Battle Tested:** S62 +**Status:** Operational | **Seedgo:** 100% | **Tests:** 1326 pass across 46 files (1322 + 4 live-hygiene skips on a fresh checkout — 2 in `test_live_mailbox_hygiene.py`, 2 in `test_live_contacts_hygiene.py`) | **Battle Tested:** S62 ## Quick Start @@ -36,12 +36,18 @@ drone @ai_mail dispatch @target "Subject" "Body" # Dispatch (send + wake in one step) drone @ai_mail dispatch @target "Subject" "Body" # Send dispatch email + wake drone @ai_mail dispatch @target "Subject" "Body" --fresh # Send + fresh wake (new session) +drone @ai_mail dispatch @t "Subj" "Body" --model sonnet # Wake on a named model drone @ai_mail dispatch wake @target # Wake only (no email) +drone @ai_mail dispatch register # What is outstanding, what is overdue +drone @ai_mail dispatch status # Last 5 spawns (see Known Issues) +drone @ai_mail dispatch daemon # Start the polling dispatch daemon # Send mail (no wake) drone @ai_mail email @target "Subject" "Body" # Send to one branch drone @ai_mail email @all "Subject" "Body" # Broadcast to all branches drone @ai_mail email @target "Subj" "Body" --from @spawn # Explicit sender override +drone @ai_mail email @t "Subj" "Body" --reply-to @x # Route replies elsewhere +drone @ai_mail email @t "Subj" "Body" --upsert-key KEY # Repeat signal, one slot (below) # Read mail drone @ai_mail inbox # List all emails (new + opened) @@ -363,7 +369,10 @@ in the body, as evidence, not as a route. ## Help Flags — Explain, Never Execute A help flag anywhere in the argument list means *describe this command*, and all three -modules check for it as the first statement in `handle_command`, before anything routes. +modules check for it as the first thing after the command-ownership guard in +`handle_command` — before any argument is read and before anything routes. It is not +literally statement one, and cannot be: a module has to establish the command is *its* +before it may answer for it (`email.py`, `dispatch.py`, `email_send.py`). They used to gate help at `args[0]` only, so a flag one position later was discarded and the command ran instead. On a messaging branch that is not a cosmetic bug: @@ -580,6 +589,10 @@ Out of scope: @baud is a citizen of hosted project 'baud', not the AIPass fleet existing message from @baud, or use the feedback channel. ``` +The branch count is **computed, not written** — `len(branches)` for the caller's own +scope, handed to `_describe_unresolved_address()`. It reads 18 today and drifts with the +fleet; the number above is an example of the shape, never a constant. + - **The refusal is unchanged — only the reason is now true.** Fleet→project initiation stays walled for every non-admin caller, exit `2`, sent record stamped `refused`. - **`_describe_unresolved_address()` runs on the failure path only**, so a successful @@ -630,18 +643,73 @@ The polling daemon (`daemon.py`) watches inboxes for `auto_execute` dispatch ema ## Sender Identity -Branch identity detection follows a priority chain in `detect_branch_from_pwd()`: - -1. `AIPASS_CALLER_BRANCH` env var (set by drone router from passport or `AIPASS_BRANCH_NAME`) -2. Contacts address book lookup (fastest path for registered branches) -3. Registry lookup by name -4. `AIPASS_CALLER_CWD` / `Path.cwd()` walk-up to find `.trinity/passport.json` -5. Registry lookup by path - -If all fail, detection returns `None` and the operation fails loudly. Wrong identity is worse than no identity. +Branch identity detection runs in `detect_branch_from_pwd()`. It is **not** a flat +waterfall — a fence runs first, and the env-var lane and the walk-up lane are +alternatives, not neighbours. + +**0. The identity fence.** `AIPASS_CALLER_CWD` set but standing outside any branch → +refused outright, *unless* `AIPASS_CALLER_IDENTITY_SOURCE` is `assigned` or `passport`. +@drone stamps which kind of evidence named the caller, and a credential travels where a +location does not. `project` — a registry-derived *project* name — answers "which project +am I in", never "who am I", and stays refused: that is the $1.41 wake, where drone +standing at the repo root stamped `aipass` the directory, which spells the same as +`@aipass` the citizen. An **absent** `AIPASS_CALLER_CWD` is not contradicting evidence and +leaves everything below untouched — in-process callers depend on that. + +**1. `AIPASS_CALLER_BRANCH` is set** → registry lookup by name, **then** contacts, then an +identity synthesized from the env vars alone (recorded `unverified` — no passport, no +registry row). + +> **The registry is asked before contacts, and the order is the whole fix.** +> `AIPASS_REGISTRY.json` is the authoritative catalog; `contacts.json` is a learned, +> writable cache. Asking the cache first let one poisoned row outrank the catalog for a +> citizen the catalog knew perfectly well — found live 2026-08-23, when +> `drone @ai_mail inbox` served @flow's mailbox from inside @ai_mail's own directory, +> logged as name AI_MAIL / email @ai_mail / path `.../flow`, confidence **verified**. +> The `is_dir()` staleness guard could never have caught it: the wrong root was a live +> branch with a real mailbox in it. Contacts keep their real job — resolving external +> callers the registry has never heard of. + +**2. No `AIPASS_CALLER_BRANCH` at all** → walk up `AIPASS_CALLER_CWD` (or, with no caller +env, this process's `Path.cwd()`) for `.trinity/passport.json`, then registry lookup by +path. The `Path.cwd()` leg is recorded `unverified` deliberately: it is correct for a +dispatched agent standing in its own tree and silently wrong anywhere else. + +Every exit is stamped by `_record_resolution()` with the winning strategy and a +confidence, so a wrong sender can be traced to the path that produced it. If all fail, +detection returns `None` and the operation fails loudly. Wrong identity is worse than no +identity. The `--from @branch` flag on send/email commands provides an explicit sender override for callers outside branch directories. +### Registry Rows Leave the Reader Absolute + +**A registry row's `path` is relative to THE REGISTRY THAT HOLDS IT.** Returned raw it +carries no memory of which registry answered, and every consumer then joins it to the +AIPass repo root — right for AIPass citizens by coincidence, wrong for every project +citizen. + +`_rooted()` absolutises a row against its own registry at the point of read, in both lanes +of `_lookup_branch_by_name()` and both of `get_branch_info_from_registry()`. Rows already +absolute pass through untouched. It lives at the reader rather than at the nine call sites +that join a registry path, because a consumer cannot re-derive a root it was never given — +and nine copies of that join is how they drift. + +**It fabricated rather than failing, which is why this is a rule and not a footnote.** +Found live 2026-08-24: a `projects/*` citizen read *"Inbox is empty"* against a file +holding four unread messages, and `reply ` answered *"Message not found"* for an id +read out of that same file. `projects/baud` + row `src/baud/baud` had resolved to +`/src/baud/baud`. That path sits **inside** the AIPass tree, so the mail lane +created it — a phantom `.ai_mail.local/` holding a reply its author believed he had sent, +in a directory belonging to no citizen. A refusal would have been loud; a confident wrong +address was not. + +The caller-registry fallback in `_lookup_branch_by_name()` is **not** admin-gated, and is +a different question from the admin-only cross-project sweep above: it resolves a citizen +of the caller's *own* project. It cannot reach @baud from a fleet seat — walking up from a +fleet citizen's `AIPASS_CALLER_CWD` finds `AIPASS_REGISTRY.json` first, which does not +list him. + ### Verified-Caller Rail `--from` and `--sender` are **claims, not credentials**. Both land on @@ -736,6 +804,9 @@ ai_mail/ │ │ ├── daemon.py # Polls inboxes, spawns agents for dispatch emails │ │ ├── wake.py # Wakes branches via claude subprocess │ │ ├── dispatch_monitor.py # Wraps claude process (bounce, lock cleanup, sandbox, broker fd) +│ │ ├── register.py # Append-only dispatch register — open/close/outstanding +│ │ ├── report.py # Completion report — build/write, emails_sent, memories_edited +│ │ ├── session_pointer.py # Durable resume-session pointer (replaces `claude -c`'s mtime guess) │ │ ├── status.py # Dispatch log I/O │ │ └── test_token.py # AIPASS-TEST ping protocol (auto-ack) │ ├── cli/ @@ -754,7 +825,7 @@ ai_mail/ │ ├── paths.py # Shared find_repo_root() utility │ ├── notify.py # Notification feed writer (JSONL, BAUD reads) │ └── central_writer.py # Central inbox stats aggregation -└── tests/ # 1326 tests across 46 test files +└── tests/ # 1326 tests across 46 test files (selection below) ├── conftest.py # Shared fixtures (mock_logger, mock_json_handler) ├── test_daemon.py # Daemon config, state, kill switch, dispatch check ├── test_dispatch_monitor.py # Monitor safety features, env stripping @@ -762,8 +833,9 @@ ai_mail/ ├── test_wake.py # Branch resolution, PID checks, lock files ├── test_wake_blocklist.py # Wake protection for @devpulse ├── test_delivery.py # Inbox migration, private branches, pipeline - ├── test_send_identity.py # Sender identity chain (36 tests) - ├── test_user_paths.py # Mailbox path resolution (13 tests) + ├── test_send_identity.py # Sender identity chain (62 tests) + ├── test_identity_fence.py # Every verb refuses outside a branch + ├── test_user_paths.py # Mailbox path resolution (22 tests) ├── test_contacts.py # Address book operations ├── test_inbox_ops.py # Inbox loading + migration ├── test_registry_read.py # Registry parsing + branch lookup @@ -778,6 +850,12 @@ ai_mail/ ├── test_public_surface.py # Package doors: feed_path, register_path, outstanding (22 tests) ├── test_message_correlation.py # sent_id back-reference + shared id resolver (12 tests) ├── test_live_mailbox_hygiene.py # Guard: no test fixtures in real mailboxes (4 tests) + ├── test_live_contacts_hygiene.py # Guard: no tmp paths in live contacts.json (3 tests) + ├── test_dispatch_register.py # Append-only register, later-record-wins reconstruction + ├── test_dispatch_report.py # Completion report contents + durability + ├── test_session_pointer.py # Resume pointer (47 tests) + ├── test_admin_lane.py # 5-leg admin verdict on the wake path + ├── test_cross_project_bridge.py # Admin-only resolution + reply return path └── test_paths.py # find_repo_root() utility ``` @@ -786,9 +864,14 @@ ai_mail/ ### Depends On - `aipass.prax` — Logging via `system_logger` - `aipass.cli` — Console output and display formatting -- `aipass.drone` — Command routing and `@branch` resolution -- `aipass.trigger` — `trigger.fire()` for `email_dispatched` events -- Python stdlib (`pathlib`, `json`, `argparse`, `importlib`, `subprocess`, `fcntl`) +- `aipass.drone` — broker-socket IPC for a sandboxed dispatch child + (`create_identified_connection`). This is the **only** `aipass.drone` import in the + package: `@branch` resolution is internal (`registry/read.py`, `users/branch_detection.py`), + and "drone routes commands to us" describes how `drone @ai_mail …` invokes this branch + from outside, not a dependency +- `aipass.trigger` — `trigger.fire()` for `email_dispatched` / `dispatch_completed` events +- Python stdlib (`pathlib`, `json`, `importlib`, `subprocess`, `fcntl`) — argument parsing + is hand-rolled in `send_args.py`, not `argparse` ### Provides To - **All branches** — inter-branch messaging (send/receive/reply/close) @@ -799,9 +882,13 @@ ai_mail/ ## Bypass Registry -`.seedgo/bypass.json` holds **18** rules — 17 survivors of the prune below, plus one added -the same day *with* `cli/help_flags.py` and measured live (99% with it off, 100% on), which -is the opposite of a prune candidate. +`.seedgo/bypass.json` holds **20** rules — 17 survivors of the prune below, one added the +same day *with* `cli/help_flags.py` and measured live (99% with it off, 100% on, the +opposite of a prune candidate), plus two added since: `handlers` on +`dispatch/report.py` (FPLAN-0452 P1) and `unused_function` on the package `__init__.py` +(S154, re-checked against the built wire). The 99%-with-everything-off figure was measured +on 2026-08-13 against the 18-rule registry and has **not** been re-run since — the 100% +audit score below is current, that one is dated. It held 51 until the 2026-08-13 audit measured every one of them in **both** lanes — the audit lane (`audit aipass @ai_mail --full`, walks @@ -834,6 +921,12 @@ re-measure per lane. - **`--from` is undocumented in `email --help`** — it is in this README and in the code, but not in the module's own FLAGS block. Open in APLAN-0006. - **`--model` help names retired models** ("Claude Opus 4.6", "Sonnet 4.6"). Open in APLAN-0006. +- **`dispatch status` reports "No dispatches recorded yet." while dispatches are running.** + Reproduced 2026-08-25 with two live entries visible in `dispatch register` at the same + moment. The register is the trustworthy view; `status` reads a different log and its + empty answer is a false negative, not an empty state. Open in APLAN-0006. +- **`dispatch wake` prints "see step status above"** when the step status prints below it + (`dispatch.py`). Named under *Output ordering*; open in APLAN-0006. --- diff --git a/src/aipass/aipass/README.md b/src/aipass/aipass/README.md index 3e7a068ca..f0982381f 100644 --- a/src/aipass/aipass/README.md +++ b/src/aipass/aipass/README.md @@ -12,7 +12,7 @@ aipass what does drone do # Same — free text falls through to help aipass read drone # Full branch README, rendered in the terminal aipass new myapp --template python # Create a new project aipass adopt myapp --dry-run # Preview adopting an existing projects/ dir -aipass init # Guided setup (10 stages, resumable) +aipass init run # Guided setup (10 stages, resumable) ``` ## Invoke @@ -32,10 +32,10 @@ aipass/ │ │ ├── doctor.py # System health aggregation + cross-OS pre-flight (--cross-os) │ │ ├── _doctor_fix.py # Remediation report (--fix, --json) [internal] │ │ ├── _doctor_wire.py # Auto-wire provider settings + stale-deny re-export [internal] -│ │ ├── handoff.py # CLI handoff — tmux (Linux/Mac) / wt.exe (Windows) session launch +│ │ ├── handoff.py # CLI handoff — thin coordinator, delegates to handoff_platform/ │ │ ├── help_chat.py # README-backed Q&A (reads via readme_map handler) -│ │ ├── init_flow.py # 10-stage guided setup -│ │ ├── install.py # aipass install — one-command bootstrap (clone + setup + init) +│ │ ├── init_flow.py # 10-stage guided setup + update / scaffold / agent forms +│ │ ├── install.py # aipass install — one-command bootstrap (clone + setup + chat) │ │ ├── new_project.py # aipass new — create projects inside the installation │ │ ├── adopt.py # aipass adopt — bring an existing projects/ dir into AIPass │ │ ├── profile.py # User profile read/write @@ -44,20 +44,26 @@ aipass/ │ │ └── feedback.py # Feedback pulse toggle — aipass feedback on/off │ ├── handlers/ │ │ ├── cross_os/ # Cross-OS pre-flight: gap_registry, preflight, run_record -│ │ ├── handoff_platform/ # Platform-specific handoff detection -│ │ ├── init/ # bootstrap.py, scaffold_content.py, git_auth.py -│ │ ├── new_project/ # Project creation logic (registry, template, scaffold, git init) +│ │ ├── handoff_platform/ # OS-dispatched CLI session launch — tmux, wt.exe, inline +│ │ ├── init/ # bootstrap.py, git_auth.py (re-exports shared/scaffold_content.py) +│ │ ├── new_project/ # Project creation logic (registry, template, scaffold, repo init) │ │ │ └── adopt.py # Project adoption logic (additive scaffold onto an existing dir) -│ │ ├── json/ # JSON read/write utilities +│ │ ├── json/ # Branch-local shim — delegates to shared/json_handler.py +│ │ ├── help_flag.py # wants_help() — --help detection in any argv position │ │ ├── ping_sweep/ # Branch reachability verification │ │ ├── provider_reconcile.py # Stale deny-rule detection + fix +│ │ ├── provider_wire.py # Provider settings wiring (used by doctor --fix) │ │ ├── readme_map/ # Live file reads + branch routing +│ │ ├── sandbox_check/ # Sandbox / containment detection │ │ ├── structure_scan/ # Agent placement + pollution detection │ │ ├── system_detect/ # OS, shell, Python, RAM, CPU │ │ ├── telegram_readiness.py # Telegram bot-host readiness checks (doctor) -│ │ └── ui/ # Progress bars, menus, banners -│ └── plugins/ -├── tests/ # 981 passing +│ │ └── ui/ # Rich progress bars, spinners, check glyphs, step headers +│ ├── integrations/ # Placeholder — no code yet +│ └── plugins/ # Placeholder — no code yet +├── shared/ # Cross-handler code — json_handler, json_ops, +│ # project_home, registry_discovery, scaffold_content +├── tests/ # 1034 passing ├── requirements.project.txt # Project-specific Python dependencies ├── .trinity/ # Identity + session history + observations └── README.md @@ -70,21 +76,36 @@ aipass/ | `aipass` | Show available commands | | `aipass help [Q]` | README-backed Q&A with branch routing | | `aipass doctor` | System health — structure, registry, hooks, pytest | +| `aipass doctor --verbose` | Same, with per-check detail | | `aipass doctor --fix` | Remediation report with `drone @spawn repair` commands | -| `aipass doctor --json` | JSON output for structure scan results | +| `aipass doctor --fix --json` | JSON remediation report — `--json` alone falls through to the normal report | | `aipass doctor --cross-os` | Cross-OS pre-flight — OS-gap cross-ref + routing/versions/hookstatus | | `aipass doctor --cross-os --e2e` | ...also runs the real e2e wiring suite (heavy, opt-in) | | `aipass doctor --cross-os --record [PATH]` | Write a machine-filled Run Record for the human acceptance pass | -| `aipass init` | 10-stage guided setup (resumable) | -| `aipass init update [target]` | Refresh managed scaffold + provision owner-tier git auth | -| `aipass init update --dry-run` | Preview the git-auth repairs only — writes nothing | -| `aipass install` | One-command bootstrap — clone + setup.sh + hooks, then hand off to init | -| `aipass profile` | Show/edit user profile | +| `aipass init` | Print init usage — bare `init` does NOT start the guided setup | +| `aipass init run` | 10-stage guided setup (resumable) | +| `aipass init run --non-interactive` | CI/headless run | +| `aipass init run --name/--cli/--style/--template ` | Pre-fill a stage answer | +| `aipass init run --dry-run` | Walk all stages, write nothing | +| `aipass init --list` | List available project templates | +| `aipass init [name]` | Scaffold AIPass files into an existing path (absent from `init --help`) | +| `aipass init agent ` | Create an agent via `drone @spawn` (absent from `init --help`) | +| `aipass init update [target]` | Refresh managed scaffold + provision owner-tier repo auth | +| `aipass init update --dry-run` | Preview the auth repairs only — writes nothing | +| `aipass install` | One-command bootstrap — clone + setup.sh + hooks, then a concierge welcome chat | +| `aipass install --path DIR` / `--here` | Choose the install home | +| `aipass install --non-interactive` / `--no-chat` / `--chat-only` | Headless, install-only, or chat-only | +| `aipass install --no-symlink` / `--force-symlink` | Control the global CLI symlinks | +| `aipass install --dry-run` | Walk the steps, no side effects | +| `aipass profile` | Show user profile | +| `aipass profile set ` | Update a profile field | +| `aipass profile clear [--yes]` | Reset the profile | | `aipass read [branch]` | View a branch README rendered in the terminal (bare: module info + branch list) | -| `aipass handoff` | Show CLI-handoff status | -| `aipass handoff launch [--cli claude\|codex]` | Launch chosen CLI in a new session (tmux / wt.exe) | +| `aipass handoff` | Print handoff usage — bare does NOT show status, use `--info` | +| `aipass handoff --info` | Show stored CLI + platform status | +| `aipass handoff launch [--cli claude\|codex] [--cwd PATH] [--flag VARIANT]` | Launch chosen CLI in a new session (tmux / wt.exe) | | `aipass ` | Multi-word unknown input falls through to `aipass help` | -| `aipass new ` | Create a project in projects/ — own git repo, AIPass scaffold, resident agent | +| `aipass new ` | Create a project in projects/ — own repo, AIPass scaffold, resident agent | | `aipass new --template python` | Create with Python template (pyproject + src/) | | `aipass new --no-agent` | Create without resident agent | | `aipass adopt ` | Turn an existing `projects/` directory into a full project — additive scaffold only | @@ -100,26 +121,32 @@ aipass/ ### Depends On -- `@drone` — routing, command dispatch -- `@seedgo` — standards audit -- `@spawn` — first agent creation + structural repair -- `@flow` — plan lifecycle (open/close) -- `@ai_mail` — test emails -- `@prax` — health signals, logging -- `pytest` — test execution +- `@drone` — routing; every outbound command in this branch is a `drone` subprocess call +- `@spawn` — agent creation (`init run`, `init agent`, `new`) + registry sync during `install` +- `@hooks` — `feedback` delegates to it outright; `doctor` and the cross-OS pre-flight check `drone @hooks status` +- `@ai_mail` — test-convention ping emails (`ping_sweep`) +- `@prax` — logging, imported by nearly every module and handler +- `@trigger` — soft dependency: `trigger.fire()` on write-failure cleanup, wrapped in try/except so it degrades +- `pytest` — test execution (`doctor` shells out to collect the suite) + +`@seedgo` and `@flow` are part of this branch's working practice — audits before "done", plans for builds — but no code path here calls either. They appear in `ping_sweep`'s reachability list only. ### Provides To -Humans only. Nothing in AIPass depends on this branch. +Humans only. No `.py` source elsewhere in AIPass imports this branch. ## Tests -981 passing — `pytest src/aipass/aipass/tests/` +1034 passing — `pytest src/aipass/aipass/tests/` ## Known Issues - Running the file directly (`python apps/aipass.py`) fails on package imports (ModuleNotFoundError) — use the installed `aipass` entry point, which works from any directory. +- `aipass --help` omits `feedback` and `handoff`, and its example line still claims `aipass init` starts the guided setup. Bare `aipass` lists `feedback` but also omits `handoff`. +- `aipass init --help` documents neither `aipass init [name]` nor `aipass init agent `, and omits the `--style` flag that `init run` accepts. +- `aipass handoff --help` claims bare `aipass handoff` shows status; it prints the usage block instead. +- `aipass install --no-chat` returns before the doctor pre-flight runs, so the pre-flight is skipped along with the chat. ## Last Updated -Last Updated: 2026-08-13 +Last Updated: 2026-08-25 diff --git a/src/aipass/api/README.md b/src/aipass/api/README.md index 2e68ad36d..2de827168 100644 --- a/src/aipass/api/README.md +++ b/src/aipass/api/README.md @@ -5,8 +5,8 @@ > Centralized external API gateway — authenticated service clients for all external APIs **Module:** `aipass.api` | **Role:** `api_gateway` -**Seedgo:** 100% (45/45) | **Tests:** 1579 pass | **Functions:** 225 public (205 tested) -**Last Updated:** 2026-08-21 +**Seedgo:** 100% (46/46) | **Tests:** 1579 pass | **Functions:** 225 public (205 tested) +**Last Updated:** 2026-08-25 *THE BOARD WAS RED FOR THIS BRANCH IN THREE PLACES AND ONE OF THEM ONLY EXISTED IN CI. seedgo scored the file lane's statics module with 4 unresolved @@ -252,7 +252,7 @@ drone @api stats api/ ├── apps/ │ ├── api.py # Entry point — module discovery, command routing -│ ├── modules/ # Orchestration layer (9 modules) +│ ├── modules/ # Orchestration layer (10 modules) │ │ ├── api_key.py # Key retrieval, validation, provider listing │ │ ├── secrets.py # Cross-branch secrets door (in-process API) │ │ ├── openrouter_client.py # OpenRouter client — calls, models, status @@ -263,20 +263,21 @@ api/ │ │ ├── integrations_manager.py # Contract dispatch — integrations list/call │ │ ├── registry.py # Driver auto-discovery (load_drivers) │ │ └── host_serve.py # host_api sub-router — serve/--detach, status, stop -│ ├── handlers/ # Business logic (9 packages, 35 files) +│ ├── handlers/ # Business logic (8 packages, 34 files) │ │ ├── auth/env.py, keys.py, secrets.py │ │ ├── config/provider.py │ │ ├── google/auth.py, service_factory.py, retry.py │ │ ├── host/config.py, tokens.py, server.py, feed.py, fleet.py, face.py, verbs.py, attach.py, uploads.py │ │ ├── host/statics.py (bundle cache policy), lifetime.py (detached serve), refusals.py (unreadable-root memory) │ │ ├── host/reads.py (resolution, files, dirs), git_reads.py (the whole git surface): patch, changes, log, commit, remote +│ │ ├── host/pump.py (the attach socket's two directions), settings.py (the desktop's two gears), memory_config.py (@memory's limits, served) │ │ ├── integrations/list.py, call.py │ │ ├── json/json_handler.py │ │ ├── openrouter/caller.py, client.py, models.py, provision.py │ │ └── usage/aggregation.py, cleanup.py, tracking.py │ └── integrations/ # Private driver space (gitignored) │ └── {project}/driver.py -└── tests/ # 1359 test functions across 46 files +└── tests/ # 1483 test functions across 49 files (1579 collected, parametrised) └── conformance/settings/ # 39 shared goldens both runtimes must satisfy ``` @@ -809,9 +810,13 @@ argument to the shared half. **`ended` is a fact, not a success flag.** `ended: true` means a live session was ended; `ended: false` with `ok: true` means there was nothing to end, which is the goal state rather than a failure. Both travel, because the phone shows different -sentences and flattening them here would make that impossible. An unknown branch -is a *refusal*, never nothing-to-end — both show `room: null` and they are -opposite facts, so `ok` is what tells them apart. +sentences and flattening them here would make that impossible. A refusal is never +nothing-to-end, and the two never share a shape: an unknown branch is refused +before the exec by `citizen_address`, so it leaves as a **400 `verb_refused`** +carrying no `room` and no `ok` at all, while a refusal spoken by @baud's own +envelope comes back 200 with `ok: false` and their sentence in `detail`. Only +the second one can be mistaken for nothing-to-end, and there `ok` is what tells +them apart. The exec lives in `fleet.py`, which already owns @baud's binary — one resolution, one cwd rule, one parser for their envelope. That is what lets the verb lane keep @@ -1073,10 +1078,10 @@ Private drivers in `apps/integrations/{project}/driver.py` (gitignored) register - Backup branch credential migration pending (`~/.aipass/` → `~/.secrets/aipass/`; legacy dir still present) - No rate limiting on OpenRouter calls (S117 finding) -**Troubleshooting:** `openai`/Google auth `ModuleNotFoundError` despite a working venv → the `[llm]`/`[drive]` extras were added after the venv was last built; re-run `setup.sh` (installs `.[dev,memory,llm,drive]`) to resync, no code fix needed. Both import cleanly as of 2026-08-13. +**Troubleshooting:** `openai`/Google auth `ModuleNotFoundError` despite a working venv → the `[llm]`/`[drive]` extras were added after the venv was last built; re-run `setup.sh` (installs `.[dev,memory,llm,drive]`) to resync, no code fix needed. Both import cleanly as of 2026-08-25 (`openai` 2.49.0, `google.auth` + `googleapiclient`), verified by import alone — no call goes out. --- -*Last Updated: 2026-08-14* +*Last Updated: 2026-08-25* [← Back to AIPass](../../../README.md) diff --git a/src/aipass/backup/README.md b/src/aipass/backup/README.md index 4e38e11b7..4eca3e827 100644 --- a/src/aipass/backup/README.md +++ b/src/aipass/backup/README.md @@ -4,7 +4,7 @@ **Module:** `aipass.backup` **Version:** 1.0.0 **Created:** 2026-04-16 -**Last Updated:** 2026-08-13 +**Last Updated:** 2026-08-25 --- @@ -15,7 +15,7 @@ - Back up any project directory on the system (not just AIPass projects) - Each project owns its backup config (`.backup/`) and ignore patterns (`.backupignore`) - Snapshot mode: full mirror copy -- Versioned mode: incremental timestamped backups with automatic pruning +- Versioned mode: incremental timestamped backups (append-only — there is no pruning; see "Store Cleanup") - Project registry for name-based lookups (`backup snapshot @AIPass`) ### How I Work @@ -44,6 +44,7 @@ apps/ │ ├── status.py # Backup status display │ └── versioned.py # Incremental timestamped backup └── handlers/ + ├── cleanup/ # Mirror cleanup — removes snapshot files whose source is gone ├── copy/ # File copying (snapshot + versioned) ├── diff/ # Diff generation + restore from the versioned store ├── drive/ # Google Drive handlers (auth, upload, tracker, share) @@ -52,11 +53,15 @@ apps/ ├── path/ # Backup path building + caller-CWD resolution ├── project/ # Config, registry, setup (.backup/) ├── report/ # Result formatting - ├── scan/ # Directory walking + filtering + ├── scan/ # Directory walking + filtering + the run ceiling ├── state/ # Changelog, metadata, timestamps └── ui/ # Settings window (archived — see ui/.archive/) ``` +`apps/integrations/` and `apps/plugins/` also exist on disk but are empty +scaffolds (a README and an `__init__.py`, no code) — they are left out of the +tree above deliberately, not by oversight. + --- ## Commands @@ -65,7 +70,7 @@ apps/ backup register [--name ] # Register a project for backup backup snapshot # Full mirror backup backup versioned # Incremental timestamped backup -backup all # Snapshot + versioned + drive +backup all # Snapshot + versioned + drive sync backup status # Show backup info and history backup restore list # List available versions of a file backup restore file # Restore a file version to output path @@ -77,7 +82,11 @@ backup drive_clear --force # Clear the LOCAL tracker (remote files backup share [--public] # Upload one file to Drive, return share link ``` -All 12 commands are auto-discovered by the entry point router. +The router auto-discovers every file in `apps/modules/` that exposes a +`handle_command()`. That is 13 modules: the 12 verbs above, plus `display`, +which is a rendering helper rather than a backup verb — it answers only to its +own name (`drone @backup display` prints its introspection and does nothing +else) and is intentionally undocumented as a command. **A `--help` anywhere in the arguments prints help and runs nothing** — `drone @backup snapshot @myapp --help` is a safe probe, not a backup. @@ -155,6 +164,11 @@ There is no static fallback. The seed IS the safety mechanism — an empty or mi - To change ignores for an **existing** project → edit its `.backupignore` The repo-root `/.backupignore` ships intentionally as the curated default so users don't snapshot junk. +It is hand-maintained, not generated from the seed, and it has **drifted**: as of +2026-08-25 it is missing `target/` and `logs/`, and its header still cites the +old source (`handlers/ignore/patterns.py`) instead of +`templates/backupignore.template`. AIPass's own tree is therefore not covered +against the exact runaway class the `target/` pattern was added for. **A miss in the seed is expensive.** The template covers `build/`, `dist/`, `target/`, `node_modules/`, `.venv/` and friends precisely because an uncovered build-artifact tree is indistinguishable from real source to the walker. `target/` was added on 2026-08-20 after a Rust `src-tauri/target` tree (33,093 files / 18GB) was walked and copied for 7.5h, writing 50GB into the stores. Patterns are unanchored on purpose: baud's tree was `app/src-tauri/target`, so an anchored `/target/` would have missed it. @@ -194,7 +208,14 @@ Mirror cleanup (`handlers/cleanup/mirror.py`) removes snapshot files **whose sou There is **no lane** that removes files which are now *ignored* but still present in the source tree. A directory added to `.backupignore` after a backup stays in `snapshots/` and `versioned/` indefinitely: - `snapshots/` — `_should_delete()` keeps any file whose source still exists, and an ignored-but-present `target/` still exists. `cleanup_deleted_files()` accepts a `should_ignore` callback and **never calls it** — the ignore-aware sweep is unimplemented, not merely unused. -- `versioned/` — has no cleanup path at all. The store is append-only by design, and holds two copies of every new file (current + baseline), so it grows to roughly 2× the source. +- `versioned/` — has no cleanup path at all. The store is append-only, and holds two copies of every new file (current + baseline), so it grows to roughly 2× the source. + +**`max_versions` does nothing.** `.backup/config.json` carries a `max_versions` +key (default `10`), `register` writes it, and `status` prints it as "Max +versions" — but no code reads it. Nothing prunes old versions. The only file +deletion anywhere in this branch is `mirror.py:59`, the vanished-source snapshot +sweep above. Treat the key as advertised-but-unimplemented until a pruning lane +exists. Removing a now-ignored tree from a store is currently a manual `rm -rf` of the corresponding path under `.backup/`. @@ -203,8 +224,9 @@ Removing a now-ignored tree from a store is currently a manual `rm -rf` of the c ## Integration Points ### Depends On -- @prax — logging -- @cli — Rich console output +- @prax — logging (`logger`, `append_jsonl`) +- @cli — Rich console output (`console`, `error`, `header`, `success`, `warning`) +- @api — Google Drive auth + retry, via `google_client` (Drive commands only) ### Provides To - Any project on the PC — backups are project-owned (`.backup/` in target root) diff --git a/src/aipass/canary/README.md b/src/aipass/canary/README.md index f8881a518..1a8c3476d 100644 --- a/src/aipass/canary/README.md +++ b/src/aipass/canary/README.md @@ -3,7 +3,7 @@ **Purpose:** Permanent test citizen. Exists to be spawned, dispatched, resumed, broken and re-scaffolded so the working fleet never is. All mail, logs and memories here are TEST DATA by definition — never production work. Sibling of @finch (projects tier) and @wren (external tier): three homes covering three different fence contexts. **Module:** `aipass.canary` **Created:** 2026-08-20 -**Last Updated:** 2026-08-22 +**Last Updated:** 2026-08-25 --- @@ -24,6 +24,11 @@ Run the branch's own suite from the repo root, which is how CI runs it: pytest src/aipass/canary/tests -v ``` +Three of those functions are parametrized, so pytest collects and passes more +cases than there are `def test_` lines. Both counts are true of different +things; the tree below states the function count, which is what the standards +audit measures. + --- ## Overview @@ -50,16 +55,24 @@ fleet, and saying so is part of every report. CANARY/ ├── apps/ │ ├── canary.py # Entry point -│ ├── modules/ # Business logic (empty by design — added per test) +│ ├── modules/ # Business logic — no .py here by design, added per test │ ├── handlers/ │ │ └── json/ # JSON handler shim over aipass.aipass.shared -│ └── plugins/ # Extensions +│ ├── integrations/ # Scaffold, empty +│ └── plugins/ # Scaffold, empty ├── artifacts/ # Test artifacts written during dispatches +├── canary_json/ # Where the json shim writes — test data, nothing depends on it +├── tests/ # 38 test functions, all passing as of 2026-08-25 ├── docs/ -├── tests/ └── README.md ``` +The branch also carries the standard spawn scaffold — `.trinity/`, +`.aipass/`, `.ai_mail.local/`, `.archive/`, `.seedgo/`, `.spawn/`, +`docs.local/`, `dropbox/`, `logs/`, `templates/`, `tools/` — README-only +placeholders except where a service writes into them. `logs/` holds dispatch +transcripts written by @ai_mail, not output from canary's own code. + --- ## Commands @@ -71,12 +84,18 @@ now rather than a fixed catalogue. | Flag | What it does | |------|--------------| | *(none)* | Print the self-map: identity, purpose, discovered modules | -| `--help`, `-h` | Usage, flags and examples | -| `--version`, `-V` | Branch name and version | +| `--help`, `-h`, `help` | Usage, flags and examples | +| `--version`, `-V` | Branch name and version (`CANARY v2.0.0`) | -`drone @canary --help` shows that subcommand's help without executing it. An unknown command is refused and exits non-zero — a refusal that exits 0 is a -lie to every non-human caller, and that one is pinned by test here. +lie to every non-human caller, and that one is pinned by test here +(`test_unknown_command_exits_nonzero`). + +`main()` also routes ` --help` to that subcommand's own help without +executing it, and a test pins it against a stub module. It cannot be reached +from a live `drone @canary` today: with no modules registered, every subcommand +is unknown, so `drone @canary --help` prints `❌ Unknown command` and +exits 1. Documented as code that exists, not as behaviour you can observe here. --- @@ -85,7 +104,13 @@ lie to every non-human caller, and that one is pinned by test here. ### Depends On - **@cli** — `console`, `error` for all terminal output. -- **@prax** — the logger; every fallback and failure path writes a line. +- **@prax** — the logger. Wired on the module-discovery paths: import fallback, + module load failure, a module raising mid-route, and an unhandled error in + `main()`. All four are dead while `modules/` is empty, so canary has written + no prax log of its own — there is no `canary_canary.log` in `system_logs/`. + The one failure path that *does* fire today, the unknown-command refusal, + goes through @cli's `error()`, which marks the command failed but writes no + prax line. - **@ai_mail** — how work arrives; canary is dispatched, it does not self-start. - **@spawn** — owns the framework template this branch was scaffolded from. diff --git a/src/aipass/cli/README.md b/src/aipass/cli/README.md index 927bb6bf1..cb3670c28 100644 --- a/src/aipass/cli/README.md +++ b/src/aipass/cli/README.md @@ -7,7 +7,7 @@ **Version:** 2.1.0 **Seedgo:** 100% **Tests:** 192 tests across 10 files — 201 passing, 0 skipped (parametrized cases expand at runtime) -**Last Updated:** 2026-08-19 +**Last Updated:** 2026-08-25 ## Quick Start @@ -113,6 +113,8 @@ cli/ │ │ ├── display.py # header, success, error, warning, fatal, section, exit-code API │ │ └── templates.py # operation_start, operation_complete │ ├── handlers/ # PRIVATE — internal implementation +│ │ ├── cli/ +│ │ │ └── help_flags.py # wants_help() — whole-sequence help detection │ │ ├── json/ │ │ │ └── json_handler.py # JSON lifecycle (CRUD, validation, rotation) │ │ └── templates/ # Scaffold placeholder @@ -133,9 +135,15 @@ cli/ │ └── parked/ # TRACKED, not run — collect_ignore_glob barrier (archive doctrine, 2026-08-18) ├── cli_json/ # Auto-created JSON (config, data, log) ├── logs/ # Branch-level logs -└── .archive/ # Archived stubs (extensions/, json_templates/, drone_adapter, __main__, test_scaffold) +└── .archive/ # Archived stubs (extensions/, json_templates/, drone_adapter, __main__, init_project leftovers) ``` +Branch-standard scaffold dirs are omitted from the tree above: `artifacts/`, `docs/`, +`docs.local/`, `dropbox/`, `templates/`, `tools/`, and the dot-dirs (`.trinity/`, +`.aipass/`, `.ai_mail.local/`, `.seedgo/`, `.spawn/`, `.daemon/`, `.backup/`). +The scaffold `test_scaffold` moved out of `.archive/` to `tests/parked/scaffold(disabled).py` +on 2026-08-19 — tracked, not collected. + **Testing display output:** never assert on raw captured bytes. Build the console with `make_capture_console()` from `tests/conftest.py` and assert through its `get_output()`, which strips ANSI. Rich decides whether to emit escapes by probing the environment, so a @@ -169,9 +177,9 @@ json_handler.ensure_module_jsons("cli") # Create all 3 if missing ## Integration Points ### Depends On -- `rich` — Terminal formatting (Table, Panel, Text, Console) -- `aipass.prax` — Logging (imported in cli.py only, not in modules/) -- Python stdlib (`sys`, `importlib`, `pathlib`, `json`) +- `rich` — Terminal formatting (Console, Panel, Table, Text, Columns, box) +- `aipass.prax` — Logging (one import, in `apps/cli.py` only — never in modules/ or handlers/) +- Python stdlib (`sys`, `os`, `json`, `time`, `tempfile`, `inspect`, `importlib`, `pathlib`, `datetime`, `typing`) ### Cannot Import (in modules/) - `aipass.prax` — Circular dependency (prax depends on cli). Bypassed in `.seedgo/bypass.json`. @@ -186,13 +194,13 @@ json_handler.ensure_module_jsons("cli") # Create all 3 if missing | Entry | Command | How | |-------|---------|-----| | drone | `drone @cli [command]` | Routes to `apps/cli.py:main()` | -| Import | `from aipass.cli import ...` | The real entry point — 252 call sites fleet-wide | +| Import | `from aipass.cli import ...` | The real entry point — 356 import statements in 264 files across 17 branches (measured 2026-08-25; 33 of them in test files) | `python -m aipass.cli` is **not** an entry point — `__main__.py` was archived 2026-05-02 (no branch in the fleet ships one). `cli_entry()` still exists in `__init__.py` but is no longer wired: `pyproject.toml` maps the `aipass` script to `aipass.aipass.apps.aipass:main`. See APLAN-0002 for the keep-or-retire decision. --- -*Last Updated: 2026-08-19* +*Last Updated: 2026-08-25* --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/commons/README.md b/src/aipass/commons/README.md index 109503926..a7d0e8d7b 100644 --- a/src/aipass/commons/README.md +++ b/src/aipass/commons/README.md @@ -14,7 +14,7 @@ Commons is the social layer of AIPass. It gives branches a shared space beyond task-driven work -- a place to share observations, ask questions, craft artifacts, explore hidden rooms, trade items, and just talk. -Backed by SQLite with WAL journal mode and FTS5 full-text search. 108 Python files (82 under `apps/`) across 22 modules and 20 handler domains. +Backed by SQLite with WAL journal mode (`handlers/database/db.py`) and FTS5 full-text search (`posts_fts`, `comments_fts`). 109 Python files (82 under `apps/`) across 22 modules and 20 handler domains -- excluding the 21 pre-refactor modules parked in `apps/modules/.archive/`. ### Quick Start @@ -43,7 +43,9 @@ drone @commons catchup > `drone @commons prompt --help` posts a real daily prompt to the feed. Use > `drone @commons --help` (no command) until this is fixed. -Caller identity is auto-detected from PWD. Run from your branch directory to post as that branch. +Caller identity is resolved in order: the `AIPASS_CALLER_CWD` env var drone sets (walked up to a `.trinity/passport.json`), then the real PWD as fallback, then `AIPASS_CALLER_BRANCH` +(`handlers/identity/identity_ops.py::get_caller_branch`). Under drone the env var is what identifies you; running the entry point directly falls back to PWD. Commons' +own `--help` text still says "auto-detected from PWD" -- that wording is imprecise and lives in `apps/commons.py`, not here. --- @@ -56,12 +58,13 @@ All commands are invoked via `drone @commons [args]`. | Command | Description | |---------|-------------| | `post "room" "Title" "Content"` | Create a post (types: discussion, review, question, announcement) | -| `feed` | Browse posts (`--room`, `--sort hot/new/top/activity`, `--limit`) | +| `feed` | Browse posts (`--room`, `--sort hot/new/top/activity`, `--limit`, `--offset`, `--page`) | | `thread ` | View a post with all comments | | `comment "text"` | Comment on a post (`--parent ` for nested replies) | | `vote post/comment up/down` | Vote on content | -| `delete ` | Delete your own post | +| `delete ` | Delete your own post (rejects a post you don't author) | | `room list/create/join/leave` | Manage rooms | +| `database` | Database module introspection | | `whoami` | Show the branch identity commons resolved for you | ### Spatial @@ -77,15 +80,15 @@ All commands are invoked via `drone @commons [args]`. | Command | Description | |---------|-------------| -| `craft "name" "desc"` | Create an artifact (`--rarity`, `--type`) | -| `artifacts` | List your artifacts (`--all` for everyone's) | +| `craft "name" "desc"` | Create an artifact (`--rarity`, `--type`, `--metadata '{...}'`) | +| `artifacts` | List your artifacts (`--all` for everyone's, `--type`/`--rarity` to filter) | | `inspect ` | Inspect artifact details (`--full` for provenance) | | `gift @branch` | Gift an artifact to another branch | | `trade @branch` | Propose a trade | -| `drop "name" "desc" [--expires N]` | Drop a new ephemeral item in a room | +| `drop "name" "desc" [--expires N]` | Drop a new ephemeral item in a room (N in minutes, default 5, clamped 1-1440) | | `find ` | Pick up an ephemeral item | | `mint "Event Name" @branch1 @branch2` | Mint proof-of-attendance event badges | -| `collab "name" "desc" @signer1 @signer2` | Initiate a joint artifact | +| `collab "name" "desc" @signer1 @signer2` | Initiate a joint artifact (`--rarity`, default `rare`) | | `sign ` | Sign a pending joint artifact | Counterparties for `gift`/`trade`/`mint`/`collab` are resolved from `AIPASS_REGISTRY.json` @@ -96,7 +99,7 @@ outside that file can post and comment, but cannot yet be named as a trade partn | Command | Description | |---------|-------------| -| `capsule "title" "content" ` | Seal a time capsule (1-365 days) | +| `capsule "title" "content" ` | Seal a time capsule -- `days` is **silently clamped** to 1-365, never rejected (`capsule_ops.py:52`) | | `capsules` | List all time capsules with countdowns | | `open ` | Open a capsule (when ready) | @@ -106,16 +109,18 @@ outside that file can post and comment, but cannot yet be named as a trade partn |---------|-------------| | `catchup` | Summary of what you missed since last visit | | `activity` | Recent comments across all threads | -| `watch ` | All notifications for a target | -| `mute ` | Silence notifications | -| `track ` | Mentions/replies only | +| `watch ` | All notifications for a target | +| `mute ` | Silence notifications | +| `track ` | Mentions/replies only | | `preferences` | View notification settings | +`thread` is accepted as a target type and behaves identically to `post` (`notification_ops.py:109`). + ### Social and Profiles | Command | Description | |---------|-------------| -| `profile` | View/edit social profile | +| `profile` | View social profile; edit with `profile set bio\|status\|role "text"` | | `who` | List all community members with status | | `welcome [branch]` | Welcome new branches (`--dry-run` supported) | @@ -141,7 +146,7 @@ outside that file can post and comment, but cannot yet be named as a trade partn |---------|-------------| | `explore` | Discover hints about secret rooms | | `secrets` | List secret rooms you've found | -| `leaderboard` | Rankings (artifacts, trades, posts, rooms, karma) | +| `leaderboard` | Rankings (artifacts, trades, posts, rooms, karma); `leaderboards` is an accepted alias | | `trending` | Show trending posts | | `react ` | Add a reaction to content | | `unreact ` | Remove your reaction | @@ -173,23 +178,30 @@ drone @commons pin drone @commons search "routing proposal" ``` -Boardrooms were first used for DPLAN-0053 (drone architecture), where multiple branches contributed design input through posts and threaded comments. +"Boardroom" is a convention, not a code feature -- the word appears nowhere in the schema or the modules; a boardroom is an ordinary room used for one design thread. + +The one boardroom on record in `commons.db` is `boardroom-compass-v3` (created by `devpulse`): a single RFC post on Compass curation v2 (DPLAN-0246) carrying 11 threaded +comments from four branches -- @seedgo, @memory, @hooks, @devpulse. An earlier edition of this README credited DPLAN-0053 ("drone architecture") as the first use; no +such post exists in the database and DPLAN-0053 is documented elsewhere in the repo as hook architecture research, so that citation is withdrawn rather than replaced. --- ## Introspection System -Commons uses a two-tier introspection system that differs from other branches. Other branches are single-purpose (one module = one command set). Commons has 22 modules with ~50 routable commands -- agents arriving fresh need a fast way to discover what's available without reading 22 files. +Commons uses a two-tier introspection system that differs from other branches. Other branches are single-purpose (one module = one command set). Commons has 22 modules routing 52 distinct command strings -- agents arriving fresh need a fast way to discover what's available without reading 22 files. -**Tier 1: Global discovery** (`drone @commons` with no args or `--help`) +**Tier 1: Global discovery** (`drone @commons` with no args) Lists all 22 discovered modules with one-line descriptions. This is the "what does commons do?" entry point. +`drone @commons --help` is a *different* view: it calls `print_help()` (`apps/commons.py:168`), which prints the grouped command reference, not the module list. Both are +top-level discovery; only the no-args form does module discovery. + **Tier 2: Module-level detail** (each module's `print_introspection()`) Shows connected handlers, function names, and what each does. This is the "how do I use this specific feature?" level. Every module retains its `print_introspection()` function by design. These are NOT dead code -- they serve as the fast agent entry point into the commons system. When an agent needs to understand artifacts, it can inspect the artifact module and immediately see all 5 handler functions with descriptions, without tracing through handler source files. -**Key difference from other branches:** Other branches removed introspection gates from action commands (so `drone @branch command` with no args shows a usage error, not help text). Commons did the same -- the gates were removed from the action modules in S15/S16. Five subcommand modules still keep a no-args gate (`notification.py`, `space.py`, `room.py`, `database.py`, `reaction.py`), because those dispatch subcommands rather than performing one action. The `print_introspection()` functions themselves remain as the discovery layer. +**Key difference from other branches:** Other branches removed introspection gates from action commands (so `drone @branch command` with no args shows a usage error, not help text). Commons did the same -- the gates were removed from the action modules (session provenance recorded as S15/S16; not re-verified here). Five subcommand modules still keep a no-args gate (`notification.py`, `space.py`, `room.py`, `database.py`, `reaction.py`), because those dispatch subcommands rather than performing one action. The `print_introspection()` functions themselves remain as the discovery layer. --- @@ -216,11 +228,11 @@ Every module retains its `print_introspection()` function by design. These are N commons/ ├── apps/ │ ├── commons.py # Entry point (Layer 1) -│ ├── modules/ # Layer 2: Thin routers (22 modules) +│ ├── modules/ # Layer 2: Thin routers (22 modules; .archive/ holds 21 pre-refactor originals) │ │ ├── post.py # post, thread, delete │ │ ├── comment.py # comment, vote │ │ ├── feed.py # feed -│ │ ├── room.py # room list/create/join +│ │ ├── room.py # room list/create/join/leave │ │ ├── commons_identity.py # Branch detection (shared utility), whoami │ │ ├── catchup.py # catchup │ │ ├── activity.py # activity @@ -235,41 +247,49 @@ commons/ │ │ ├── artifact.py # craft, artifacts, inspect, collab, sign │ │ ├── space.py # enter, look, decorate, visitors │ │ ├── trade.py # gift, trade, drop, find, mint -│ │ ├── leaderboard.py # leaderboard +│ │ ├── leaderboard.py # leaderboard (alias: leaderboards) │ │ ├── explore.py # explore, secrets │ │ ├── capsule.py # capsule, capsules, open │ │ └── database.py # database init, connection management -│ └── handlers/ # Layer 3: Implementation (20 domains) -│ ├── database/ # Schema, CRUD, migrations -│ ├── json/ # JSON tracking-file helpers -│ ├── posts/ # Post operations + reward drops -│ ├── comments/ # Comment operations + reward drops -│ ├── feed/ # Feed sorting/filtering -│ ├── rooms/ # Room ops, spatial, explore -│ ├── catchup/ # Catchup queries -│ ├── activity/ # Cross-thread activity feed -│ ├── central/ # Central data file writer -│ ├── notifications/ # Mentions, preferences, dashboard (tiered) -│ ├── profiles/ # Profile operations -│ ├── search/ # FTS5 search, log export -│ ├── welcome/ # Welcome post generation -│ ├── curation/ # Reactions, pins, trending -│ ├── engagement/ # Prompts, events -│ ├── digest/ # Activity digests -│ ├── artifacts/ # Artifacts, trading, capsules, rewards -│ ├── social/ # Leaderboards -│ ├── identity/ # Identity detection -│ └── dashboard/ # Dashboard file writer -├── tools/ # Utilities -├── tests/ # Test suite -├── docs/ # Documentation +│ ├── handlers/ # Layer 3: Implementation (20 domains) +│ │ ├── database/ # Schema, CRUD, migrations +│ │ ├── json/ # JSON tracking-file helpers +│ │ ├── posts/ # Post operations +│ │ ├── comments/ # Comment operations +│ │ ├── feed/ # Feed sorting/filtering +│ │ ├── rooms/ # Room ops, spatial, explore +│ │ ├── catchup/ # Catchup queries +│ │ ├── activity/ # Cross-thread activity feed +│ │ ├── central/ # Central data file writer +│ │ ├── notifications/ # Mentions, preferences, dashboard (tiered) +│ │ ├── profiles/ # Profile operations +│ │ ├── search/ # FTS5 search, log export +│ │ ├── welcome/ # Welcome post generation +│ │ ├── curation/ # Reactions, pins, trending +│ │ ├── engagement/ # Prompts, events +│ │ ├── digest/ # Activity digests +│ │ ├── artifacts/ # Artifacts, trading, time capsules +│ │ ├── social/ # Leaderboards +│ │ ├── identity/ # Identity detection +│ │ └── dashboard/ # Dashboard file writer +│ ├── integrations/ # (README only — no code yet) +│ ├── json_templates/ # Default JSON tracking templates +│ ├── plugins/ # (README + __init__ only — no plugins yet) +│ └── logs/ # Entry-point log output (currently empty) +├── tools/ # Utilities (2 files) +├── tests/ # Test suite (24 files) +├── docs/ # (empty — README + .gitkeep only) +├── docs.local/ # Sub-agent drops, not shipped ├── commons_json/ # JSON tracking directory +├── artifacts/ # Branch artifacts + birth certificate +├── dropbox/ # Inbound file drops +├── templates/ # Template directory +├── logs/ # Per-handler log output └── README.md ``` ### Special Mechanics -- **Reward Drops:** 10% chance of finding a surprise artifact when posting or commenting - **Secret Rooms:** Hidden rooms discoverable through exploration - **Ephemeral Items:** Dropped items expire and get swept on access - **Joint Artifacts:** Require multiple signers to create (collaborative crafting) @@ -280,13 +300,17 @@ commons/ ## Integration Points ### Depends On -- `aipass.prax` -- Logging via `system_logger` (graceful fallback if unavailable) -- `aipass.cli` -- Console output and headers (graceful fallback if unavailable) +- `aipass.prax` -- Logging via `system_logger`. **Hard dependency**: all 52 import sites are plain top-level imports with no `try`/`except`, including the entry point + (`apps/commons.py:47`). If prax is unavailable, commons does not start. +- `aipass.cli` -- Console output and headers. Graceful fallback to a plain `rich` Console in every module and in `handlers/curation`, `handlers/dashboard` + (`try`/`except ImportError`) -- but **not** in the entry point `apps/commons.py:48`, which imports it hard. - SQLite with FTS5 (stdlib) ### Provides To - All branches -- social platform, community gathering, artifact system -- Branch dashboards -- `commons_activity` section (mentions, unread counts, top threads) +- Branch dashboards -- `commons_activity` section (`handlers/dashboard/dashboard_writer.py`): `mentions`, `mention_details`, `new_posts_since_last_visit`, + `new_comments_since_last_visit`, `last_checked`. Top threads are **not** in this section -- `top_threads` lives in `COMMONS.central.json`, written by + `handlers/central/central_writer.py` via `push-central`. --- @@ -302,7 +326,7 @@ drone @commons --version # Version --- -*Last Updated: 2026-08-13* +*Last Updated: 2026-08-25* --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/daemon/README.md b/src/aipass/daemon/README.md index 8d4fd8159..8ab91aabd 100644 --- a/src/aipass/daemon/README.md +++ b/src/aipass/daemon/README.md @@ -2,11 +2,11 @@ # DAEMON -**Purpose:** Cron-triggered task scheduler with plugin system. Routes commands to modules for scheduled tasks, activity reports, action management, and status digests. +**Purpose:** Decentralized task scheduler fired by a systemd user timer. Discovers every citizen's `.daemon/schedule.json`, wakes due owners, and reports fleet activity. The plugin system it was born with is retired — see **Plugins** below. **Module:** `aipass.daemon` **Created:** 2026-03-07 **Citizen Class:** aipass_framework -**Last Updated:** 2026-08-16 +**Last Updated:** 2026-08-25 --- @@ -80,13 +80,15 @@ daemon/ │ │ │ ├── rotation.py # Steward roster, pointer state, prompt rendering │ │ │ ├── runstate.py # last_run/next_run tracking + due-logic │ │ │ ├── telegram_notifier.py # Fail-soft lifecycle pings via @skills -│ │ │ └── .archive/ # assistant_notifier, task_registry, plugin_processor +│ │ │ └── .archive/ # assistant_notifier, task_registry, plugin_processor, +│ │ │ # telegram_notifier (superseded copy) │ │ ├── telegram/ # ARCHIVED — moving to skills system │ │ │ └── .archive/ # assistant_chat (archived) │ │ └── update/ │ │ └── data_loader.py # Data loading for status digests │ ├── extensions/ # Extension point for additional capabilities -│ ├── json_templates/ # JSON template definitions +│ ├── integrations/ # Private branch-local wrappers — gitignored except its README +│ ├── json_templates/ # JSON template definitions (default/: config, data, log) │ └── plugins/ │ ├── __init__.py # discover_plugins() — ORPHANED, no live caller │ └── .archive/ # ALL plugins archived: heartbeat, daily_audit, @@ -109,16 +111,12 @@ drone @daemon --help # Rich-formatted help with all commands drone @daemon --version # Print version drone @daemon update # Status digest — inbox, session info, escalations (partial — reads stale data paths) -drone @daemon schedule list # List pending scheduled tasks -drone @daemon schedule create "task" --due 7d --to @branch -drone @daemon schedule run-due # Fire all due tasks (sends emails) drone @daemon activity # Quick 24h activity summary drone @daemon activity-report # Full detailed report (--json for raw) drone @daemon branch-health BRANCH # Single branch deep dive -drone @daemon actions list # Action registry -drone @daemon actions on/off # Toggle action -drone @daemon actions set reminder 7d "msg" --to @branch -drone @daemon actions set schedule @branch "prompt" daily 04:00 + +drone @daemon queue # Unified job queue view (--json for frozen schema) +drone @daemon run # One scheduler tick — fire every due job now drone @daemon install-timer # Install + enable systemd user timer drone @daemon uninstall-timer # Stop + remove systemd user timer @@ -131,6 +129,12 @@ drone @daemon rotation # Roster, whose turn is next, recent turns drone @daemon rotation --json # Same state, machine-readable ``` +`schedule` and `actions` are still routable, but only as retirement notices — they ignore every +argument and print the same migration text pointing at `.daemon/schedule.json`. There is no +`schedule list`, `schedule create`, `schedule run-due`, `actions list`, `actions set` or +`actions on/off`; those subcommands were documented here long after the modules were retired. +Use `run`, `queue` and per-branch `.daemon/schedule.json` instead. + Each module accepts `--help` for module-specific usage: ```bash drone @daemon --help @@ -160,7 +164,18 @@ drone @daemon --help Each citizen owns its schedule at `/.daemon/schedule.json`. The daemon discovers and fires — citizens define their own jobs. -Two trees are swept: framework citizens under `src/aipass/*` (listed in `AIPASS_REGISTRY.json`) and project citizens under `projects//*` (listed in that project's own sealed `_REGISTRY.json`). A project registry's paths resolve against its own project root, never the repo root — `src/baud/baud` exists in both trees, and resolving repo-first picks the wrong directory. Vera-Studio is a separate repo and is out of scope until multi-root discovery exists. +**Nothing here is resident.** `install-timer` writes a systemd *user* timer, `daemon-tick.timer` +(`OnActiveSec=30s`, `OnUnitActiveSec=2min`, `Persistent=true`), which fires `daemon-tick.service` — +`Type=oneshot`, running `python3 -m aipass.daemon.apps.daemon run` once and exiting. Between ticks +there is no daemon process at all. + +The 2 min is nominal, not exact: `OnUnitActiveSec` measures from the *last activation*, and systemd's +default `AccuracySec=1min` batches the wakeup, so observed gaps run **2–3 min**. A tick costs about +**0.6s of CPU** (`systemctl --user show daemon-tick.service -p CPUUsageNSec`, measured 2026-08-25: +`626320000` ns) and roughly 1s wall. `systemctl --user list-timers` shows the next fire; the tick's +own output appends to `~/.aipass/daemon-tick.log`. + +Two trees are swept: framework citizens under `src/aipass/*` (listed in `AIPASS_REGISTRY.json`) and project citizens under `projects//*` (listed in that project's own sealed `_REGISTRY.json`). A project registry's paths resolve against its own project root, never the repo root — BAUD's registry row reads `src/baud/baud`, which is a path that also *could* resolve under this repo, and resolving repo-first picks the wrong directory. That collision was live until the phantom `/src/baud/` was removed (2026-08-24); the guard and its test stay, because the row is still relative and a repo-first join would recreate the phantom rather than fail. Vera-Studio is a separate repo and is out of scope until multi-root discovery exists. ### Job file schema @@ -197,7 +212,7 @@ Two trees are swept: framework citizens under `src/aipass/*` (listed in `AIPASS_ ### Staggering -No native offset field. To stagger jobs, seed different `last_run` values in `daemon_json/daemon_runstate.json`. +No native offset field. To stagger jobs, seed different `last_run` values in `daemon_json/daemon_runstate.json`. Within a single tick, jobs that fire together are already separated by a fixed 1s sleep (`run.py`) — that is not configurable and is not a substitute for offsetting the schedules themselves. --- @@ -266,12 +281,18 @@ surfaces here rather than blanking this report. ## Integration Points ### Depends On -- `rich` -- Console output and formatted display -- Python stdlib (`sys`, `typing`, `logging`) +- `rich` — console output and formatted display (via `@cli`'s shared console) +- `@prax` — the logger every module writes through +- `@ai_mail` — `wake_branch()`, the only way a job or a sweep actually wakes a citizen; also its + `WAKE_BLOCKLIST`, read by the sweep's wake policy +- `@memory` — `get_branch_health()`, rendered by `branch-health` (module layer only, see below) +- `@skills` — fail-soft Telegram lifecycle pings; imported lazily, absence is not an error +- Python stdlib, and a systemd *user* instance for the tick timer ### Provides To -- All modules — background task scheduling, activity monitoring, action tracking -- Plugins — extensible plugin system for recurring tasks (community_rotation, daily_audit, heartbeat) +- The fleet — job discovery and firing for any citizen that writes a `.daemon/schedule.json` +- The fleet — the unread-mail backstop (`inbox-sweep`) and the steward rotation +- `@skills` bot — `queue --json`, a frozen schema - Note: Telegram handlers archived — moving to skills system. See `apps/handlers/telegram/.archive/` --- @@ -345,7 +366,7 @@ remaining import is from an archived file. Scheduling is now decentralized: each - 10/10 modules covered, 46/50 public functions tested - Seedgo audit: **100%** with bypasses, **99%** with the bypass list emptied (22 entries) -*Last Updated: 2026-08-16* +*Last Updated: 2026-08-25* --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/devpulse/README.md b/src/aipass/devpulse/README.md index 724fba75a..67d0aaf83 100644 --- a/src/aipass/devpulse/README.md +++ b/src/aipass/devpulse/README.md @@ -56,11 +56,15 @@ src/aipass/devpulse/ │ │ ├── json/ # JSON operation logging (json_handler) │ │ ├── owner/ # Owner gate + admin grant (keygen, mint, 5-leg verify) │ │ └── watchdog/ # Agent, timer, schedule, registry -│ └── plugins/ # Plugin extension point +│ ├── integrations/ # Extension point (empty — README only) +│ └── plugins/ # Plugin extension point (empty — README only) ├── devpulse_json/ # JSON handler storage (config, data, logs per module) -├── tests/ # 448 tests +├── tests/ # 598 tests (594 passed, 4 skipped — 2026-08-25) +├── tools/ # One-shot scanners & probes (~30 scripts) + reports/ +├── prototypes/ # Shape-exploration prototypes +├── templates/ # Local templates ├── artifacts/ # Birth certificate, reports -├── dropbox/ # Received files, archived plans, install audit +├── dropbox/ # Received files, archived plans, campaign logs ├── docs/ # Transition notes └── DASHBOARD.local.json # Live state (refreshed by prax) ``` @@ -138,19 +142,22 @@ registries. | Command | What it does | |---|---| | `watchdog baseline` | Sign this session in to receive dispatch reports (logs out any older session) | +| `watchdog baseline --once` | Wire until the first delivered completion (run_in_background form) | | `watchdog agent @target [--timeout s]` | Stall-watch one dispatched agent (default 600 s) | | `watchdog timer ` | Wake after duration (5m, 30s, 2h, 1h30m) | | `watchdog timer start/stop ` | Named duration tracking | -| `watchdog schedule ` | Wait until a specific time | +| `watchdog timer list / report` | Active + historical timers / formatted session summary | +| `watchdog schedule [command]` | Wait until a time (or +duration), optionally run a command | | `watchdog status` | Signed-in session, outstanding dispatches, overdue entries | -| `watchdog cancel ` | Cancel a running watchdog | -| `watchdog list` | List all watchdog entries | +| `watchdog cancel ` | Cancel one watch (`--all` kills every active watch) | +| `watchdog list` | Alias for status | ### Feedback — the owner-to-owner channel (owner-only) -ai_mail and dispatch stop at the project boundary — **cross-project comms is -impossible by design, except feedback.** Project owners (managers) talk owner-to-owner -through it: an external project's owner runs `drone @devpulse feedback send ...` from +Dispatch crosses the project boundary in ONE direction only — devpulse's admin +seat reaches out; a dispatched project citizen answers on ai_mail's reply lane +(replies-only return path). For everything else, **feedback is the cross-project +channel**: an external project's owner runs `drone @devpulse feedback send ...` from their project and it lands in devpulse's feedback mailbox; devpulse answers with `feedback reply`. Same owner gate as watchdog — unseated projects are refused until `aipass doctor --fix` seats them. @@ -162,6 +169,7 @@ their project and it lands in devpulse's feedback mailbox; devpulse answers with | `feedback view ` | Read a message | | `feedback reply "msg"` | Reply to sender | | `feedback send "subject" "body"` | Send feedback to devpulse (any project's owner may call) | +| `feedback clear ` | Remove a message (`--all` removes all read) | ### Compass — rated decision store @@ -169,8 +177,8 @@ Curated truth-store of rated decisions (`good` / `bad` / `impressive` / `interes | Command | What it does | |---|---| -| `compass add "context" "decision" --rating R` | Store a rated decision (`--note`, `--tags`, `--source`) | -| `compass query "question" [--rating R] [--limit N]` | Search decisions (rating shown per hit) | +| `compass add "context" "decision" --rating R` | Store a rated decision (`--note`, `--tags`, `--source devpulse\|user`, `--supersedes N` archives+links the corrected entry) | +| `compass query "question" [--rating R] [--limit N] [--include-archived]` | Search decisions (rating shown per hit) | | `compass stats` | Counts by rating / status | | `compass rate ` | Re-rate a decision | | `compass archive ` | Archive a decision | @@ -219,7 +227,7 @@ drone @git log # Recent commits | drone | Command routing, subprocess, @branch resolution | | ai_mail | Dispatch (send + wake agents), email delivery | | flow | FPLANs (building), DPLANs (planning), APLANs (autonomous) | -| seedgo | Standards audits, checkers (35 standards) | +| seedgo | Standards audits, checkers (45 standards) | | prax | Monitoring, logs, dashboard | | memory | ChromaDB vectors, archival, search | @@ -227,7 +235,16 @@ drone @git log # Recent commits All branches via dispatch orchestration. Watchdog reporting for every dispatched agent. Feedback channel for cross-branch communication. Git operations (commit, PR, merge) for the entire project. -*Last Updated: 2026-08-19* +## Status & Known Issues + +Verified 2026-08-25 (README truth pass — every command above run read-only, tests executed, counts measured). + +- **Live bug** — `watchdog/registry.py:425` `killed or True` is always True: `watchdog cancel` prints KILLED even when the process survived SIGTERM. No test covers it (all 3 assert True). Fix queued as its own commit. +- **Refusals that exit 0** — feedback, admin_grant (×2) and compass (×3) use `warning()` for refusals, which skips `mark_command_failed()`, so a refused command exits 0. Fleet-wide sweep pending (@canary). +- **statusline.sh untracked** — the watchdog statusline lives at `~/.claude/statusline.sh`, outside the repo; on any other machine watchdog paints red until hand-copied. Fix direction undecided (Patrick's call — provider config dir). +- **Foreground-wire gap** — a `baseline` wire armed without the Monitor tool still paints `watchdog:in` green with no listener; `via=monitor` field offered, awaiting go-ahead. + +*Last Updated: 2026-08-25* --- diff --git a/src/aipass/drone/README.md b/src/aipass/drone/README.md index 18bb2a5c1..82f674dd7 100644 --- a/src/aipass/drone/README.md +++ b/src/aipass/drone/README.md @@ -17,7 +17,7 @@ - Manage git workflows: tier-based access (global read-only, owner write), commit, diff, log, sync, merge - Discover and scan available commands across the system - Provide `drone systems` introspection of all registered components -- Support external AIPass projects via dual registry lookup and module fallback +- Support external AIPass projects via dual registry lookup and module routing --- @@ -109,28 +109,33 @@ from aipass.drone import resolve_branch, list_branches, route_command # Resolve @name to absolute path path = resolve_branch("@seedgo") -# List all registered branches -branches = list_branches() # All branches -active = list_branches(status="active") # Filter by status +# List registered branches — status defaults to "active", it is not "all" +active = list_branches() # 18 today; the default IS a filter +by_type = list_branches(branch_type="core") # 0 today — see note below # Route a command to a branch -result = route_command("@seedgo", "verify") +result = route_command("@seedgo", "audit", args=["aipass", "@drone"]) print(result.stdout) # Command output print(result.exit_code) # 0 on success ``` +`list_branches(branch_type=...)` is a live parameter with nothing to match: no entry in `AIPASS_REGISTRY.json` carries a `branch_type` field, and all 18 carry `status: "active"`, so today the type filter always returns `[]` and any status but `active` returns `[]` too. Documented as it behaves, not as it reads — verified 2026-08-25. + ### Registry Management ```python -from aipass.drone import set_registry_path, get_registry_path +from aipass.drone import set_registry_path, get_registry_path, reset_registry_path # Use a custom registry location set_registry_path("/path/to/AIPASS_REGISTRY.json") +reset_registry_path() # back to normal resolution # Or set via environment variable -# export AIPASS_REGISTRY_PATH=/path/to/registry.json +# export AIPASS_REGISTRY=/path/to/registry.json ``` +Path resolution order (`get_registry_path()` → `find_registry()`): explicit `set_registry_path()` → **`AIPASS_REGISTRY`** env var → walk up from cwd, skipping any registry whose `metadata.id` conflicts with the nearest passport's `citizenship.registry_id` → `AIPASS_HOME` → walk up from the drone package → package-relative default. The env var is `AIPASS_REGISTRY`, not `AIPASS_REGISTRY_PATH` — the latter name appeared here until 2026-08-25 and was never read by any code. + ### Error Handling ```python @@ -167,7 +172,7 @@ drone/ │ │ ├── module_registry.py # Internal module routing │ │ ├── registry.py # Registry query operations │ │ ├── commands.py # Custom command shortcut orchestrator -│ │ ├── git_module.py # Git workflow (tier-based access, 22 commands) +│ │ ├── git_module.py # Git workflow (tier-based access, 24 commands) │ │ ├── scan.py # Branch command scanning │ │ ├── rm.py # Contained safe-delete orchestrator │ │ └── broker.py # Broker daemon orchestrator (sandbox delete) @@ -180,7 +185,9 @@ drone/ │ │ ├── module_registry_handler.py # Module loading (internal + external) │ │ ├── generic_adapter.py # StringIO capture for external modules │ │ ├── help_flags.py # wants_help() — whole-sequence help detection (rule E) +│ │ ├── json_flags.py # wants_json() / strip_json_flag() — --json in any slot │ │ ├── rm_handler.py # Path containment checks + deletion +│ │ ├── deletion_log.py # Deletion record — JSONL store + prax line (both lanes) │ │ ├── routing_config.json # External module declarations │ │ ├── broker/ │ │ │ ├── daemon.py # Broker daemon (unix socket, openat2, audit) @@ -202,6 +209,7 @@ drone/ │ │ ├── diff_handler.py # Scoped git diff (--staged support) │ │ ├── log_handler.py # Scoped git log (configurable count) │ │ ├── show_handler.py # Read history at a commit (repo-wide, NOT branch-scoped) +│ │ ├── remote_handler.py # List remotes (credentials redacted) │ │ ├── commit_handler.py # Commit changes (--all, selective files, or pre-staged) │ │ ├── checkout_handler.py # Branch switching (main/dev guard) │ │ ├── dev_pr_handler.py # Push dev and create PR to main @@ -224,7 +232,7 @@ drone/ ├── docs/ # Public documentation ├── docs.local/ # Investigation reports and policies ├── artifacts/ # Live acceptance test scripts -└── tests/ # 1019 tests across 25 test files +└── tests/ # 1205 tests across 32 test files ``` ### Routing Flow @@ -236,7 +244,9 @@ drone/ 5. **Bare module names** → auto-discovered from `apps/modules/*.py`, routed via `importlib` 6. **Custom commands** → greedy multi-word matching against `drone_command_registry.json` -There is **no module fallback on the error path.** A `BranchNotFoundError` from a branch that the registry *does* list is a real fault and fails loud — `resolve_branch()` also refuses a registry path that escapes the project root, and the old fallback answered that security refusal by quietly running the module instead. +The `@target` lane has **no module fallback on the error path.** A `BranchNotFoundError` from a branch that the registry *does* list is a real fault and fails loud — `resolve_branch()` also refuses a registry path that escapes the project root, and the old fallback answered that security refusal by quietly running the module instead. + +One fallback deliberately survives, and only in the **custom-command** lane (`_handle_custom_command()`, `apps/drone.py:376`): a registered shortcut whose target is a module but not a branch here falls back to module routing, logged at INFO. That lane resolves its target from `drone_command_registry.json` rather than the argv, so the look-before-route check the `@target` lane uses does not apply to it. Unverified whether it should — it has never been measured for happy-path firing the way `@git status` was. ### Module System @@ -294,9 +304,27 @@ Auth centralized via `verify_git_access()` in `apps/plugins/devpulse_ops/auth.py | **Owner** | The project's registry-declared owner — **earned, never hardcoded** (devpulse in AIPass) | `pr`, `commit`, `checkout`, `dev-pr`, `delete-branch`, `prune-temp`, `close-pr`, `sync`, `unlock`, `merge`, `smart-sync`, `fix`, `tag` | - Auth is checked once at the top of `git_module.handle_command()` before any handler is called -- Unauthorized commands are refused with a message naming the caller, its `citizen_class`, and the tier required +- A refusal names the caller and why it was refused — see the two refusal species below; only the authority species names `citizen_class` and the tier required - **A command in neither tier is unreachable, not merely ungated** — `verify_git_access()` refuses anything it cannot find in a tier as `Unknown git command`, so registering a verb in `_COMMANDS` and wiring it to a handler does not make it callable. `prune-temp` shipped that way and no caller could reach it (found in the APLAN-0003 audit, tier ruled by @devpulse). `test_every_registered_command_holds_a_tier` now asserts the rule rather than the instance +### Two refusal species — authority vs capability + +Owner tier refuses for two reasons that are not interchangeable, and conflating them cost a false page *and* a real hole (commit `2b7e6bcc`). + +| | **Authority** | **Capability** | +|---|---|---| +| What happened | the caller is not, or cannot be shown to be, this repo's owner | the caller **is** the proven owner, but the verb is not translated for this repo | +| Where it is decided | any of the four owner checks — `citizen_class: manager`, registry readable, registry tenancy, listed with `owner: true` + passport path-binding | only **after** all four pass | +| Log level | `ERROR` — fault-shaped, someone should look | `WARNING` — by design, nothing is broken | +| Message | `Branch 'x' is not authorized for 'pr': ` | `Branch 'x' cannot run 'pr' in this repo: ` | +| `AIPASS_GIT_AUTH_MODE=warn` | **lifts it** — rolling back the authority migration is exactly that switch's job (F59 6.1) | **does not lift it**, and cannot: the capability branch raises before `warn_only` is ever read | + +Why the wording differs: a proven owner sent to audit their passport is a false trail — they never had an authority problem. Why the rollback is scoped: one flag tested against every refusal species also lifted this wall, and `pr` ran to completion inside an external repo. A rollback named for one migration has no business re-arming a half-run of our merge flow in someone else's repository. + +**Which verbs refuse outside AIPass:** `dev-pr`, `pr`, `close-pr`, `merge`, `smart-sync`, `fix`, `delete-branch` — they assume a `dev` branch, our PR conventions, or `pyproject` versioning, so against an arbitrary repo they would half-run and leave a mess. `commit` and `sync` were translated in DPLAN-0281 P2, `tag` in DPLAN-0290 item 1; the refusal names those three so a manager who hits the wall learns what they *can* use. + +**WARNING is not silence.** @trigger's `watch_branch_log_warnings` feeds branch-log WARNINGs into the escalation digest — ten occurrences of one signature in 60 minutes mails @devpulse. A capability wall walked into repeatedly still reaches an operator, as a digest rather than a page here. + ### Subprocess timeouts Routed commands run with a timeout resolved in this order — **explicit flag > per-command policy > default**: @@ -384,6 +412,8 @@ Four things worth knowing: - **Severity is INFO on both channels** (compass #273). A deletion through the sanctioned path is chosen behaviour, not a fault. The guards keep their own WARNING when they refuse — that is the guard speaking, and it is a separate line from the record. - **Identity is resolved, never guessed.** `resolve_caller_identity()` — the same passport/registry resolver routing and git attribution use, not a fifth one and not path-shape matching. Unresolvable callers are recorded as `unknown`; a wrong-but-plausible name on a deletion record is worse than an honest gap. +**Known gap in the `caller` field — read it with this in mind.** The resolver's last resort before `unknown` is the *project*, not a citizen: with no `AIPASS_BRANCH_NAME` assigned and no `.trinity/passport.json` anywhere up the tree, it derives a name from the registry that answered (for AIPass, the `AIPASS_REGISTRY.json` filename → `aipass`). A delete run from the repo root therefore records `caller: "aipass"` — a directory, not the citizen who typed it. The live store carries 8 such records, one of them a deletion inside @devpulse's own tree. Nothing is fabricated: `aipass` is a true statement about *where* the process stood, and the same `CallerIdentity.source` distinction documented under Caller Identity applies (`project`, not `passport` or `assigned`). But a reader auditing a deletion months later wants the citizen, and for those records the ledger cannot supply one. Not fixed here — writing it down beats a reader inferring a person from a project name. + Both of drone's delete lanes feed it: `rm` (`handlers/rm_handler.py`) and `broker` (`handlers/broker/daemon.py`, which deletes on behalf of an HMAC-authenticated requester and therefore passes that identity in rather than reading its own cwd). The broker's protocol audit log is unchanged — that records requests and error codes; this records deletions. `AIPASS_DELETION_LOG` relocates the store (tests, containers). It cannot silence the prax line. @@ -441,7 +471,7 @@ Enforcement layers: ## Interactive Commands -By default, drone captures subprocess output (`capture_output=True`) with a 30s timeout. This is safe for AI-to-AI routing but strips Rich colors, buffers progress bars, and kills long-running commands. +By default, drone captures subprocess output (`capture_output=True`) with the resolved timeout — 60s unless a per-command policy or `--drone-timeout` says otherwise (see Subprocess timeouts). This is safe for AI-to-AI routing but strips Rich colors, buffers progress bars, and kills long-running commands. Commands in the interactive tuple bypass capture and inherit the terminal directly — enabling live Rich output, colors, and no timeout. @@ -501,7 +531,7 @@ Infrastructure modules (seedgo, cli, git, spawn) work from external AIPass proje **Dual registry lookup:** `registry_handler.py` merges local project registry with `AIPASS_HOME` registry. Local entries win on name collision. -**Module fallback:** When subprocess routing fails (branch not in local registry), drone falls back to module routing for registered modules. Graceful degradation: Rich output from AIPass, functional output from external projects. +**Module routing, not a fallback:** an external seat reaches `seedgo`, `cli`, `spawn` and `git` because `is_module()` is checked *before* any branch call (see Routing Flow) — the module lane is chosen, not discovered by failing the branch lane. Rich output from AIPass, functional output from external projects. The one surviving error-path fallback lives in the custom-command lane only. **AIPASS_HOME hints:** When `AIPASS_HOME` is not set and the local registry lacks core branches, drone shows setup hints: ``` @@ -529,12 +559,12 @@ Tip: set AIPASS_HOME=/path/to/AIPass to access all branches ## Testing -1199 tests collected across 30 test files (1194 pass, 5 skip), covering all layers. Counts below are pytest-collected, verified 2026-08-21: +1205 tests collected across 32 test files (1200 pass, 5 skip), covering all layers. Counts below are pytest-collected, verified 2026-08-25 — every file on disk appears in exactly one row, so the rows sum to the total: | Area | Files | Tests | |------|-------|-------| | Core routing | `test_resolver.py`, `test_router.py`, `test_activation.py`, `test_registry.py` | 183 | -| Git operations | `test_git_access.py`, `test_git_module.py`, `test_tag_handler.py`, `test_devpulse_plugins.py`, `test_system_pr.py` | 335 | +| Git operations | `test_git_access.py`, `test_git_module.py`, `test_tag_handler.py`, `test_devpulse_plugins.py`, `test_system_pr.py` | 341 | | Handlers | `test_registry_handler.py`, `test_discovery.py`, `test_executor.py` | 125 | | Commit gate | `test_commit_gate_branch_mapping.py` | 3 | | Infrastructure | `test_module_registry.py`, `test_config.py`, `test_generic_adapter.py` | 77 | @@ -545,6 +575,8 @@ Tip: set AIPASS_HOME=/path/to/AIPass to access all branches | Help-flag safety | `test_help_flag_safety.py` | 36 | | Module routing (no detour) | `test_module_route_no_detour.py` | 6 | | Caller identity provenance | `test_caller_identity_provenance.py` | 10 | +| Machine output (`--json` doors, `remote`) | `test_git_json_and_remote.py` | 50 | +| JSON log durability | `test_json_durability.py` | 10 | Run tests: `cd src/aipass/drone && python -m pytest tests/ -q` @@ -556,12 +588,13 @@ Run tests: `cd src/aipass/drone && python -m pytest tests/ -q` - `update_command()` and `command_exists()` in `ops.py` are tested CRUD API but unused from production - Piping drone output into a truncating reader (`| head`) yields inconsistent exit codes (0, 1, or 243) — no BrokenPipe handling anywhere in the tree. Cosmetic, but blocks `drone ... | head` inside `set -e` scripts - Several bypass rules in `.seedgo/bypass.json` are **line-scoped** and drift whenever code above them moves — adding a function to `drone.py` this session pushed four write sites down and dropped the audit to 99% until the rule was refreshed. The drift is a feature in one respect: it proves the rule is still load-bearing -- Pyright warns about `json` package name shadowing stdlib — works at runtime +- `apps/drone.py`'s file header says `Version: 1.1.1` while the runtime constant two lines down is `VERSION = "1.1.0"` — the header is the one that is wrong (`__init__.py`, the README and `drone --version` all agree on 1.1.0). Cosmetic, but a version header that disagrees with its own module is exactly what a truth pass exists to catch. Found 2026-08-25; a code fix, out of scope for a README-only pass +- Pyright's `json` package-shadowing warning could **not** be reproduced on 2026-08-25 (`pyright apps/handlers/json/json_handler.py` → 0 errors/0 warnings; a full run under the root `pyrightconfig.json` → 0 errors, 3 unrelated `reportUnusedExpression` warnings in tests). It may still surface from an editor opening this directory standalone, without the root config. Left listed rather than deleted, marked unreproduced — no evidence it was never real - Recurring sync errors when working tree is dirty — operational, not code bugs --- -**Seedgo:** 100% | **Tests:** 1194 pass, 5 skip | **Last Updated:** 2026-08-21 +**Seedgo:** 100% | **Tests:** 1200 pass, 5 skip | **Last Updated:** 2026-08-25 --- [← Back to AIPass](../../../README.md) diff --git a/src/aipass/flow/README.md b/src/aipass/flow/README.md index 8ee301af3..39c42fec4 100644 --- a/src/aipass/flow/README.md +++ b/src/aipass/flow/README.md @@ -6,7 +6,7 @@ **Module:** `aipass.flow` **Version:** 2.2.1 **Created:** 2025-11-15 -**Last Updated:** 2026-08-13 +**Last Updated:** 2026-08-25 --- @@ -16,11 +16,11 @@ Flow is AIPass's plan management system. Every branch uses flow to create, track ### What I Do - Create numbered plans from type-specific templates -- Close plans with foreground archival and vector intake verification +- Close plans with foreground archival, then hand vectorisation to a detached background runner - List and filter plans across all registered types -- Reopen closed plans whose file is still at its registered location - (recovery from the `.backup/processed_plans/` archive exists but is only - reached when the plan is absent from the registry entirely — see Known Issues) +- Reopen closed plans, pulling the file back from the + `.backup/processed_plans/` archive when it is no longer at its registered + location — which after a normal close it never is (see Known Issues) - Manage plan types via filesystem-driven template registry - Aggregate plans across branches for central reporting - Self-heal registries (orphan detection, auto-close missing files, auto-register new template dirs) @@ -47,6 +47,7 @@ drone @flow templates # List available plan types drone @flow create . "Subject" # Create FPLAN (default) drone @flow create . "Subject" master # Create FPLAN master template drone @flow create . "Design topic" dplan # Create DPLAN +drone @flow create . "Field note" cplan # Create CPLAN (any registered shorthand) # Close plans drone @flow close FPLAN-0042 # Close specific plan @@ -78,12 +79,22 @@ drone @flow --help # Full help drone @flow --version # Version string ``` -**Use the short verb.** Only the short form executes (`list`, `close`, `create`, -`restore`, `registry`, `aggregate`, `templates`). The module's full name +**Use the short verb.** Only the short form executes: `list`, `close`, `create`, +`restore`, `registry`, `aggregate`, and — all four owned by `template_manager` — +`templates`, `scan`, `register`, `unregister`. The module's full name (`list_plans`, `close_plan`, …) resolves for `--help` but is rejected by the dispatcher — `post`/`post_close_runner` is the sole module accepting both. The `--help` screen currently claims otherwise; see Known Issues. +**A bare number is not an identity.** Every per-type registry numbers from +`0001`, so `0012` names a row in each of them and a bare number resolves against +`fplan_registry.json` by default. Pass the typed ID (`close TDPLAN-0012`) when +the plan is not an FPLAN. The prefix is read by an **anchored** match +(`^([A-Z]+PLAN)-` in `apps/handlers/plan/registry_routing.py`), so `TDPLAN-0012` +resolves to `tdplan_registry.json` and never collides with `DPLAN-0012`. A row +whose `file_path` carries no prefix offers no type evidence at all; the bulk and +restore paths refuse such a row rather than guess. + --- ## Architecture @@ -94,7 +105,7 @@ flow/ │ ├── flow.py # Entry point (auto-discovers modules) │ ├── modules/ # Thin orchestrators (8 modules) │ │ ├── create_plan.py # Plan creation with template support -│ │ ├── close_plan.py # Closure with foreground archival + vector verify +│ │ ├── close_plan.py # Closure: foreground archival, background vectorisation │ │ ├── list_plans.py # Plan listing and filtering │ │ ├── restore_plan.py # Reopen closed plans (+ backup recovery path) │ │ ├── registry_monitor.py # Registry scanning and auto-healing @@ -102,7 +113,8 @@ flow/ │ │ ├── post_close_runner.py # Background post-processing with lock management │ │ └── template_manager.py # Template registry management │ └── handlers/ # Implementation details -│ ├── plan/ # Lifecycle: create, close, list, restore, display, validation +│ ├── plan/ # Lifecycle: create, close, list, restore, display, validation, project scope +│ ├── cli/ # Shared --help flag detection (help_flags.py) │ ├── registry/ # Load, save, auto-heal registries │ ├── template/ # Plan type loader, template resolution, registry CRUD │ ├── dashboard/ # Status push to local, central, branch dashboards @@ -119,7 +131,8 @@ flow/ │ ├── research_plans/ # RPLAN templates (default) │ ├── team_dev_plans/ # TDPLAN templates (default) │ ├── audit_plans/ # APLAN templates (default) -│ └── playbook_plans/ # PPLAN templates (SOPs: merge, weekly_update, …) +│ ├── playbook_plans/ # PPLAN templates (SOPs: merge, weekly_update, …) +│ └── capture_plans/ # CPLAN templates (default) ├── flow_json/ # Per-type registries + template_registry.json ├── tests/ # 950 tests across 27 files └── .archive/ # Archived legacy code + orphaned registries @@ -143,6 +156,7 @@ flow/ | team_dev_plans | TDPLAN | tdplan_registry.json | default | | audit_plans | APLAN | aplan_registry.json | default | | playbook_plans | PPLAN | pplan_registry.json | default, merge, prompt_change, weekly_update | +| capture_plans | CPLAN | cplan_registry.json | default | Plans follow the naming convention `{PREFIX}-{NNNN}_topic_slug_YYYY-MM-DD.md` where NNNN auto-increments per type. @@ -217,15 +231,34 @@ time. ## Close Pipeline -On `drone @flow close`: -1. **Template check** — fast-delete empty/template-only plans -2. **Mark closed** — update plan registry with closure timestamp -3. **Archive** — move to `.backup/processed_plans/` (foreground, sets processed/cleanup flags atomically) -4. **Vector intake** — `drone @memory process-plans` + `is_plan_vectorized()` verification -5. **Dashboard updates** — local, central, and branch dashboards -6. **Append** — write to `CLOSED_PLANS.local.json` - -Vector verification displays in console: "Vectorized: N chunks in chroma" or "NOT vectorized". +On `drone @flow close` — the console prints five numbered steps, with vector +intake fired unlabelled between steps 3 and 4: + +1. **`[1/5]` Template check** — *reports only, never deletes.* An empty + template gets the warning "looks like an empty template — closing and + archiving normally" and then flows through the identical pipeline. The old + fast-delete branch was removed deliberately: `is_template_content()` is a + heuristic, and its false positives permanently destroyed FPLAN-0370 and + FPLAN-0371. +2. **`[2/5]` Mark closed** — sets `status` and the `closed` timestamp, saves the + type's registry. **Close always succeeds from this point;** every later step + is non-blocking. +3. **`[3/5]` Archive** — move to `.backup/processed_plans/` (foreground; sets + `processed`/`processed_date`/`cleanup_completed`/`cleanup_date` and saves in + one write) +4. *(unlabelled)* **Vector intake** — spawns `apps/modules/post_close_runner.py` + detached; console shows only "Vectorizing in background" +5. **`[4/5]` Dashboard updates** — local, central, and branch dashboards +6. **`[5/5]` Finalizing** — append to `CLOSED_PLANS.local.json`, fire the + `plan_closed` trigger event + +**Close does not verify vectorisation, and cannot report it.** The runner is +launched with `subprocess.Popen(..., stdout=DEVNULL, stderr=DEVNULL, +start_new_session=True)` (`_spawn_background_runner`, `close_helpers.py`), so +its result is unreadable by the closing process by construction — a failed +vectorisation is silent. Nothing in flow calls `is_plan_vectorized()`; that +function lives in `@memory` and is reached only by the separate +`drone @memory verify