Reduce archived conversation disk usage with cold storage

[Quicksaver]

Summary

This adds compressed cold storage for archived conversations. Once a thread is archived, its conversation data and attachments are moved out of the hot state database into a dedicated archive.sqlite bundle, while unarchiving restores the complete thread tree before applying the domain command.

The lifecycle is durable and restart-safe: migrations 035/036 queue existing archived and deleted threads, background work resumes interrupted archive/delete jobs, and restoration keeps the cold bundle authoritative until the unarchive command commits successfully. Attachment ownership is preserved from exact persisted IDs and cold-bundle filenames throughout archive, restore, and durable deletion.

Archived thread shells remain consistent across clients without relying on compacted event history: each WebSocket session reconciles an authoritative shell snapshot, then replays live changes without losing archive removals during the handoff.

What Changed

• Added a dedicated cold-storage service that gzip-compresses thread-scoped SQL rows and attachments into chunked records in archive.sqlite.
• Removed successfully archived rows, attachments, and per-thread provider logs from hot storage, with incremental vacuuming to reclaim database pages.
• Restored archived thread trees transactionally before unarchive commands, with rollback on command failure and cleanup of the cold bundle only after a successful commit.
• Reserved still-hot archived rows before unarchive dispatch so queued lifecycle work cannot move them cold between the restore check and command commit, and rejected unarchive when storage cannot restore or reserve the conversation.
• Scoped restored reservations to their original archive timestamp so a failed post-commit bundle finalization cannot block a later archive from moving the conversation cold.
• Serialized archive-tree lifecycle operations with idle lock eviction, restored cleanup_pending bundles, and prevented queued archive work or post-commit failures from undoing a successful restore.
• Added migrations 035/036 with persistent manifests, maintenance records, and delete queues so interrupted work and legacy archived/deleted threads are recovered after restart without replacing fork-local migration metadata.
• Rechecked archived shell state at the destructive transaction boundary and required active provider, terminal, and log writers to quiesce before archiving hot rows.
• Changed thread deletion to permanently remove hot rows, cold bundles, attachments, provider logs, and related cleanup records.
• Kept non-missing attachment/provider-log directory failures retryable instead of treating them as empty directories.
• Matched attachment ownership by exact persisted IDs and cold-bundle filenames so colliding sanitized thread segments cannot remove another live thread's files.
• Resumed cleanup_pending manifests even when the archived shell row is missing, while preserving exact attachment metadata until durable delete cleanup succeeds.
• Added one-time compaction of legacy hot storage and startup processing for pending archive/delete work.
• Closed per-thread provider log writers before lifecycle cleanup so files can be removed safely.
• Applied optimistic archive visibility consistently to rendered rows, project sorting, and keyboard navigation on web, and kept in-progress unarchive states across the web and mobile archive interfaces.
• Refreshed thread shells from an authoritative socket-owned snapshot on every completion-capable WebSocket session, with live buffering before the projection read so archived and deleted conversations cannot linger after event compaction.
• Added the supported t3-sqlite-state inspection path for both hot state.sqlite data and cold archive.sqlite manifests/chunks.
• Added coverage for archive/restore round trips, binary SQL values, attachment handling, schema evolution, bounded restore reads, failure recovery, migrations, orchestration integration, and sidebar archive visibility.

Why

With fairly moderate usage, the user .t3 folder grew to more than 30 GB. That is far too much local data growth for normal use, and without lifecycle controls it gets out of hand very quickly. Archived and deleted conversations need to stop accumulating indefinitely in hot storage while remaining reliably recoverable when a user unarchives them.

Validation

• pnpm exec vp check passed with exit 0; it reported 9 pre-existing lint warnings outside the changed files.
• pnpm exec vp run typecheck passed with exit 0.
• pnpm exec vp run lint:mobile passed with exit 0; optional SwiftLint, ktlint, and detekt checks were skipped because those tools are not installed locally.
• Focused archive storage, deletion, orchestration, migration, SQLite inspection, server snapshot handoff, client shell sync, sidebar, and mobile archive tests passed: 10 files, 214 tests.
• Playwright against isolated ports 5213/14213 created and archived a disposable thread; after the exit animation, the row was absent from the rendered and keyboard-addressable sidebar and the project showed No threads yet.
• Playwright against isolated ports 5736/13776 paired successfully, rendered the authoritative empty shell, connected both HMR and application WebSockets, and reported no console or page errors.

Proof

No attached visual proof is necessary; the data lifecycle and sidebar interaction are covered by the automated and isolated Playwright validation above.

---

Note

High Risk
Large change to conversation persistence, deletion, migrations that backfill all archived/deleted threads, and orchestration dispatch around unarchive—data loss or stuck states would be high impact if lifecycle boundaries fail.

Overview
Adds gzip-compressed cold storage in a separate archive.sqlite so archived threads keep only a lightweight shell in state.sqlite while messages, events, attachments, and related projections move into chunked bundles; provider/terminal logs are dropped on archive and are not restored.

Server lifecycle is extended with ThreadColdStorage, migrations 035/036 (manifests, delete queue, legacy compaction), and a background worker that archives on thread.archived, permanently deletes on thread.deleted, and resumes pending work after restart. Unarchive goes through restore → domain command → finalize or rollback so failed commands re-archive and successful commits drop the cold bundle without unrelated unarchives wiping data.

Clients hide archived threads optimistically in the sidebar (web) and show in-progress unarchive on web/mobile; shell sync revalidates via a socket-owned snapshot when completion markers are supported so archive removals are not lost after event compaction. t3-sqlite-state gains --database archive for inspecting cold manifests/chunks.

^{Reviewed by Cursor Bugbot for commit 6c124a6. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add cold storage archival and deletion lifecycle for conversations

• Introduces a ThreadColdStorage service and ThreadColdArchive / DeletedThreadCleanupQueue migrations (035/036) that move archived and deleted thread data to a separate archive.sqlite database, compressing row data with gzip.
• Extends ThreadDeletionReactor to handle thread.archived and thread.deleted events by enqueuing archive, delete, and legacy-compaction jobs; on startup it backfills pending jobs from the cold storage manifest tables.
• Wraps thread.unarchive dispatch in OrchestrationEngine with restore, rollback, and finalization steps against cold storage so unarchiving is transactional.
• Adds optimistic archive UI on web and mobile: threads disappear from the sidebar immediately on archive and show a spinner/disabled state during unarchive, with rollback if the command fails.
• Extends the t3-sqlite-state CLI with a --database state|archive flag to inspect either database; archive.sqlite path is now derived from ServerDerivedPaths.
• Risk: the cold storage compaction job runs on every server startup; existing databases will be migrated by migrations 35/36 which backfill manifests and cleanup queues for all currently-archived or deleted threads.

^{Macroscope summarized 6c124a6.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: e8553676-3a60-4c2d-8274-423272cb8206

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a complete cold storage system for archived conversations, including new database schema, compression/restore logic, durable lifecycle management, and UI changes. The scope and complexity of this new feature warrant thorough human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[macroscopeapp]

One Effect service convention issue found in the new ThreadColdStorage service. See the inline comment.

^{Posted via Macroscope — Effect Service Conventions}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 925af20. Configure here.}