Skip to content

memory.md

Latest commit

 

History

History
279 lines (188 loc) · 13 KB

memory.md

File metadata and controls

279 lines (188 loc) · 13 KB
Raw

Memory System

Codexkit uses a Codex-native, repo-visible memory model.

The default baseline is:

  • Codex primitives such as AGENTS.md, config layering, session history, codex resume, and compaction
  • curated repository docs under .codex/context/
  • durable markdown memory under .codex/memory/

Codexkit does not ship a native semantic memory engine of its own. Its default memory model is still file-based and git-visible, but it is more than ad-hoc notes: it is a structured workflow for durable project context that fits how Codex actually operates today.

This baseline is useful, but it is not the same thing as:

  • automatic semantic retrieval
  • cross-session ranking over large note corpora
  • cross-project user memory
  • opaque background distillation services

Those require either stronger native Codex retrieval capabilities as they mature, or an MCP-backed memory service. See docs/memory-mcp.md for the optional additive setup path.

What Codex Actually Gives You

From the current public Codex docs and config surface, the stable memory-relevant primitives are:

  • AGENTS.md and ~/.codex/AGENTS.md for durable working agreements
  • config.toml for user/project configuration
  • session history and history persistence controls
  • codex resume
  • project-doc discovery and compaction/context controls
  • native Codex memories when [features].memories = true

Codexkit builds on those primitives. It does not turn them into a full retrieval memory system on its own.

Stable vs Experimental

For Codexkit purposes:

  • Stable means publicly documented and safe to reference in default docs and workflows
  • Experimental means visible but not yet reliable enough to become foundational repo guidance
  • Observed means seen in issues, PRs, community reports, or local experiments, but not strong enough to anchor design

As native Codex memory capabilities mature further, Codexkit can adopt them more deeply. Repo-visible docs remain the trustworthy base layer.

Native Codex memories Feature

Codex CLI includes a native memories subsystem that can be enabled via config.toml:

[features]
memories = true

[memories]
generate_memories = true
use_memories = true
disable_on_external_context = false

Current status: The memories feature flag is documented in Codex CLI and exposes memory-generation and memory-injection controls in [memories].

Codexkit stance: Treat memories as enabled-by-default but additive. It complements the file-based memory model but does not replace it. Repo-visible docs (.codex/context/ and .codex/memory/) remain the canonical source of truth for project knowledge because they are:

  • git-tracked and reviewable by teammates
  • inspectable without Codex CLI running
  • portable across tools (Cursor, Claude Code, etc. can read the same files)

When to enable: Enable [features].memories = true when you want Codex to retain and reuse session context automatically. This works alongside file-based memory. Durable project decisions should still be written back to repo files.

When not to enable: If your workflow already uses /prompts:handoff and /prompts:resume effectively, and you prefer full control over what persists, the file-based model alone may be sufficient.

Directory Structure

.codex/context/
├── index.md
├── architecture.md
├── conventions.md
├── current-priorities.md
├── decision-log.md
├── worklog.md
├── gotchas.md
└── session-context.md

.codex/memory/
├── README.md
├── _templates/
├── project/
└── research/

.codex/plans/                          # Stored implementation plans (sibling to memory)

Document Purposes

index.md

The entry point. Maps all available context documents with brief descriptions. The agent reads this first when starting a session to understand what project knowledge is available.

architecture.md

Documents system boundaries, key modules, data flows, and integration points. Describes the high-level structure so the agent understands where things live and how they connect.

conventions.md

Captures coding norms: naming conventions, file organization patterns, preferred libraries, testing standards, and style rules specific to the project.

current-priorities.md

Lists what matters right now: active features, known bugs, upcoming deadlines. Updated frequently to keep the agent focused on current work.

decision-log.md

Records durable architecture decisions using a lightweight ADR format. Decisions logged here persist across sessions and prevent the agent from re-litigating settled choices.

worklog.md

A session-level trail of what happened. Updated after significant milestones. It should summarize meaningful progress, not act as a raw transcript.

Durable Context Layers

Layer Location Purpose
Context docs .codex/context/*.md Curated project knowledge and active session trail
Project memory .codex/memory/project/*.md Always-available durable facts and workflow state
Research notes .codex/memory/research/*.md Deep investigations and reference notes
Templates .codex/memory/_templates/*.md Fast creation of structured memory docs

Canonical Storage Boundaries

One memory type should have one canonical home:

Memory Type Canonical Home Should Contain Should Not Contain
Architecture decisions .codex/context/decision-log.md durable decisions, rationale, consequences session breadcrumbs, TODO streams
Current stop point .codex/context/session-context.md compact next-step state for resume long history, duplicated worklog
Session trail .codex/context/worklog.md milestone summaries, notable progress every command, raw transcripts
Durable project facts .codex/memory/project/*.md stable facts, preferences, long-lived state ephemeral blockers that expire quickly
Deep investigations .codex/memory/research/*.md deep-dive notes, synthesized research canonical architecture decisions

If you introduce a new memory location, define its boundary first. Do not create overlapping stores that compete with decision-log.md, worklog.md, session-context.md, or memory/project/state.md.

What This Replaces, And What It Does Not

Codexkit intentionally replaces ad-hoc note dumping with curated project context.

Capability Codexkit default Real memory system
Durable project guidance Yes Yes
Human-readable / git-trackable Yes Sometimes
Automatic semantic retrieval No Yes
Cross-session search/ranking Limited to file search Yes
Cross-project user memory No Usually yes
Background distillation No Usually yes

The key insight is narrower than "memory replaces memory": curated knowledge beats opaque note piles for many repo-level workflows. But if you need true retrieval memory, use MCP.

Optional MCP-Backed Memory

If your team wants real long-term memory, add an MCP memory service instead of stretching markdown files beyond their limits.

Examples:

  • a Basic Memory MCP server for durable searchable notes
  • another team-approved memory MCP when native memories are not enough

Recommended stance:

  1. Use .codex/context/ and .codex/memory/ as the default durable repo context.
  2. Add MCP-backed memory only when you truly need stronger retrieval or cross-session search.
  3. Keep repo-curated docs as the source of truth for decisions and conventions even if MCP memory is added.

For a concrete Codex-oriented example, see docs/memory-mcp.md.

Best Practices

Keep Documents Short

Context documents should be concise. If a file exceeds ~200 lines, it is too long. Summarize, archive old entries, or split into focused documents.

Update at Milestones

Update context docs after significant milestones, not after every change. Good triggers:

  • completing a feature
  • making an architecture decision
  • discovering a non-obvious convention
  • ending a work session

Curate Actively

Context docs are not append-only logs. Remove outdated information, consolidate duplicates, and rewrite unclear sections. Treat them like living documentation.

Run Hygiene Checks

Use the built-in maintenance helpers before memory drift gets expensive:

bash .codex/scripts/audit_memory_hygiene.sh
bash .codex/scripts/prepare_worklog_archive.sh /tmp/worklog-archive-draft.md 20 --retained-output /tmp/worklog-retained-draft.md
bash .codex/scripts/apply_worklog_archive.sh /tmp/worklog-archive-draft.md /tmp/worklog-retained-draft.md

The audit script is non-destructive and reports line-budget, boundary, and duplication issues. The archive helper is also non-destructive: it can prepare both an archive draft and a retained-worklog draft so you can review them before editing worklog.md.

Freshness Schedule

Use a lightweight review cadence instead of waiting for memory drift to become painful:

Cadence Review Action
End of session session-context.md, state.md keep stop point exact; remove stale blockers
Weekly worklog.md, current-priorities.md summarize completed items; archive old worklog bullets
Monthly decision-log.md, memory/project/*.md remove stale facts; tighten decisions that no longer matter
Before release architecture.md, conventions.md, key research notes ensure durable docs still match the shipped system

Treat a note as stale when it no longer changes implementation choices, duplicates a newer source, or reflects a blocker or priority that is already resolved.

Do Not Overclaim Native Memory

Do not describe experimental or community-observed Codex memory features as if they are stable defaults. If you mention them, label them clearly and keep baseline workflows independent from them.

Version with the Repo

Context and memory docs live under .codex/ and should be committed to the repository. This way, every developer and every agent session gets the same project knowledge.

Use index.md as the Map

Always keep index.md up to date. If you add a new context document, add it to the index. If you remove one, update the index. The agent reads index.md first.

Session Workflow

  1. Session start: Agent reads context/index.md and relevant memory/project/*.md files.
  2. During work: Agent consults context docs, project memory files, and research notes as needed.
  3. After milestones: Agent updates worklog.md and durable context files if new facts emerged.
  4. Decisions made: Agent records them in decision-log.md.
  5. Session end: memory/project/state.md and context/session-context.md capture the stop point.

Maintenance Workflow

When context starts feeling noisy:

  1. Run bash .codex/scripts/audit_memory_hygiene.sh
  2. If worklog.md is oversized, generate review drafts with bash .codex/scripts/prepare_worklog_archive.sh /tmp/worklog-archive-draft.md 20 --retained-output /tmp/worklog-retained-draft.md
  3. Review the archive and retained drafts, then apply them with bash .codex/scripts/apply_worklog_archive.sh /tmp/worklog-archive-draft.md /tmp/worklog-retained-draft.md
  4. Remove stale blockers from memory/project/state.md
  5. Keep decision-log.md limited to still-relevant durable decisions

Retrieval Strategy

Use the cheapest truthful layer first:

  1. AGENTS.md and .codex/context/ for instructions, conventions, and active project context
  2. .codex/memory/project/ and .codex/memory/research/ for durable project knowledge
  3. file/code search for repo-visible retrieval
  4. optional MCP-backed memory only when repo docs are not enough

This keeps Codexkit aligned with Codex’s real operating model: repo-visible context first, stronger retrieval only when necessary.

Lightweight Derived Index

Codexkit also ships a lightweight local retrieval layer that stays secondary to repo docs:

bash .codex/scripts/build_memory_index.sh
bash .codex/scripts/memory_search.sh "session lifecycle" --limit 5
CODEXKIT_MEMORY_BACKEND=jsonl bash .codex/scripts/memory_search.sh "architecture decision" --decisions-only --limit 3

Properties of this layer:

  • the index is built only from repo-visible markdown under .codex/context/ and .codex/memory/
  • the index is disposable and can be rebuilt at any time
  • search results point back to canonical repo files
  • this is not a database of truth, only a retrieval accelerator

The current implementation uses a derived SQLite/FTS index plus a JSONL export under .codex/memory/.index/. If you delete that directory, no durable knowledge is lost.

Runtime Controls

Project-local retrieval can be tuned without changing source files:

  • CODEXKIT_MEMORY_BACKEND=auto|sqlite|jsonl
  • CODEXKIT_ADVANCED_MEMORY_MODE=off|basic-memory|bridge

Use jsonl when you want a simpler deterministic fallback, and use sqlite when you want to fail fast if the richer local index is unavailable.