Skip to content

Initialize new run w/replay buffer of old run - #372

Merged
jonbinney merged 29 commits into
mainfrom
jdb/train-on-existing-selfplay-games
Jul 5, 2026
Merged

Initialize new run w/replay buffer of old run#372
jonbinney merged 29 commits into
mainfrom
jdb/train-on-existing-selfplay-games

Conversation

@jonbinney

Copy link
Copy Markdown
Owner

Instead of just the model from an old run. There are two reasons for this:

  1. If we initialize with the old model but an empty replay buffer, then we spend a bunch of time overfitting to the first few self play games the new run creates.
  2. When we change the structure of the model or the features, it isn't possible to directly use the model weights from a previous run. In this case, we now have the option to start with a replay buffer containing relatively "Smart" runs to train the model with the new structure.

I'll leave this up for a couple days before merging in case @adamantivm @alejandromarcu want to add any thoughts

jonbinney and others added 29 commits June 4, 2026 13:16
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The trainer trims by game count (len(moves_per_game) >
replay_buffer_size at trainer.py:208), not by cumulative moves.
The original spec and plan described the preload as a moves
budget; this corrects the spec wording and adds a historical
amendment to the plan. The buggy code is fixed in a follow-up
commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jonbinney
jonbinney merged commit a184a2d into main Jul 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants