backtest: implement walk-forward optimization and grid search#13
Merged
marwinsteiner merged 7 commits intomainfrom Feb 22, 2026
Merged
backtest: implement walk-forward optimization and grid search#13marwinsteiner merged 7 commits intomainfrom
marwinsteiner merged 7 commits intomainfrom
Conversation
Module docstring, all imports, class/function stubs with NotImplementedError, full type annotations, and Google-style docstrings. Test file includes class stubs and helper functions.
Uses itertools.product over value lists. Supports __iter__ and __len__. Empty grid yields a single empty dict. Includes 6 tests covering basic grid, single param, empty grid, length, and re-iteration.
Three frozen Pydantic v2 models: GridSearchResult (best_params, best_score, all_results), WalkForwardSplit (split boundaries and OOS result), WalkForwardResult (splits, combined equity, combined metrics). Includes 5 tests: construction, immutability, and JSON serialization round-trip.
Evaluates every ParameterGrid combination via run_vectorized_backtest. Sorts results by chosen metric (descending for most, ascending for max_drawdown). Validates metric against BacktestResult fields. Includes 6 tests: basic search, best-params ordering, max_drawdown ascending sort, single-param grid, all-results population, and invalid metric error.
Generates expanding train/OOS index tuples for walk-forward analysis. Training always starts at index 0 and grows by oos_step each split. Last split absorbs any remainder from integer division. Validates n_splits >= 1, train_ratio in (0, 1), and sufficient data length. Includes 7 tests: basic splits, expanding window, no overlap, length, single split, coverage, and too-short-data error.
For each split: grid_search on training data, then backtest OOS with the best parameters. OOS equity curves are scaled and concatenated to produce a combined performance estimate. Combined metrics computed via summarize_backtest. Includes 5 tests: basic walk-forward, OOS equity concatenation, combined metrics, split params variation, and invalid n_splits error.
8 additional edge-case tests: multi-param grid search, total_return metric, commission/slippage pass-through, 3-param grid, invalid train_ratio, equity continuity across walk-forward splits, combined metrics length, and single-split walk-forward. Applied ruff format. Full suite: 688 passed, 10 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements
src/sysls/backtest/optimize.pywith walk-forward analysis, grid search, and parameter optimization for the vectorized backtester.dict[str, list[Any]]viaitertools.product. Iterable with__len__.run_vectorized_backtest, sorts by chosen metric (ascending formax_drawdown, descending otherwise). Validates metric againstBacktestResultfields.train_ratioandn_splits. Last split absorbs integer division remainder.Key design choices
BacktestResultimported at runtime (Pydantic needs it for field resolution)run_vectorized_backtestandsummarize_backtestimported at runtime in function bodiesCommit discipline (7 atomic commits)
Test plan
tests/backtest/test_optimize.py- all passingruff check src/ tests/- all passedruff format --check src/ tests/- all passed