feat(metrics): generate strategy_metrics.json for AIAuditBridge watcher#119
Conversation
## Changes ### scripts/generate_strategy_metrics.py (new) - Reads monthly_shadow_build_summary.json and per-track release_index.csv - Extracts current (latest period) and baseline (all-time average) metrics - Writes strategy_metrics.json in the format expected by the AIAuditBridge strategy optimization watcher - Handles both official baseline and shadow candidate tracks ### .github/workflows/monthly_publish.yml - Run generate_strategy_metrics.py after shadow build completes - Include strategy_metrics.json in the monthly report bundle artifact so the AIAuditBridge watcher can download it The watcher gracefully skips metrics it doesn't understand, so omitting backtest-derived metrics (sharpe, cagr, etc.) is safe — they can be added as the pipeline evolves. Co-Authored-By: Claude <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
Co-Authored-By: Claude <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 080acfd2b7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
| continue | ||
|
|
||
| index = pd.read_csv(index_path) |
There was a problem hiding this comment.
When a shadow track produces zero eligible releases, build_shadow_release_history still creates release_index.csv from an empty DataFrame, which pandas writes as an effectively empty file. In that case this pd.read_csv(index_path) raises EmptyDataError before _track_metrics() can return empty metrics, so the new Monthly Publish step fails after publishing instead of emitting an empty snapshot for the watcher. This can happen for new/short-history candidate tracks or configurations where len(eligible) < pool_size for every scheduled release.
Useful? React with 👍 / 👎.
| python scripts/generate_strategy_metrics.py | ||
| env: | ||
| STRATEGY_METRICS_REPO: ${{ github.repository }} |
There was a problem hiding this comment.
When this workflow runs outside the default QuantStrategyLab/CryptoLivePoolPipelines repository, this step exports STRATEGY_METRICS_REPO but never passes it to the script, and generate_strategy_metrics.py does not read that environment variable. The generated artifact therefore embeds the hard-coded default repo, which the AIAuditBridge watcher rejects when its validated SOURCE_REPO is the actual fork/renamed repo, so metrics from those runs cannot be processed.
Useful? React with 👍 / 👎.
1 participant
背景
AIAuditBridge 的 Strategy Optimization Watcher 每天 cron 检查策略指标退化情况。它需要
strategy_metrics.json文件作为输入，该文件预期由monthly_publishworkflow 产出。问题
strategy_metrics.json目前不存在于任何地方。data/output/*被.gitignore排除，所以文件不会在 git 中。AI 审计桥需要从 CI artifact 中获取这个文件。此 PR 的变更
scripts/generate_strategy_metrics.py（新文件）monthly_shadow_build_summary.json和各 track 的release_index.csvdata/output/strategy_metrics.jsonstrategy_watch.pyschema.github/workflows/monthly_publish.ymlgenerate_strategy_metrics.pystrategy_metrics.json拷贝到 monthly report bundle artifact 中数据流
注意事项
当前包含的指标是
release_index.csv中可用的结构性指标（pool_stability、pool_churn 等）。AIAuditBridge 的默认退化策略检查sharpe、cagr、calmar、win_rate、max_dd这些回测衍生指标——它们在当前 pipeline 中尚未计算，因此 watcher 暂时不会发现退化信号。回测指标的接入可以在后续迭代中添加。🤖 Generated with Claude Code