Add deterministic replay gate for agent tests #1271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

ZackMitchell910 wants to merge 1 commit into AgentOps-AI:main from ZackMitchell910:runledger/replay-gate

ZackMitchell910 commented Dec 19, 2025 •

edited

Loading

Summary

add a replay-only RunLedger eval suite (suite/case/schema/cassette + stub agent)
add a baseline file for regression gating
add a GitHub Actions workflow using runledger/[email protected]
add a small README note + ignore runledger_out/

How to run locally

runledger run evals/runledger --mode replay --baseline baselines/runledger-demo.json

Notes

no external calls; replay-only cassette
feel free to remove the suite/workflow if it is not desired
GitHub Actions note: workflows from first-time contributors/forks may require a maintainer to click “Approve and run” before checks will execute.


          Add RunLedger replay gate

bc49ac6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet