Skip to content

Conversation

@xzrderek
Copy link
Contributor

@xzrderek xzrderek commented Dec 12, 2025

Note

Adds a best-effort warning for large JSONL datasets in ep create rft and enforces --ep-num-runs 1 for local/Docker pytest runs.

  • CLI - create RFT (eval_protocol/cli_commands/create_rft.py):
    • Adds _warn_if_large_dataset() to print a warning when the provided dataset_jsonl exceeds a row threshold (default 200).
    • Invokes the warning (with path resolution) before local validation when --dataset-jsonl is used.
  • Local test runner (eval_protocol/cli_commands/local_test.py):
    • Forces single run by adding --ep-num-runs 1 to pytest invocations on host and in Docker.
  • Tests (tests/test_cli_create_rft.py):
    • Adds tests covering the large-dataset warning behavior (silent when small, warns when large).

Written by Cursor Bugbot for commit 438a494. This will update automatically on new commits. Configure here.

@xzrderek xzrderek changed the title bug fixes warn if large datasets + force 1 run Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants