Skip to content

Conservative cleanup of provisional scratch files after successful verification#313

Open
mmprotest wants to merge 1 commit into
mainfrom
mmprotest/add-cleanup-gate-for-scratch-artifacts
Open

Conservative cleanup of provisional scratch files after successful verification#313
mmprotest wants to merge 1 commit into
mainfrom
mmprotest/add-cleanup-gate-for-scratch-artifacts

Conversation

@mmprotest

Copy link
Copy Markdown
Owner

Motivation

  • Prevent transient helper/probing files created during model-driven shell execution from inflating patches and touched-file counts when they are not needed for the final solution.
  • Use provenance (pre/post snapshots and tool classification) and containment (root detection) rather than a hardcoded filename list to decide what may be safely removed.

Description

  • Track repo file snapshots before/after mutating tool calls and record new files created by Bash as provisional scratch candidates while recording explicit Write/Patch creations separately; added helpers _snapshot_repo_files and _new_files_since in villani_code/state.py and new runner fields to hold this provenance.
  • Add a conservative cleanup gate _cleanup_provisional_scratch_after_success in villani_code/state_runtime.py that filters candidates by newly-created-only, excludes modified existing files, avoids files inside detected source/test/package roots, aborts when roots cannot be detected, and only proceeds when there is exactly one visible verification command to rerun.
  • Implement deletion + reverify semantics that back up files, temporarily remove eligible candidates, rerun the identical visible verification command, restore files exactly on verification failure, and return the list of actually-removed files; record removals via an existing event flow scratch_cleanup_applied and set runner._cleaned_scratch_files so downstream validation sees the reduced changed-file set.
  • Keep the change isolated to finalization/cleanup and provenance capture during tool execution without modifying planning, prompting, model calls, or scoring; changes touch villani_code/state.py, villani_code/state_runtime.py, and tests.

Testing

  • Added focused unit tests in tests/test_state_runtime.py covering removal-on-success, restore-on-failure, explicit-write-created source/test file exclusion, preservation of modified existing files, preservation of files inside detected source roots, and skipping cleanup when root detection is uncertain.
  • Ran the focused tests with pytest -q tests/test_state_runtime.py -k 'cleanup or explicit_write_created' and the selection passed (7 passed, 32 deselected).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant