Add configurable delay for sandbox cleanup after automation runs complete

[jpshackelford]

Summary

This PR implements configurable sandbox cleanup delay as requested in issue #31. Sandboxes can now remain available for inspection after automation runs complete, making it easier to debug automation failures.

Changes

Configuration

• Added AUTOMATION_SANDBOX_CLEANUP_DELAY_MINS environment variable (default: 60 minutes)
• When set to 0: immediate cleanup (legacy behavior)
• When > 0: sandboxes remain available for that many minutes after run completion

Database

• Added cleanup_at column to automation_runs table (nullable DateTime with index)
• New migration: 003_add_cleanup_at.py

Code Changes

automation/config.py

• Added sandbox_cleanup_delay_mins setting

automation/models.py

• Added cleanup_at field to AutomationRun model

automation/router.py

• Modified complete_run to set cleanup_at instead of immediate cleanup when delay > 0

automation/watchdog.py

• Added _compute_cleanup_at helper function
• Modified _verify_and_mark_run to set cleanup_at instead of immediate cleanup
• Added cleanup_pending_sandboxes function that processes runs past their cleanup deadline
• Updated watchdog_loop to call cleanup scanner each interval

automation/schemas.py

• Added cleanup_at field to AutomationRunResponse

Tests

• Added tests for _compute_cleanup_at
• Added tests for delayed cleanup behavior in watchdog
• Added tests for cleanup_pending_sandboxes function
• Added tests for complete_run endpoint cleanup behavior
• Added tests for the new config setting

SDK Behavior Note

Regarding the question in the issue comments:

if the user provides custom SDK code using OpenHandsCloudWorkspace(keep_alive=False) will the sandbox be shutdown by the SDK script at the end of its execution lifecycle regardless of our scheduler?

Answer: The SDK's OpenHandsCloudWorkspace does not directly delete the sandbox. When the context manager exits, it calls the automation service's callback endpoint (/runs/{run_id}/complete), and the automation service decides whether to cleanup based on:

1. The run's keep_alive flag (set on the AutomationRun model)
2. The sandbox_cleanup_delay_mins setting

So the SDK's keep_alive parameter in OpenHandsCloudWorkspace is not directly related to sandbox deletion control - that's handled entirely by the automation service.

Example Usage

# Keep sandboxes available for 2 hours after completion
export AUTOMATION_SANDBOX_CLEANUP_DELAY_MINS=120

# Immediate cleanup (legacy behavior)
export AUTOMATION_SANDBOX_CLEANUP_DELAY_MINS=0

Testing

The unit tests pass with the new behavior. Integration tests require Docker and were not run in this environment.

Closes #31

---

This PR was created by an AI assistant (OpenHands) on behalf of a user.

@jpshackelford can click here to continue refining the PR

[github-actions]

[github-actions]

🚀 Deploy Preview PR Created/Updated

A deploy preview has been created/updated for this PR.

Deploy PR: https://github.com/OpenHands/deploy/pull/3638
Automation SHA: eadea664d51fe762d703c5e6a31e12c9af390d1a
Last updated: Apr 01, 2026, 07:29:44 PM ET

Once the deploy PR's CI passes, the automation service will be deployed to the feature environment.

[rajshah4]

Updated implementation ready to merge 🟢

This PR was written against an older repo structure (automation/ → now openhands/automation/). I have ported all changes to the current structure, resolved the merge conflicts, run the test suite (684 passing, 0 failing), and the implementation is ready.

Changes ported to current main structure

File	Change
openhands/automation/config.py	sandbox_cleanup_delay_mins: int = 60 on ServiceSettings
openhands/automation/models.py	cleanup_at column on AutomationRun
openhands/automation/router.py	complete_run sets cleanup_at for deferred deletion (or immediate if delay=0)
openhands/automation/watchdog.py	_compute_cleanup_at, cleanup_pending_sandboxes, updated _verify_and_mark_run and loop
migrations/versions/009_add_cleanup_at.py	New migration (current main is at 008)

Test results

684 passed, 0 failed, 32 skipped
(200 errors = Docker/testcontainers not available in CI sandbox — same as pre-existing)

To apply

Someone with push access to this repo (e.g. @robamesbury or @rohitMalhotra123) can apply the patch:

git checkout -b feat/sandbox-cleanup-delay main
git apply automation-cleanup-delay.patch
git push origin feat/sandbox-cleanup-delay

Or ping @rohitMalhotra123 who greenlit this PR — ready to merge once pushed.

Linear: APP-2287

---

This comment was created by an AI agent (OpenHands) on behalf of Rajiv Shah.