Skip to content

fix(agent-challenge): owned CVM teardown, fail-loud list, staging stack #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
echobt wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(agent-challenge): owned CVM teardown, fail-loud list, staging stack #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,23 @@ UV_CACHE_DIR=/var/tmp/uv-cache uv run pytest tests/unit \
-k "sealer or aggregation or weights" -q
```

## Agent Challenge local staging (before live/prod)

Prefer the isolated AC staging loop before production-facing gate changes:

```bash
packages/challenges/agent-challenge/scripts/staging/run_staging.sh
```

Details: [`packages/challenges/agent-challenge/docs/staging.md`](packages/challenges/agent-challenge/docs/staging.md)
(host loopback `127.0.0.1:18082`, project `ac-staging`; not master embed `:18081`).

- **One command** above is the iteration loop. Driving the **prod** validator over SSH is a last resort and must never be the day-to-day loop.
- Any keypair works for local submit/sign: AC verifies signatures only (no metagraph membership check).
- CVMs are **real** Phala TDX machines (billable). Staging tears down **only CVMs this run owns** (`work/owned_cvms.txt` + per-run track). It never account-sweeps foreign/prod CVMs. Always tear down owned CVMs before you leave.

Real Phala TDX CVMs + dual attestation flags; always tear down to a verified CVM count of 0 via paginated list (never trust bare `GET /cvms` empty arrays).
Comment on lines +96 to +98

@coderabbitai coderabbitai Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Preserve the owned-CVM teardown boundary consistently.

The documentation mixes owned-only cleanup with account-wide zero-count and sweep language, which could lead to deletion of foreign or production CVMs.

  • AGENTS.md#L96-L98: verify that only the run-owned CVM count reaches zero; do not require an account-wide zero.
  • packages/challenges/agent-challenge/docs/staging.md#L102-L102: state that --down removes only owned CVMs and does not sweep the account.
📍 Affects 2 files
  • AGENTS.md#L96-L98 (this comment)
  • packages/challenges/agent-challenge/docs/staging.md#L102-L102
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@AGENTS.md` around lines 96 - 98, The documentation must consistently define
teardown as owned-CVM-only. In AGENTS.md, replace account-wide zero-count
language with verification that the run-owned CVM count reaches zero, while
preserving paginated verification. In
packages/challenges/agent-challenge/docs/staging.md, update the --down
documentation to state that it removes only owned CVMs and never sweeps
account-wide or foreign/production CVMs.


## Runtime topology (production)

Supported install is **Docker Compose master + PostgreSQL only**:
Expand Down
71 changes: 71 additions & 0 deletions packages/challenges/agent-challenge/docker-compose.staging.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Local Agent Challenge staging stack — isolated from prod.
# Host port 18082 (prod embed uses 18081 inside master). Named volume + project
# prefix keep DB/artifacts off any prod path.
#
# Build from monorepo root:
# docker compose -f packages/challenges/agent-challenge/docker-compose.staging.yml build
# Or use scripts/staging/run_staging.sh (preferred).

name: ac-staging

services:
agent-challenge:
image: ghcr.io/baseintelligence/agent-challenge:staging-local
build:
context: .
dockerfile: Dockerfile
target: runtime
additional_contexts:
monorepo: ../../..
container_name: ac-staging-validator
restart: "no"
ports:
- "127.0.0.1:18082:8000"
volumes:
- ac_staging_data:/data
- ./scripts/staging/config/review_evidence_encryption_key:/run/secrets/base/review_evidence_encryption_key:ro
- ./scripts/staging/config/challenge_token:/run/secrets/base/challenge_token:ro
# Frozen Terminal-Bench 2.1 digest (eval/prepare fails closed without this).
- ./golden:/app/golden:ro
- ./golden:/opt/agent-challenge/golden:ro
# dcap-qvl is baked into the runtime image; host bind is optional fallback
- /root/.cargo/bin/dcap-qvl:/usr/local/bin/dcap-qvl:ro
Comment on lines +31 to +32

@coderabbitai coderabbitai Bot Jul 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Unconditional host bind of dcap-qvl will shadow the baked binary on hosts lacking /root/.cargo/bin/dcap-qvl.

Docker creates an empty directory at a missing bind source, then mounts it over /usr/local/bin/dcap-qvl, turning the baked binary into a directory and failing attestation startup. A "fallback" bind can't be conditional in a compose file — move it to an opt-in override (e.g. docker-compose.staging.dcap.yml) instead.

🔧 Proposed change 
-      # dcap-qvl is baked into the runtime image; host bind is optional fallback
-      - /root/.cargo/bin/dcap-qvl:/usr/local/bin/dcap-qvl:ro
+      # dcap-qvl is baked into the runtime image. If you need a host override,
+      # add it via an extra `-f` override file rather than here — a missing
+      # source path would be auto-created as a directory and shadow the binary.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# dcap-qvl is baked into the runtime image; host bind is optional fallback
- /root/.cargo/bin/dcap-qvl:/usr/local/bin/dcap-qvl:ro
# dcap-qvl is baked into the runtime image. If you need a host override,
# add it via an extra `-f` override file rather than here — a missing
# source path would be auto-created as a directory and shadow the binary.
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/challenges/agent-challenge/docker-compose.staging.yml` around lines
31 - 32, Remove the unconditional dcap-qvl host bind from the base staging
Compose configuration so the baked /usr/local/bin/dcap-qvl binary remains
available on hosts without the source path. If host overrides are needed, place
this read-only bind in a separate opt-in staging override file such as
docker-compose.staging.dcap.yml.

env_file:
- ./scripts/staging/config/challenge.env
environment:
CHALLENGE_DATABASE_URL: sqlite+aiosqlite:////data/agent-challenge.sqlite3
CHALLENGE_DATA_DIR: /data
CHALLENGE_ARTIFACT_ROOT: /data/agents
CHALLENGE_SHARED_TOKEN_FILE: /run/secrets/base/challenge_token
CHALLENGE_REVIEW_EVIDENCE_ENCRYPTION_KEY_FILE: /run/secrets/base/review_evidence_encryption_key
CHALLENGE_COMBINED_WORKER: "true"
CHALLENGE_DOCKER_ENABLED: "false"
CHALLENGE_RAW_WEIGHT_PUSH_ENABLED: "false"
CHALLENGE_PHALA_ATTESTATION_ENABLED: "true"
CHALLENGE_ATTESTED_REVIEW_ENABLED: "true"
# Prod path: terminal-bench + frozen digest (default package backend is swe_forge).
CHALLENGE_BENCHMARK_BACKEND: terminal_bench
CHALLENGE_TERMINAL_BENCH_EXECUTION_BACKEND: own_runner
CHALLENGE_OWN_RUNNER_DIGEST_MANIFEST: /app/golden/dataset-digest.json
# Eval result signer — substrate dev URI (local only; never production wallet)
CHALLENGE_EVAL_RESULT_SIGNER_URI: "//Alice"
CHALLENGE_LOG_LEVEL: INFO
# Shortest viable eval for spend control (still real tasks)
CHALLENGE_EVALUATION_TASK_COUNT: "1"
CHALLENGE_EVAL_K: "1"
healthcheck:
test:
[
"CMD",
"python",
"-c",
"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3)",
]
interval: 5s
timeout: 5s
retries: 30
start_period: 15s

volumes:
ac_staging_data:
name: ac-staging-data
2 changes: 2 additions & 0 deletions packages/challenges/agent-challenge/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ inside the package.
| Interactive API | `https://chain.joinbase.ai/challenges/agent-challenge/docs` |
| Package product pin | [`../README.md`](../README.md) |
| Self-deploy CLI accuracy fixtures | [`miner/self-deploy.md`](miner/self-deploy.md), [`validator/self-deploy.md`](validator/self-deploy.md) |
| Local staging loop | [`staging.md`](staging.md) |
| Prod compose pin upgrade | [`prod-compose-upgrade.md`](prod-compose-upgrade.md) |

**API truth is OpenAPI** (and the in-process challenge app `/openapi.json`).
Audience essays (lifecycle dumps, route catalogs, architecture novels) were
Expand Down
Loading
Loading