diff --git a/docs/scoring.md b/docs/scoring.md index ec4f8ec..b27d785 100644 --- a/docs/scoring.md +++ b/docs/scoring.md @@ -15,9 +15,11 @@ then by recomputed prequential **bits-per-byte**. The challenge owns every numbe > wins remain **provisional only**. Challenge-owned **`prism_train_series.v1`** is visibility + residual > densify for sample-eff/stability only — **never sole primary** over held-out/bpb. When an Official > grade pin **requires** the series and it is missing/corrupt, Official grade **fail-closes** (not -> silent PASS); miner dashboards remain non-authoritative. Provider trust + **IMAGE_PIN** govern -> worker integrity; **REAL-PROVIDER TEE** is **retired** for Prism product (historical tables may -> still say BLOCKED; never a production scoring gate). +> silent PASS); miner dashboards remain non-authoritative. Worker-plane integrity uses +> **constation** (`constation_ok` over the six-mechanism bundle). **REAL-PROVIDER TEE** is +> **retired** for Prism product (historical tables may still say BLOCKED; never a production +> scoring gate). Under P1, a missing or failed constation bundle means **no score row at all**, +> so attestation is **decisive for score presence** (not a soft label orthogonal to ranking). ```mermaid flowchart LR Loss[Single-pass online loss stream] --> Bpb[Prequential bits-per-byte] @@ -155,10 +157,28 @@ Every number above is recomputed by the challenge from the challenge-authored raw-loss term and the v1-NAS architecture/training ownership pools are retired from the score. Miner self-reports remain non-authoritative on both the emission path and Official Comparison mode -(including scorecard v1.1, Complete View, and train series). Provider-trust / IMAGE_PIN / LAB-GPU -labels are orthogonal to ranking; **REAL-PROVIDER TEE** is a retired product goal (historical -non-claims only): see [Official Comparison](official-comparison.md) §17 train series telemetry, -scorecard honesty, and [Security](security.md). +(including scorecard v1.1, Complete View, and train series). + +## Constation and score presence (P1 fail-closed) + +Worker-plane Lium runs are admitted to the emission path only when Prism accepts a full +constation bundle and `constation_ok` is true. That predicate is the sole route to +`effective_tier == 1` (ceiling **1**; no TEE). Self-reported image digests and historical +"IMAGE_PIN match" labels do **not** elevate tier alone and do **not** create a score row +when the bundle is missing or fails. + +| Outcome | Score row? | Ranking impact | +| --- | --- | --- | +| Valid bundle, `constation_ok` | Yes (then held-out/bpb ranking applies) | Eligible for emission crown | +| Missing bundle | **No** (`miner_fault:missing_constation_bundle`) | Not ranked | +| Failed mechanism(s) | **No** (`miner_fault:`) | Not ranked | +| Infra outage after retries | **No** (`infra_fault:*`) unless audited break-glass | Break-glass admits at tier 0 only | + +So constation is **not orthogonal to ranking**: without a score row there is nothing to rank. +Among admitted rows, ranking remains held-out primary / bpb secondary as above. Trust model +detail and the six mechanisms with honest limitations: [Security](security.md) and +[prism-recipe security](../../prism-recipe/docs/security.md). Official Comparison notes: +[Official Comparison](official-comparison.md) §17 train series telemetry and scorecard honesty. ## Reference Studies diff --git a/docs/security.md b/docs/security.md index e4a3478..a56c837 100644 --- a/docs/security.md +++ b/docs/security.md @@ -65,23 +65,63 @@ or legacy reduced bodies fail closed with a 422 before scoring or persistence. P plausibility gates run before finalization; duplicate deliveries are idempotent and conflicts refuse mutation of sealed scores. -## Provider Trust And IMAGE_PIN - -Prism does **not** ship a TEE-attestation verifier and does **not** require TEE evidence to finalize -production scores. Integrity for miner-funded GPU work rests on: - -- **PROVIDER_TRUST** — operators trust **Lium/Targon** as compute providers (no Prism crypto TEE path). -- **IMAGE_PIN** — `worker_plane.pinned_image_digest` match grants audit effective tier **1** (maximum); - pin mismatch yields an honest non-elevated downgrade (not silent ignore). -- **DEPLOY SMOKE** — paid provider lifecycle proofs are reachability/infra only; always terminate pods. -- **LAB-GPU** — remote CUDA lab scores under Official Comparison; scientific only. -- Ordinary ExecutionProof envelope + worker signature checks on the worker-plane path. - -**REAL-PROVIDER TEE** (cryptographic provider attestation PASS as a Prism product goal) is -**retired**. Historical lab tables and reports may still show `real_provider_tee=BLOCKED` as honesty -history; do not implement or document a TEE production scoring gate as the live path. Opaque -non-empty `tdx_quote_b64` / `gpu_eat_jwt` fields never imply tier 2 by presence alone (max effective -tier is 1 via image pin). +## Worker-plane integrity: constation (tamper-evidence only) + +### Threat model (binding) + +- The miner **rents and controls** the Lium pod, including **root**. +- Integrity checks are **tamper-evidence**, not tamper-prevention. A determined root miner can + defeat all six mechanisms below. +- Prism does **not** ship a TEE verifier and does **not** require TEE evidence to finalize scores. +- `effective_tier` **ceiling is 1**. Claimed tier ≥ 2 always collapses to 0. Opaque + `tdx_quote_b64` / `gpu_eat_jwt` fields never imply tier 2 by presence alone. +- Lium TDX (or similar host quotes), if present in the wild, attest **executor infrastructure**, + not the renter workload inside the pod. +- The Lium API image/template digest is **declared configuration**, not a runtime measurement of + the container rootfs. + +### Sole elevation and score gate + +| Predicate | Role | +| --- | --- | +| `constation_ok(bundle)` | **Only** path that can grant `effective_tier == 1` | +| Self-reported `PRISM_IMAGE_DIGEST` / pin string match | Telemetry only; **never** elevates alone | +| Missing or failed constation bundle | **No score row written** (P1 fail-closed) | +| Operator break-glass | May admit **infra_fault** runs only; never `miner_fault`; score at tier 0 | + +Historical label **IMAGE_PIN** meant "digest pin matched." Under P1 that label is **not** a soft +orthogonal badge: without a valid constation bundle there is no score to rank. See +[scoring.md](scoring.md). + +`attestation_mode` on proofs is `miner_rent_image_pin_evidence_v1` (evidence under miner-rent pods). +Forbidden modes include any TEE-implying string (`tee`, `tdx`, `sev`, …). + +### Six mechanisms (each with honest limitation) + +1. **Nonce-bound attestation** — BASE per-run nonce; sidecar signs bound fields; defeats replay. + *Limit:* live root with the build secret can still answer fresh challenges. +2. **Same-account corroboration** — Lium API declared digest must agree with sidecar report. + *Limit:* **never independent**; miner supplies the API key and owns the pod. Agreement alone + never elevates if another check fails. +3. **Continuous constation** — start / end / randomized polls; gap or drift fails the bundle. + *Limit:* shrinks swap windows; does not seal the kernel against a twin process. +4. **BASE-built digest allowlist** — only BASE-produced digests for a registered commit score. + *Limit:* proves build provenance, not that the pod still runs that digest after deploy. +5. **In-image self-measurement** — sidecar hashes sealed harness / rules / data-window at runtime. + *Limit:* software on a miner-controlled host; root can patch the measurer. +6. **Build-time per-build secret** — unique BASE-injected secret required in signed responses. + *Limit:* root on the pod can extract `/run/prism/attestation_hmac_key`. + +Full recipe-side narrative: [prism-recipe security](../../prism-recipe/docs/security.md). + +### Provider trust (operational, not crypto) + +Operators still choose which GPU providers to enable (e.g. Lium). That is operational trust in +billing and reachability, **not** a cryptographic claim that the renter workload is honest. +Deploy smoke checks are reachability/infra only; always terminate pods. + +**REAL-PROVIDER TEE** as a Prism product goal is **retired**. Historical lab tables may still show +`real_provider_tee=BLOCKED` as honesty history; do not implement a TEE production scoring gate. ## Locked Data, No Network @@ -106,6 +146,10 @@ Host-side static instantiation and held-out scoring run in bounded child process containers; evaluation is the long-lived challenge runtime (or external worker-plane GPUs on trusted providers when enabled). +On the **miner-rent Lium** path, isolation is weaker by design: the miner is root on their pod. +Constation is the evidence layer for that path, not a substitute for container isolation on BASE-owned +evaluators. + ## ZIP Hardening ZIP extraction rejects symlinks, path traversal, unsafe paths, unsupported file types, and excessive @@ -132,5 +176,7 @@ their own wallets. The challenge and master never write weights on-chain. - Keep public submissions disabled when PRISM is deployed only behind BASE. - Keep the eval container on `network=none` and the rootfs read-only except `artifacts_dir`. - Do **not** configure LLM gateway URL/token fields; those surfaces are gone and residual knobs fail closed. -- Prefer `worker_plane.pinned_image_digest` for IMAGE_PIN tier-1; do not enable any removed TEE production scoring path. +- Require a full constation bundle for Lium worker-plane scores; do not treat pin string match as tier 1. +- Do not enable any removed TEE production scoring path. - Monitor rejected, failed, and completed submissions separately (legacy held is not a live path). +- Attribute rejects with `miner_fault:*` vs `infra_fault:*`; break-glass only for infra with operator audit. diff --git a/pyproject.toml b/pyproject.toml index 7a6fd37..2f412ed 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,7 +7,7 @@ dependencies = [ "aiosqlite>=0.20", "fastapi>=0.115", "httpx>=0.27", - "base @ https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl#sha256=3a61c2d3a343ed6de55e80215486e3de0c9639276443d08f2ed316bc807f2ff0", + "base @ https://github.com/BaseIntelligence/base/releases/download/v3.2.0/base-3.2.0-py3-none-any.whl#sha256=0fdd79b8eacda77132d01173df2057730350f3e82fb5a8a4478e7a53e61852c3", "pydantic>=2.8", "pydantic-settings>=2.4", "PyYAML>=6.0", diff --git a/src/prism_challenge/app.py b/src/prism_challenge/app.py index c7407b0..5f866fe 100644 --- a/src/prism_challenge/app.py +++ b/src/prism_challenge/app.py @@ -39,6 +39,113 @@ from .weights import get_weights +def _constation_ingest_kwargs( + app_settings: PrismSettings, + result_payload: object, + *, + app: FastAPI | None = None, +) -> dict: + """Resolve constation kwargs for ingest: prod wire OR insecure test seam. + + Production: deserialize ``result.constation_bundle`` and attach checkers. + Preference: + 1) BASE HTTP when ``constation_base_url`` + token (durable master SoT) + 2) in-process app.state services (same issuer as local challenge routes) + Missing bundle returns {} so ingest fail-closes (P1). Never auto-injects + synthetic bundles in prod. + """ + if getattr(app_settings, "allow_insecure_signatures", False): + return _test_constation_kwargs(app_settings, result_payload) + + if not isinstance(result_payload, dict): + return {} + raw_bundle = result_payload.get("constation_bundle") + if raw_bundle is None: + return {} + + from .constation import constation_bundle_from_dict + + try: + bundle = constation_bundle_from_dict(raw_bundle) + except (TypeError, ValueError): + # Malformed embedded bundle — leave to gate as missing/invalid. + return {} + + base_url = getattr(app_settings, "constation_base_url", None) + token = getattr(app_settings, "constation_internal_token", None) or "" + if base_url and token: + from .constation_checkers import BaseHttpConstationClient + + client = BaseHttpConstationClient(base_url=str(base_url), token=str(token)) + return { + "constation_bundle": bundle, + "check_allowlist": client.check_allowlist, + "check_nonce": client.check_nonce, + "verify_constation_signature": client.verify_signature, + } + + if app is not None: + from .attestation_routes import make_inprocess_checkers + + try: + checkers = make_inprocess_checkers(app) + return { + "constation_bundle": bundle, + "check_allowlist": checkers["check_allowlist"], + "check_nonce": checkers["check_nonce"], + "verify_constation_signature": checkers["verify_constation_signature"], + } + except Exception: + pass + + # Bundle present but checkers not configured: still pass bundle so + # constation_ok fails without elevating (checkers required → fail closed). + return {"constation_bundle": bundle} + + +def _test_constation_kwargs(app_settings: PrismSettings, result_payload: object) -> dict: + """Supply a valid constation bundle only under allow_insecure_signatures (unit tests). + + Production (allow_insecure_signatures=False) never auto-injects — missing bundle + fail-closes with no score (P1 / todo 22). Result payloads may still carry an + explicit constation block later; this helper is the test seam only. + """ + if not getattr(app_settings, "allow_insecure_signatures", False): + return {} + # If the payload already embeds constation markers, do not override. + if isinstance(result_payload, dict) and result_payload.get("constation_bundle") is not None: + return {} + from .constation import CheckOutcome, ConstationBundle + + def _ok(**_k: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + man = {"route-test-harness.py": "a" * 64} + digest = "sha256:" + ("11" * 32) + bundle = ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="route-wu", + miner_hotkey="route-hk", + pod_id="route-pod", + nonce="route-nonce", + signed_attestation={"route": True}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + return { + "constation_bundle": bundle, + "check_allowlist": _ok, + "check_nonce": _ok, + "verify_constation_signature": lambda _s: _ok(), + } + + def create_app( app_settings: PrismSettings | None = None, *, @@ -147,6 +254,15 @@ async def _run_raw_weight_push(app: FastAPI) -> None: app.state.checkpoint_publisher = publisher app.state.checkpoint_intake = checkpoint_intake + # Attestation constation hosts (public challenge via routes; internal check/*). + from .attestation_routes import ( + build_attestation_internal_router, + ensure_default_constation_services, + ) + + ensure_default_constation_services(app) + app.include_router(build_attestation_internal_router()) + @app.post("/internal/v1/worker/process-next", dependencies=[Depends(authenticate_internal)]) async def process_next() -> dict[str, str | None]: return {"submission_id": await worker.process_next()} @@ -320,6 +436,7 @@ async def work_unit_result(request: Request) -> dict[str, object]: result=result_payload, pinned_image_digest=app_settings.worker_plane.pinned_image_digest, audit_sampler=sampler, + **_constation_ingest_kwargs(app_settings, result_payload, app=request.app), ) except ResultIngestionError as exc: # A transient finalization failure is retryable -> 503 so the forwarder retries; the @@ -343,6 +460,15 @@ async def work_unit_result(request: Request) -> dict[str, object]: status.HTTP_409_CONFLICT, {"code": outcome.reason, "detail": "conflicting result for finalized unit"}, ) + if outcome.status == "rejected": + # P1 fail-closed: no score written; surface the miner/infra fault code. + raise HTTPException( + status.HTTP_422_UNPROCESSABLE_ENTITY, + { + "code": outcome.reason or "constation_rejected", + "detail": "constation gate refused score", + }, + ) return outcome.to_response() @app.post( diff --git a/src/prism_challenge/attestation_routes.py b/src/prism_challenge/attestation_routes.py new file mode 100644 index 0000000..0060d8c --- /dev/null +++ b/src/prism_challenge/attestation_routes.py @@ -0,0 +1,381 @@ +"""Prism-hosted attestation routes (public via BASE challenge proxy). + +Published (proxy strip ``/challenges/prism``): + GET /challenges/prism/v1/attestation/challenge + POST /challenges/prism/v1/attestation/answer + +Internal (prism process, shared-token): + POST /internal/v1/constation/register_digest + POST /internal/v1/constation/check_allowlist + POST /internal/v1/constation/check_nonce + POST /internal/v1/constation/verify_attestation + +Durable allowlist/nonce SoT may live on BASE master; when +``constation_base_url`` is set, challenge issuance and checkers call master +internal HTTP. Otherwise in-process services on ``app.state`` (tests / single-node). + +Challenge routes belong on the challenge app — not on base master. +""" + +from __future__ import annotations + +import hmac +from datetime import timedelta +from typing import Any + +import httpx +from base.challenge_sdk.roles import public_route +from base.compute.attestation_nonce import ( + AttestationNonceService, + NonceBinding, + NonceConsumeHit, +) +from base.compute.digest_allowlist import ( + AllowlistHit, + DigestAllowlist, + DigestRecord, + ImageVariant, +) +from fastapi import APIRouter, Depends, HTTPException, Query, Request, status +from pydantic import BaseModel, Field + +from .auth import authenticate_internal +from .constation import CheckOutcome + +try: + from base.attestation.payload import ( + AttestationPayload, + AttestationVerifyReason, + SignedAttestation, + verify_attestation_payload, + ) +except ImportError: # pragma: no cover - optional in minimal installs + AttestationPayload = Any # type: ignore[misc, assignment] + AttestationVerifyReason = None # type: ignore[misc, assignment] + SignedAttestation = Any # type: ignore[misc, assignment] + verify_attestation_payload = None # type: ignore[assignment] + + +class RegisterDigestBody(BaseModel): + commit_sha: str + tree_sha: str + variant: str + digest: str + + +class CheckAllowlistBody(BaseModel): + digest: str + commit_sha: str + tree_sha: str + variant: str + + +class CheckNonceBody(BaseModel): + nonce: str + work_unit_id: str + miner_hotkey: str + pod_id: str + + +class VerifyAttestationBody(BaseModel): + signed: dict[str, Any] + key_hex: str | None = None + + +class AnswerBody(BaseModel): + model_config = {"extra": "allow"} + + nonce: str = Field(min_length=1) + phase: str | None = None + + +def _bearer_ok(authorization: str | None, expected: str | None) -> bool: + if not expected: + return True + if not authorization or not authorization.lower().startswith("bearer "): + return False + presented = authorization.split(" ", 1)[1].strip() + return bool(presented) and hmac.compare_digest(presented, expected) + + +def _signed_from_wire(raw: dict[str, Any]) -> Any: + if verify_attestation_payload is None: + raise RuntimeError("attestation payload helpers unavailable") + if "payload" in raw and isinstance(raw["payload"], dict): + pl_raw = raw["payload"] + sig = raw.get("signature") + alg = raw.get("algorithm", "hmac-sha256") + schema = raw.get("schema_version", "prism_attestation_payload.v1") + else: + pl_raw = raw + sig = raw.get("signature") + alg = raw.get("algorithm", "hmac-sha256") + schema = raw.get("schema_version", "prism_attestation_payload.v1") + if not isinstance(sig, str): + raise ValueError("signature must be a hex string") + payload = AttestationPayload( + nonce=str(pl_raw["nonce"]), + digest=str(pl_raw["digest"]), + pod_id=str(pl_raw["pod_id"]), + variant=pl_raw["variant"], + sealed_manifest_hashes=dict(pl_raw["sealed_manifest_hashes"]), + build_secret_response=str(pl_raw["build_secret_response"]), + ) + return SignedAttestation( + payload=payload, + signature=sig, + algorithm=str(alg), + schema_version=str(schema), + ) + + +def ensure_default_constation_services(app: Any, *, ttl: timedelta | None = None) -> None: + """Attach in-process allowlist + nonce when missing (tests / single-node).""" + state = app.state + if getattr(state, "digest_allowlist", None) is None: + state.digest_allowlist = DigestAllowlist() + if getattr(state, "attestation_nonce_service", None) is None: + state.attestation_nonce_service = AttestationNonceService(ttl=ttl or timedelta(hours=2)) + if getattr(state, "attestation_answers", None) is None: + state.attestation_answers = [] + + +def make_inprocess_checkers(app: Any) -> dict[str, Any]: + """Sync checkers bound to app.state in-process services.""" + ensure_default_constation_services(app) + allowlist: DigestAllowlist = app.state.digest_allowlist + nonce_svc: AttestationNonceService = app.state.attestation_nonce_service + verify_key: bytes | None = getattr(app.state, "attestation_verify_key", None) + + def check_allowlist( + *, + digest: str, + commit_sha: str, + tree_sha: str, + variant: str, + ) -> CheckOutcome: + result = allowlist.lookup( + digest=digest, + commit_sha=commit_sha, + tree_sha=tree_sha, + variant=variant, + ) + if isinstance(result, AllowlistHit): + return CheckOutcome(ok=True, reason="ok") + return CheckOutcome(ok=False, reason=result.reason.value) + + def check_nonce( + *, + nonce: str, + work_unit_id: str, + miner_hotkey: str, + pod_id: str, + ) -> CheckOutcome: + result = nonce_svc.consume( + nonce, + NonceBinding( + work_unit_id=work_unit_id, + miner_hotkey=miner_hotkey, + pod_id=pod_id, + ), + ) + if isinstance(result, NonceConsumeHit): + return CheckOutcome(ok=True, reason="ok") + return CheckOutcome(ok=False, reason=result.reason.value) + + def verify_signature(signed: object) -> CheckOutcome: + if verify_attestation_payload is None: + return CheckOutcome(ok=False, reason="attestation_helpers_missing") + if verify_key is None: + # Soft path for harnesses that don't sign: accept dict markers. + if isinstance(signed, dict) and signed.get("sig"): + return CheckOutcome(ok=True, reason="ok") + return CheckOutcome(ok=False, reason="empty_key") + try: + if not isinstance(signed, dict): + return CheckOutcome(ok=False, reason="invalid_signed_shape") + wire = _signed_from_wire(signed) + outcome = verify_attestation_payload(wire, verify_key=verify_key) + return CheckOutcome(ok=outcome.ok, reason=outcome.reason.value) + except Exception as exc: # noqa: BLE001 + return CheckOutcome(ok=False, reason=f"signature_error:{exc}") + + return { + "check_allowlist": check_allowlist, + "check_nonce": check_nonce, + "verify_constation_signature": verify_signature, + } + + +def build_attestation_public_router() -> APIRouter: + """Public attestation challenge/answer under the prism ``/v1`` prefix.""" + router = APIRouter(tags=["attestation"]) + + @public_route(tags=["attestation"]) + @router.get("/attestation/challenge") + async def attestation_challenge( + request: Request, + phase: str = Query(default="interval"), + work_unit_id: str | None = Query(default=None), + miner_hotkey: str | None = Query(default=None), + pod_id: str | None = Query(default=None), + ) -> dict[str, Any]: + settings = getattr(request.app.state, "settings", None) + base_url = getattr(settings, "constation_base_url", None) if settings else None + token = getattr(settings, "constation_internal_token", None) or "" if settings else "" + phase_key = phase.strip().lower() + if phase_key in {"random", "mid"}: + phase_key = "interval" + if phase_key not in {"start", "interval", "end"}: + raise HTTPException( + status.HTTP_422_UNPROCESSABLE_ENTITY, + detail=f"unknown challenge phase: {phase!r}", + ) + + default_binding = getattr(request.app.state, "default_nonce_binding", None) + if default_binding is not None: + binding = default_binding + else: + if not (work_unit_id and miner_hotkey and pod_id): + raise HTTPException( + status.HTTP_422_UNPROCESSABLE_ENTITY, + detail=( + "work_unit_id, miner_hotkey, and pod_id query params " + "required when no default binding configured" + ), + ) + binding = NonceBinding( + work_unit_id=work_unit_id, + miner_hotkey=miner_hotkey, + pod_id=pod_id, + ) + + # Durable SoT on BASE master: issue via internal HTTP. + if base_url and token: + url = f"{str(base_url).rstrip('/')}/internal/v1/constation/issue_nonce" + payload = { + "work_unit_id": binding.work_unit_id, + "miner_hotkey": binding.miner_hotkey, + "pod_id": binding.pod_id, + "phase": phase_key, + } + try: + async with httpx.AsyncClient(timeout=10.0) as client: + response = await client.post( + url, + json=payload, + headers={ + "Authorization": f"Bearer {token}", + "Accept": "application/json", + }, + ) + response.raise_for_status() + body = response.json() + except Exception as exc: # noqa: BLE001 + raise HTTPException( + status.HTTP_503_SERVICE_UNAVAILABLE, + detail=f"constation issue_nonce failed: {exc}", + ) from exc + if not isinstance(body, dict) or "nonce" not in body: + raise HTTPException( + status.HTTP_502_BAD_GATEWAY, + detail="constation issue_nonce malformed response", + ) + return { + "nonce": body["nonce"], + "phase": body.get("phase", phase_key), + "challenge_id": body.get("challenge_id", body["nonce"]), + "work_unit_id": body.get("work_unit_id", binding.work_unit_id), + "expires_at": body.get("expires_at"), + } + + ensure_default_constation_services(request.app) + nonce_svc: AttestationNonceService = request.app.state.attestation_nonce_service + record = nonce_svc.issue(binding) + return { + "nonce": record.nonce, + "phase": phase_key, + "challenge_id": record.nonce, + "work_unit_id": binding.work_unit_id, + "expires_at": record.expires_at.isoformat(), + } + + @public_route(tags=["attestation"]) + @router.post("/attestation/answer") + async def attestation_answer(request: Request, body: AnswerBody) -> dict[str, str]: + ensure_default_constation_services(request.app) + answers: list[dict[str, Any]] = request.app.state.attestation_answers + answers.append(body.model_dump()) + return {"status": "accepted"} + + return router + + +def build_attestation_internal_router() -> APIRouter: + """Internal check/register surfaces on the prism process (shared token).""" + router = APIRouter(tags=["constation-internal"]) + + @router.post( + "/internal/v1/constation/register_digest", + dependencies=[Depends(authenticate_internal)], + ) + async def register_digest(request: Request, body: RegisterDigestBody) -> dict[str, str]: + ensure_default_constation_services(request.app) + allowlist: DigestAllowlist = request.app.state.digest_allowlist + record = DigestRecord( + commit_sha=body.commit_sha, + tree_sha=body.tree_sha, + variant=ImageVariant(body.variant.strip().lower()), + digest=body.digest, + ) + allowlist.register(record) + return {"status": "registered", "digest": record.digest} + + @router.post( + "/internal/v1/constation/check_allowlist", + dependencies=[Depends(authenticate_internal)], + ) + async def check_allowlist(request: Request, body: CheckAllowlistBody) -> dict[str, Any]: + checkers = make_inprocess_checkers(request.app) + outcome = checkers["check_allowlist"]( + digest=body.digest, + commit_sha=body.commit_sha, + tree_sha=body.tree_sha, + variant=body.variant, + ) + return {"ok": outcome.ok, "reason": outcome.reason} + + @router.post( + "/internal/v1/constation/check_nonce", + dependencies=[Depends(authenticate_internal)], + ) + async def check_nonce(request: Request, body: CheckNonceBody) -> dict[str, Any]: + checkers = make_inprocess_checkers(request.app) + outcome = checkers["check_nonce"]( + nonce=body.nonce, + work_unit_id=body.work_unit_id, + miner_hotkey=body.miner_hotkey, + pod_id=body.pod_id, + ) + return {"ok": outcome.ok, "reason": outcome.reason} + + @router.post( + "/internal/v1/constation/verify_attestation", + dependencies=[Depends(authenticate_internal)], + ) + async def verify_attestation(request: Request, body: VerifyAttestationBody) -> dict[str, Any]: + if body.key_hex: + request.app.state.attestation_verify_key = bytes.fromhex(body.key_hex) + checkers = make_inprocess_checkers(request.app) + outcome = checkers["verify_constation_signature"](body.signed) + return {"ok": outcome.ok, "reason": outcome.reason} + + return router + + +__all__ = [ + "build_attestation_internal_router", + "build_attestation_public_router", + "ensure_default_constation_services", + "make_inprocess_checkers", +] diff --git a/src/prism_challenge/audit.py b/src/prism_challenge/audit.py index 288b33d..3131974 100644 --- a/src/prism_challenge/audit.py +++ b/src/prism_challenge/audit.py @@ -6,8 +6,8 @@ * claimed tier >= 2 -> effective 0 always (Prism no longer elevates via TEE; claim/attestation fields may remain on the wire for compatibility but never authorize tier 2); -* claimed tier 1 -> effective 1 iff the proof's ``image_digest`` equals the configured pinned - evaluator/worker digest AND provider pod binding is present, else effective 0 (IMAGE_PIN); +* claimed tier 1 -> effective 1 iff ``constation_ok`` is True (M14 sole elevation predicate), + else effective 0. Self-reported image digests and pin match alone never elevate; * claimed tier 0 (or any unknown tier) -> effective 0. Max effective tier is **1**. :class:`AuditSampler` then samples finalized results at the per-tier @@ -64,39 +64,58 @@ def effective_tier( proof: ExecutionProof, *, pinned_image_digest: str | None = None, + constation_ok_result: bool | object | None = None, ) -> int: - """Return the VERIFIED tier for ``proof`` (never higher than image-pin verifiable backing). + """Return the VERIFIED tier for ``proof`` (never higher than constation-backed tier 1). - Claimed tier never controls elevation. Max effective tier is 1 (IMAGE_PIN match + pod binding). - Claimed tier 2 / attestation fields never elevate — Prism has no TEE verifier path. + **M14 — sole elevation path.** Tier 1 is granted only when ``constation_ok_result`` + is truthy (a ``True`` bool or a result object whose ``ok`` attribute is True). + Self-reported ``PRISM_IMAGE_DIGEST`` / ``pinned_image_digest`` match alone never + elevates (todo 20/21). Claimed tier never controls elevation. Max effective tier is 1. + Claimed tier >= 2 always collapses to 0 (Prism has no TEE verifier path). + + ``pinned_image_digest`` is retained for call-site compatibility and telemetry correlation + only; it is not an elevation predicate. """ + del pinned_image_digest # telemetry / compat only — never elevates claimed = int(getattr(proof, "tier", 0) or 0) - # Any claim of tier 2+ without a TEE product path downgrades to 0 (never silent tier-2, - # never fall through to tier-1 unless the independent pin policy below applies — and - # claimed>=2 intentionally does NOT fall through: pin match alone cannot re-elevate a - # hardware-attestation claim). + # Claimed tier 2+ never elevates and never falls through to tier 1. if claimed >= 2: return 0 if claimed == 1: - provider = proof.provider - matches_digest = bool(pinned_image_digest) and proof.image_digest == pinned_image_digest - has_pod = provider is not None and bool(provider.pod_id) - return 1 if matches_digest and has_pod else 0 + return 1 if _constation_truthy(constation_ok_result) else 0 return 0 +def _constation_truthy(constation_ok_result: bool | object | None) -> bool: + """Interpret bool or ConstationResult-like object as the elevation predicate.""" + if constation_ok_result is None or constation_ok_result is False: + return False + if constation_ok_result is True: + return True + ok_attr = getattr(constation_ok_result, "ok", None) + if ok_attr is not None: + return bool(ok_attr) + return bool(constation_ok_result) + + def is_tier_downgraded( proof: ExecutionProof, *, pinned_image_digest: str | None = None, + constation_ok_result: bool | object | None = None, ) -> bool: """Whether ``proof``'s claimed tier is higher than its verified effective tier.""" - return int(proof.tier) != effective_tier(proof, pinned_image_digest=pinned_image_digest) + return int(proof.tier) != effective_tier( + proof, + pinned_image_digest=pinned_image_digest, + constation_ok_result=constation_ok_result, + ) @dataclass(frozen=True) @@ -155,10 +174,15 @@ def decide( work_unit_id: str, proof: ExecutionProof, pinned_image_digest: str | None = None, + constation_ok_result: bool | object | None = None, ) -> AuditDecision: """Verify ``proof``'s tier and decide whether it is sampled at its EFFECTIVE rate.""" - tier = effective_tier(proof, pinned_image_digest=pinned_image_digest) + tier = effective_tier( + proof, + pinned_image_digest=pinned_image_digest, + constation_ok_result=constation_ok_result, + ) return AuditDecision( work_unit_id=work_unit_id, claimed_tier=int(proof.tier), diff --git a/src/prism_challenge/breakglass.py b/src/prism_challenge/breakglass.py new file mode 100644 index 0000000..0dfb7ce --- /dev/null +++ b/src/prism_challenge/breakglass.py @@ -0,0 +1,152 @@ +"""Audited operator break-glass for infra-fault attestation failures (todo 23). + +P1 fail-closed scoring erases runs without a valid constation bundle. When the +failure is classified as BASE/infra fault (constation service down, Lium 5xx, +network partition), an explicit operator override may admit the run. Overrides +are never automatic, always attributable to an operator identity, and always +written to an audit log. Miner-fault runs can never be admitted this way. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from datetime import UTC, datetime +from typing import Any, Final, Literal + +FaultClass = Literal["miner_fault", "infra_fault"] + +MINER_FAULT: Final[Literal["miner_fault"]] = "miner_fault" +INFRA_FAULT: Final[Literal["infra_fault"]] = "infra_fault" + + +@dataclass(frozen=True, slots=True) +class BreakGlassRequest: + """Operator request to admit an infra-fault run.""" + + operator_id: str + reason: str + work_unit_id: str + fault_code: str # e.g. infra_fault:constation_unavailable + + +@dataclass(frozen=True, slots=True) +class BreakGlassDecision: + """Outcome of evaluating a break-glass request.""" + + admitted: bool + reason: str + audit_entry: dict[str, Any] | None = None + + +@dataclass +class BreakGlassAuditLog: + """In-memory / injectable audit sink for break-glass decisions.""" + + entries: list[dict[str, Any]] = field(default_factory=list) + + def append(self, entry: dict[str, Any]) -> None: + self.entries.append(dict(entry)) + + def to_list(self) -> list[dict[str, Any]]: + return list(self.entries) + + +def fault_class_of(reason_code: str) -> FaultClass: + """Return miner_fault or infra_fault from a dotted/colon reason code.""" + code = (reason_code or "").strip().lower() + if code.startswith("infra_fault") or code.startswith(f"{INFRA_FAULT}:"): + return INFRA_FAULT + if code.startswith("miner_fault") or code.startswith(f"{MINER_FAULT}:"): + return MINER_FAULT + # bare codes + if code in { + "constation_unavailable", + "lium_5xx", + "network_partition", + "constation_retry_exhausted", + "lium_auth_revoked_infra", + }: + return INFRA_FAULT + return MINER_FAULT + + +def format_fault_reason(fault_class: FaultClass, code: str) -> str: + """Normalize to ``miner_fault:`` / ``infra_fault:``.""" + bare = code + for prefix in (f"{MINER_FAULT}:", f"{INFRA_FAULT}:"): + if bare.startswith(prefix): + bare = bare[len(prefix) :] + break + bare = bare.strip() or "unknown" + return f"{fault_class}:{bare}" + + +def evaluate_break_glass( + request: BreakGlassRequest, + *, + fault_reason: str, + audit_log: BreakGlassAuditLog | None = None, +) -> BreakGlassDecision: + """Admit only infra_fault runs; refuse miner_fault. Always audit.""" + operator = (request.operator_id or "").strip() + if not operator: + entry = _entry(request, fault_reason, admitted=False, detail="missing_operator_id") + if audit_log is not None: + audit_log.append(entry) + return BreakGlassDecision( + admitted=False, reason="breakglass_missing_operator", audit_entry=entry + ) + + cls = fault_class_of(fault_reason) + if cls != INFRA_FAULT: + entry = _entry( + request, + fault_reason, + admitted=False, + detail="miner_fault_override_refused", + ) + if audit_log is not None: + audit_log.append(entry) + return BreakGlassDecision( + admitted=False, + reason="breakglass_refused_miner_fault", + audit_entry=entry, + ) + + entry = _entry(request, fault_reason, admitted=True, detail="infra_fault_admitted") + if audit_log is not None: + audit_log.append(entry) + return BreakGlassDecision(admitted=True, reason="breakglass_admitted", audit_entry=entry) + + +def _entry( + request: BreakGlassRequest, + fault_reason: str, + *, + admitted: bool, + detail: str, +) -> dict[str, Any]: + return { + "event": "break_glass", + "admitted": admitted, + "detail": detail, + "operator_id": request.operator_id, + "operator_reason": request.reason, + "work_unit_id": request.work_unit_id, + "fault_reason": fault_reason, + "fault_class": fault_class_of(fault_reason), + "requested_fault_code": request.fault_code, + "ts": datetime.now(UTC).isoformat(), + } + + +__all__ = [ + "BreakGlassAuditLog", + "BreakGlassDecision", + "BreakGlassRequest", + "INFRA_FAULT", + "MINER_FAULT", + "evaluate_break_glass", + "fault_class_of", + "format_fault_reason", +] diff --git a/src/prism_challenge/config.py b/src/prism_challenge/config.py index fda2b82..14d2a4a 100644 --- a/src/prism_challenge/config.py +++ b/src/prism_challenge/config.py @@ -302,6 +302,9 @@ def _known_environment_names(cls) -> set[str]: validation_alias=AliasChoices("PRISM_SHARED_TOKEN_FILE", "CHALLENGE_SHARED_TOKEN_FILE"), ) allow_insecure_signatures: bool = False + # Production constation: BASE checkers host for allowlist/nonce/sig. + constation_base_url: str | None = None + constation_internal_token: str | None = Field(default=None, repr=False) signature_ttl_seconds: int = 300 epoch_seconds: int = 21_600 max_code_bytes: int = 7_500_000 diff --git a/src/prism_challenge/constation.py b/src/prism_challenge/constation.py new file mode 100644 index 0000000..a8c6de1 --- /dev/null +++ b/src/prism_challenge/constation.py @@ -0,0 +1,482 @@ +"""Sole elevation predicate for Prism image-constation (M14). + +``constation_ok(bundle)`` is the **only** function in Prism that may authorize +tier elevation from a constation bundle. No other module, path, or helper may +grant a tier from attestation, allowlist, nonce, signature, or Lium data alone +(M14). Callers that need an effective tier must route through this predicate +(wiring lands in a later todo — this module deliberately exposes **no** +``effective_tier`` / ``grant_tier`` API). + +Honesty constraints (binding): + +* A True result is **tamper-evidence**, not tamper-prevention. The miner rents + and controls the pod. +* Corroboration (Lium-declared digest vs sidecar digest) is a **negative-only** + signal: mismatch fails; agreement alone never elevates. +* Signature validity proves only that an entity holding the in-image secret + responded — never sufficient alone (B3). +* Dependencies are injected (allowlist / nonce / signature checkers) so this + module stays pure and testable without live Lium. + +Required conjunction (all must pass): + +1. Allowlist hit for ``(commit_sha, tree_sha, variant, digest)`` +2. Nonce valid (BASE-issued, unexpired, single-use, binding match) +3. Attestation signature valid +4. Sealed manifest hashes match (expected vs reported) +5. Corroboration not contradicting (negative only) +6. No constation gap beyond budget +""" + +from __future__ import annotations + +from collections.abc import Mapping +from dataclasses import dataclass +from enum import StrEnum +from typing import Final, Protocol + +# Allowlist miss reason strings — must match base.compute.digest_allowlist.AllowlistMissReason +_ALLOWLIST_UNKNOWN: Final[str] = "unknown_digest" +_ALLOWLIST_VARIANT: Final[str] = "variant_mismatch" +_ALLOWLIST_COMMIT: Final[str] = "commit_mismatch" +_ALLOWLIST_REVOKED: Final[str] = "revoked" + +# Nonce miss reason strings — must match base.compute.attestation_nonce.NonceConsumeReason +_NONCE_ALREADY: Final[str] = "already_consumed" +_NONCE_UNKNOWN: Final[str] = "unknown_nonce" +_NONCE_EXPIRED: Final[str] = "expired" +_NONCE_WORK_UNIT: Final[str] = "work_unit_mismatch" +_NONCE_HOTKEY: Final[str] = "miner_hotkey_mismatch" +_NONCE_POD: Final[str] = "pod_mismatch" + + +class ConstationFailReason(StrEnum): + """Machine-consumed outcome codes for ``constation_ok``. + + Callers must not string-match prose. Allowlist/nonce detail codes mirror + base miss enums so adapters can pass reasons through without remapping + tables beyond the thin maps below. + """ + + OK = "ok" + ALLOWLIST_UNKNOWN_DIGEST = "allowlist_unknown_digest" + ALLOWLIST_VARIANT_MISMATCH = "allowlist_variant_mismatch" + ALLOWLIST_COMMIT_MISMATCH = "allowlist_commit_mismatch" + ALLOWLIST_REVOKED = "allowlist_revoked" + ALLOWLIST_FAILED = "allowlist_failed" + NONCE_ALREADY_CONSUMED = "nonce_already_consumed" + NONCE_UNKNOWN = "nonce_unknown" + NONCE_EXPIRED = "nonce_expired" + NONCE_WORK_UNIT_MISMATCH = "nonce_work_unit_mismatch" + NONCE_MINER_HOTKEY_MISMATCH = "nonce_miner_hotkey_mismatch" + NONCE_POD_MISMATCH = "nonce_pod_mismatch" + NONCE_FAILED = "nonce_failed" + SIGNATURE_INVALID = "signature_invalid" + SEALED_MANIFEST_MISMATCH = "sealed_manifest_mismatch" + CORROBORATION_MISMATCH = "corroboration_mismatch" + CONSTATION_GAP = "constation_gap" + + +@dataclass(frozen=True, slots=True) +class CheckOutcome: + """Structured result from an injected checker (allowlist / nonce / signature). + + ``reason`` is a machine code. For allowlist/nonce, prefer the base miss + reason value (e.g. ``unknown_digest``); ``constation_ok`` maps it onto + :class:`ConstationFailReason`. + """ + + ok: bool + reason: str = "ok" + + +@dataclass(frozen=True, slots=True) +class ConstationBundle: + """Inputs required to evaluate the six-mechanism elevation conjunction. + + Field meanings (tamper-evidence only): + + * Identity: ``commit_sha``, ``tree_sha``, ``variant``, ``digest`` + * Nonce binding: ``work_unit_id``, ``miner_hotkey``, ``pod_id``, ``nonce`` + * ``signed_attestation`` — opaque object passed to ``verify_signature`` + * Sealed surface: ``expected_sealed_manifest_hashes`` (build-time) vs + ``reported_sealed_manifest_hashes`` (sidecar self-measure) + * ``lium_declared_digest`` — same-account corroboration channel (optional; + absence is not a contradiction; mismatch is fatal) + * Gap budget: ``constation_gap_budget_seconds`` vs + ``constation_observed_max_gap_seconds`` + """ + + commit_sha: str + tree_sha: str + variant: str + digest: str + work_unit_id: str + miner_hotkey: str + pod_id: str + nonce: str + signed_attestation: object + expected_sealed_manifest_hashes: Mapping[str, str] + reported_sealed_manifest_hashes: Mapping[str, str] + lium_declared_digest: str | None + constation_gap_budget_seconds: float + constation_observed_max_gap_seconds: float + + +@dataclass(frozen=True, slots=True) +class ConstationResult: + """Structured predicate outcome. ``ok`` never implies hardware trust.""" + + ok: bool + reason: ConstationFailReason + detail: str | None = None + + def __bool__(self) -> bool: + return self.ok + + +class AllowlistChecker(Protocol): + """Protocol matching ``DigestAllowlist.lookup`` miss-reason surface.""" + + def __call__( + self, + *, + digest: str, + commit_sha: str, + tree_sha: str, + variant: str, + ) -> CheckOutcome: ... + + +class NonceChecker(Protocol): + """Protocol matching single-use nonce consume surface.""" + + def __call__( + self, + *, + nonce: str, + work_unit_id: str, + miner_hotkey: str, + pod_id: str, + ) -> CheckOutcome: ... + + +class SignatureVerifier(Protocol): + """Protocol matching attestation payload verify surface.""" + + def __call__(self, signed: object) -> CheckOutcome: ... + + +_ALLOWLIST_MAP: Final[dict[str, ConstationFailReason]] = { + _ALLOWLIST_UNKNOWN: ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST, + _ALLOWLIST_VARIANT: ConstationFailReason.ALLOWLIST_VARIANT_MISMATCH, + _ALLOWLIST_COMMIT: ConstationFailReason.ALLOWLIST_COMMIT_MISMATCH, + _ALLOWLIST_REVOKED: ConstationFailReason.ALLOWLIST_REVOKED, +} + +_NONCE_MAP: Final[dict[str, ConstationFailReason]] = { + _NONCE_ALREADY: ConstationFailReason.NONCE_ALREADY_CONSUMED, + _NONCE_UNKNOWN: ConstationFailReason.NONCE_UNKNOWN, + _NONCE_EXPIRED: ConstationFailReason.NONCE_EXPIRED, + _NONCE_WORK_UNIT: ConstationFailReason.NONCE_WORK_UNIT_MISMATCH, + _NONCE_HOTKEY: ConstationFailReason.NONCE_MINER_HOTKEY_MISMATCH, + _NONCE_POD: ConstationFailReason.NONCE_POD_MISMATCH, +} + + +def constation_ok( + bundle: ConstationBundle, + *, + check_allowlist: AllowlistChecker, + check_nonce: NonceChecker, + verify_signature: SignatureVerifier, +) -> ConstationResult: + """Return whether ``bundle`` satisfies every elevation requirement. + + **M14 — sole elevation predicate.** No other Prism path may grant a tier. + This function returns a structured :class:`ConstationResult`; it does not + itself assign tiers. Downstream ``effective_tier`` wiring (separate todo) + must call this predicate and must not bypass it. + + Check order (first failure wins, distinct reason each): + + 1. allowlist hit + 2. nonce valid + 3. signature valid + 4. sealed manifest hashes match + 5. corroboration not contradicting (negative only) + 6. constation gap within budget + + All checkers are injected so unit tests never need live Lium, BASE DB, or + network I/O. + """ + allow = check_allowlist( + digest=bundle.digest, + commit_sha=bundle.commit_sha, + tree_sha=bundle.tree_sha, + variant=bundle.variant, + ) + if not allow.ok: + mapped = _ALLOWLIST_MAP.get(allow.reason, ConstationFailReason.ALLOWLIST_FAILED) + return ConstationResult(ok=False, reason=mapped, detail=allow.reason) + + nonce = check_nonce( + nonce=bundle.nonce, + work_unit_id=bundle.work_unit_id, + miner_hotkey=bundle.miner_hotkey, + pod_id=bundle.pod_id, + ) + if not nonce.ok: + mapped = _NONCE_MAP.get(nonce.reason, ConstationFailReason.NONCE_FAILED) + return ConstationResult(ok=False, reason=mapped, detail=nonce.reason) + + sig = verify_signature(bundle.signed_attestation) + if not sig.ok: + return ConstationResult( + ok=False, + reason=ConstationFailReason.SIGNATURE_INVALID, + detail=sig.reason, + ) + + if not _manifests_match( + bundle.expected_sealed_manifest_hashes, + bundle.reported_sealed_manifest_hashes, + ): + return ConstationResult( + ok=False, + reason=ConstationFailReason.SEALED_MANIFEST_MISMATCH, + ) + + # Corroboration is negative-only: mismatch fails; absence or agreement + # does not grant elevation by itself (other checks already required). + if bundle.lium_declared_digest is not None: + declared = bundle.lium_declared_digest.strip().lower() + sidecar = bundle.digest.strip().lower() + if declared != sidecar: + return ConstationResult( + ok=False, + reason=ConstationFailReason.CORROBORATION_MISMATCH, + detail=f"lium={declared} sidecar={sidecar}", + ) + + if bundle.constation_observed_max_gap_seconds > bundle.constation_gap_budget_seconds: + return ConstationResult( + ok=False, + reason=ConstationFailReason.CONSTATION_GAP, + detail=( + f"observed={bundle.constation_observed_max_gap_seconds}" + f" budget={bundle.constation_gap_budget_seconds}" + ), + ) + + return ConstationResult(ok=True, reason=ConstationFailReason.OK) + + +def _manifests_match( + expected: Mapping[str, str], + reported: Mapping[str, str], +) -> bool: + """Exact path→hash equality (order-insensitive). Empty expected is not a free pass.""" + if not expected: + return False + exp = {str(k): str(v).strip().lower() for k, v in expected.items()} + rep = {str(k): str(v).strip().lower() for k, v in reported.items()} + return exp == rep + + +def adapt_allowlist_lookup(lookup_result: object) -> CheckOutcome: + """Thin adapter: base ``AllowlistHit`` / ``AllowlistMiss`` → :class:`CheckOutcome`. + + Accepts any object with optional ``reason`` attribute (miss) or hit without + a failing reason. Does not import base so prism stays runnable against older + wheels; pass the live lookup result from ``DigestAllowlist.lookup``. + """ + reason_obj = getattr(lookup_result, "reason", None) + if reason_obj is None: + # Hit (or unknown success shape) + return CheckOutcome(ok=True, reason="ok") + reason = getattr(reason_obj, "value", None) or str(reason_obj) + return CheckOutcome(ok=False, reason=str(reason)) + + +def adapt_nonce_consume(consume_result: object) -> CheckOutcome: + """Thin adapter: base ``NonceConsumeHit`` / ``NonceConsumeMiss`` → CheckOutcome.""" + reason_obj = getattr(consume_result, "reason", None) + if reason_obj is None: + return CheckOutcome(ok=True, reason="ok") + reason = getattr(reason_obj, "value", None) or str(reason_obj) + return CheckOutcome(ok=False, reason=str(reason)) + + +def adapt_attestation_verify(verify_result: object) -> CheckOutcome: + """Thin adapter: base ``AttestationVerifyResult`` → CheckOutcome.""" + ok = bool(getattr(verify_result, "ok", False)) + reason_obj = getattr(verify_result, "reason", "ok") + reason = getattr(reason_obj, "value", None) or str(reason_obj) + return CheckOutcome(ok=ok, reason=str(reason)) + + +# --- Fault attribution (todo 22) --------------------------------------------------------------- + +#: ConstationFailReason values that are miner-attributable. +_MINER_FAIL_REASONS: Final[frozenset[ConstationFailReason]] = frozenset( + { + ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST, + ConstationFailReason.ALLOWLIST_VARIANT_MISMATCH, + ConstationFailReason.ALLOWLIST_COMMIT_MISMATCH, + ConstationFailReason.ALLOWLIST_REVOKED, + ConstationFailReason.ALLOWLIST_FAILED, + ConstationFailReason.NONCE_ALREADY_CONSUMED, + ConstationFailReason.NONCE_UNKNOWN, + ConstationFailReason.NONCE_EXPIRED, + ConstationFailReason.NONCE_WORK_UNIT_MISMATCH, + ConstationFailReason.NONCE_MINER_HOTKEY_MISMATCH, + ConstationFailReason.NONCE_POD_MISMATCH, + ConstationFailReason.NONCE_FAILED, + ConstationFailReason.SIGNATURE_INVALID, + ConstationFailReason.SEALED_MANIFEST_MISMATCH, + ConstationFailReason.CORROBORATION_MISMATCH, + ConstationFailReason.CONSTATION_GAP, # sidecar gap is miner-side + } +) + +#: Stable bare codes for infra faults (no ConstationFailReason enum value). +INFRA_FAULT_CONSTATION_UNAVAILABLE = "constation_unavailable" +INFRA_FAULT_LIUM_5XX = "lium_5xx" +INFRA_FAULT_NETWORK_PARTITION = "network_partition" +INFRA_FAULT_RETRY_EXHAUSTED = "constation_retry_exhausted" + +#: Map fail reasons → bare miner_fault codes used in ingestion reason strings. +_MINER_BARE_CODES: Final[dict[ConstationFailReason, str]] = { + ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST: "unknown_digest", + ConstationFailReason.ALLOWLIST_VARIANT_MISMATCH: "variant_mismatch", + ConstationFailReason.ALLOWLIST_COMMIT_MISMATCH: "commit_mismatch", + ConstationFailReason.ALLOWLIST_REVOKED: "revoked_digest", + ConstationFailReason.ALLOWLIST_FAILED: "allowlist_failed", + ConstationFailReason.NONCE_ALREADY_CONSUMED: "replayed_nonce", + ConstationFailReason.NONCE_UNKNOWN: "unknown_nonce", + ConstationFailReason.NONCE_EXPIRED: "expired_nonce", + ConstationFailReason.NONCE_WORK_UNIT_MISMATCH: "nonce_work_unit_mismatch", + ConstationFailReason.NONCE_MINER_HOTKEY_MISMATCH: "nonce_hotkey_mismatch", + ConstationFailReason.NONCE_POD_MISMATCH: "nonce_pod_mismatch", + ConstationFailReason.NONCE_FAILED: "nonce_failed", + ConstationFailReason.SIGNATURE_INVALID: "signature_invalid", + ConstationFailReason.SEALED_MANIFEST_MISMATCH: "manifest_mismatch", + ConstationFailReason.CORROBORATION_MISMATCH: "corroboration_mismatch", + ConstationFailReason.CONSTATION_GAP: "constation_gap", +} + + +def classify_constation_fault(result: ConstationResult) -> str: + """Return ``miner_fault:`` or ``infra_fault:`` for a failed result. + + A successful result returns ``ok``. Missing-bundle / service-down cases are + classified by callers via :func:`infra_fault_reason` / :func:`miner_fault_reason`. + """ + if result.ok: + return "ok" + bare = _MINER_BARE_CODES.get(result.reason, str(result.reason.value)) + if result.reason in _MINER_FAIL_REASONS: + return f"miner_fault:{bare}" + return f"infra_fault:{bare}" + + +def miner_fault_reason(code: str) -> str: + bare = code.removeprefix("miner_fault:").strip() or "unknown" + return f"miner_fault:{bare}" + + +def infra_fault_reason(code: str) -> str: + bare = code.removeprefix("infra_fault:").strip() or "unknown" + return f"infra_fault:{bare}" + + +def constation_bundle_to_dict(bundle: ConstationBundle) -> dict[str, object]: + """Serialize a :class:`ConstationBundle` for wire / result payload embedding.""" + return { + "commit_sha": bundle.commit_sha, + "tree_sha": bundle.tree_sha, + "variant": bundle.variant, + "digest": bundle.digest, + "work_unit_id": bundle.work_unit_id, + "miner_hotkey": bundle.miner_hotkey, + "pod_id": bundle.pod_id, + "nonce": bundle.nonce, + "signed_attestation": bundle.signed_attestation, + "expected_sealed_manifest_hashes": dict(bundle.expected_sealed_manifest_hashes), + "reported_sealed_manifest_hashes": dict(bundle.reported_sealed_manifest_hashes), + "lium_declared_digest": bundle.lium_declared_digest, + "constation_gap_budget_seconds": float(bundle.constation_gap_budget_seconds), + "constation_observed_max_gap_seconds": float(bundle.constation_observed_max_gap_seconds), + } + + +def constation_bundle_from_dict(raw: object) -> ConstationBundle: + """Parse a wire dict into :class:`ConstationBundle` (boundary validation).""" + if not isinstance(raw, dict): + raise ValueError("constation_bundle must be an object") + required = ( + "commit_sha", + "tree_sha", + "variant", + "digest", + "work_unit_id", + "miner_hotkey", + "pod_id", + "nonce", + "signed_attestation", + "expected_sealed_manifest_hashes", + "reported_sealed_manifest_hashes", + "constation_gap_budget_seconds", + "constation_observed_max_gap_seconds", + ) + missing = [k for k in required if k not in raw] + if missing: + raise ValueError(f"constation_bundle missing fields: {missing}") + exp = raw["expected_sealed_manifest_hashes"] + rep = raw["reported_sealed_manifest_hashes"] + if not isinstance(exp, Mapping) or not isinstance(rep, Mapping): + raise ValueError("sealed manifest hashes must be objects") + lium = raw.get("lium_declared_digest") + if lium is not None and not isinstance(lium, str): + raise ValueError("lium_declared_digest must be string or null") + return ConstationBundle( + commit_sha=str(raw["commit_sha"]), + tree_sha=str(raw["tree_sha"]), + variant=str(raw["variant"]), + digest=str(raw["digest"]), + work_unit_id=str(raw["work_unit_id"]), + miner_hotkey=str(raw["miner_hotkey"]), + pod_id=str(raw["pod_id"]), + nonce=str(raw["nonce"]), + signed_attestation=raw["signed_attestation"], + expected_sealed_manifest_hashes={str(k): str(v) for k, v in exp.items()}, + reported_sealed_manifest_hashes={str(k): str(v) for k, v in rep.items()}, + lium_declared_digest=lium, + constation_gap_budget_seconds=float(raw["constation_gap_budget_seconds"]), + constation_observed_max_gap_seconds=float(raw["constation_observed_max_gap_seconds"]), + ) + + +__all__ = [ + "AllowlistChecker", + "CheckOutcome", + "ConstationBundle", + "ConstationFailReason", + "ConstationResult", + "NonceChecker", + "SignatureVerifier", + "adapt_allowlist_lookup", + "adapt_attestation_verify", + "adapt_nonce_consume", + "constation_ok", + "classify_constation_fault", + "miner_fault_reason", + "infra_fault_reason", + "INFRA_FAULT_CONSTATION_UNAVAILABLE", + "INFRA_FAULT_LIUM_5XX", + "INFRA_FAULT_NETWORK_PARTITION", + "INFRA_FAULT_RETRY_EXHAUSTED", + "constation_bundle_to_dict", + "constation_bundle_from_dict", +] diff --git a/src/prism_challenge/constation_checkers.py b/src/prism_challenge/constation_checkers.py new file mode 100644 index 0000000..6cdaa79 --- /dev/null +++ b/src/prism_challenge/constation_checkers.py @@ -0,0 +1,89 @@ +"""BASE-backed HTTP checkers for prism production constation_ok.""" + +from __future__ import annotations + +from typing import Any + +import httpx + +from .constation import CheckOutcome + + +class BaseHttpConstationClient: + """Thin client for BASE internal constation checker endpoints.""" + + def __init__( + self, + *, + base_url: str, + token: str, + timeout_s: float = 10.0, + transport: httpx.BaseTransport | None = None, + ) -> None: + if not base_url.strip(): + raise ValueError("constation base_url must be non-empty") + self._base_url = base_url.rstrip("/") + self._token = token + self._timeout_s = timeout_s + self._transport = transport + + def _headers(self) -> dict[str, str]: + return { + "Authorization": f"Bearer {self._token}", + "Accept": "application/json", + "Content-Type": "application/json", + } + + def check_allowlist( + self, + *, + digest: str, + commit_sha: str, + tree_sha: str, + variant: str, + ) -> CheckOutcome: + body = { + "digest": digest, + "commit_sha": commit_sha, + "tree_sha": tree_sha, + "variant": variant, + } + return self._post_outcome("/internal/v1/constation/check_allowlist", body) + + def check_nonce( + self, + *, + nonce: str, + work_unit_id: str, + miner_hotkey: str, + pod_id: str, + ) -> CheckOutcome: + body = { + "nonce": nonce, + "work_unit_id": work_unit_id, + "miner_hotkey": miner_hotkey, + "pod_id": pod_id, + } + return self._post_outcome("/internal/v1/constation/check_nonce", body) + + def verify_signature(self, signed: object) -> CheckOutcome: + body = {"signed": signed if isinstance(signed, dict) else {"value": signed}} + return self._post_outcome("/internal/v1/constation/verify_attestation", body) + + def _post_outcome(self, path: str, body: dict[str, Any]) -> CheckOutcome: + url = f"{self._base_url}{path}" + try: + with httpx.Client(timeout=self._timeout_s, transport=self._transport) as client: + response = client.post(url, json=body, headers=self._headers()) + response.raise_for_status() + data = response.json() + except Exception as exc: # noqa: BLE001 - map to checker fail-closed + return CheckOutcome(ok=False, reason=f"checker_transport_error:{exc}") + if not isinstance(data, dict): + return CheckOutcome(ok=False, reason="checker_malformed_response") + ok = bool(data.get("ok")) + reason = data.get("reason", "ok" if ok else "checker_rejected") + return CheckOutcome(ok=ok, reason=str(reason)) + + +__all__ = ["BaseHttpConstationClient"] diff --git a/src/prism_challenge/ingestion.py b/src/prism_challenge/ingestion.py index 0e8f83c..04fb9bd 100644 --- a/src/prism_challenge/ingestion.py +++ b/src/prism_challenge/ingestion.py @@ -35,12 +35,35 @@ from .audit import AuditSampler, effective_tier from .auth import verify_hotkey_signature +from .breakglass import ( + BreakGlassAuditLog, + BreakGlassRequest, + evaluate_break_glass, + fault_class_of, +) +from .constation import ( + INFRA_FAULT_CONSTATION_UNAVAILABLE, + INFRA_FAULT_RETRY_EXHAUSTED, + AllowlistChecker, + ConstationBundle, + ConstationResult, + NonceChecker, + classify_constation_fault, + constation_ok, + infra_fault_reason, + miner_fault_reason, +) +from .constation import ( + SignatureVerifier as ConstationSignatureVerifier, +) from .plausibility import check_manifest_plausibility from .proof import ( + ATTESTATION_MODE_V1, EXECUTION_PROOF_VERSION, MANIFEST_PAYLOAD_KEY, PROOF_PAYLOAD_KEY, ExecutionProof, + attestation_mode_of, compute_manifest_sha256, verify_execution_proof, ) @@ -51,6 +74,9 @@ #: A verified 64-char lowercase-hex manifest digest (VAL-PRISM-018). _MANIFEST_SHA256_RE = re.compile(r"^[0-9a-f]{64}$") +#: Bounded retry before discarding an infra-fault run (todo 22). +DEFAULT_MAX_CONSTATION_ATTEMPTS = 3 + SignatureVerifier = Callable[[str, bytes, str], bool] @@ -100,7 +126,7 @@ def __init__(self, reason: str, message: str = "") -> None: class IngestionOutcome: """The observable outcome of ingesting one forwarded result.""" - status: str # "accepted" | "conflict" + status: str # "accepted" | "conflict" | "rejected" work_unit_id: str submission_id: str claimed_tier: int @@ -112,6 +138,9 @@ class IngestionOutcome: audit_sampled: bool | None = None audit_unit_id: str | None = None reason: str | None = None + attestation_mode: str | None = None + break_glass_admitted: bool = False + score_written: bool = False def to_response(self) -> dict[str, Any]: payload: dict[str, Any] = { @@ -124,6 +153,8 @@ def to_response(self) -> dict[str, Any]: "idempotent": self.idempotent, "finalized": self.finalized, "submission_status": self.submission_status, + "score_written": self.score_written, + "break_glass_admitted": self.break_glass_admitted, } if self.audit_sampled is not None: payload["audit_sampled"] = self.audit_sampled @@ -131,6 +162,8 @@ def to_response(self) -> dict[str, Any]: payload["audit_unit_id"] = self.audit_unit_id if self.reason is not None: payload["reason"] = self.reason + if self.attestation_mode is not None: + payload["attestation_mode"] = self.attestation_mode return payload @@ -204,22 +237,33 @@ async def ingest_work_unit_result( pinned_image_digest: str | None = None, audit_sampler: AuditSampler | None = None, verify: SignatureVerifier = verify_hotkey_signature, + constation_bundle: ConstationBundle | None = None, + check_allowlist: AllowlistChecker | None = None, + check_nonce: NonceChecker | None = None, + verify_constation_signature: ConstationSignatureVerifier | None = None, + constation_infra_fault: str | None = None, + constation_attempt: int = 1, + max_constation_attempts: int = DEFAULT_MAX_CONSTATION_ATTEMPTS, + break_glass: BreakGlassRequest | None = None, + break_glass_audit_log: BreakGlassAuditLog | None = None, ) -> IngestionOutcome: - """Verify a forwarded worker result and finalize the submission idempotently. + """Verify a forwarded worker result and finalize under strict fail-closed constation (P1). ``work_unit_id`` is prism's stable unit id (``== submission_id``). Verification (shape -> integrity) runs BEFORE any scoring; a rejected result raises :class:`ResultIngestionError` and - leaves the submission untouched (eligible for retry). A verified first delivery is then run - through the plausibility gate (architecture.md 3.5; VAL-PRISM-009): an implausible manifest - raises :class:`~prism_challenge.plausibility.PlausibilityError` (a reason DISTINCT from the - proof-verification reasons) and is never scored, while a plausible manifest passes through - UNCHANGED and finalizes via the CAS-guarded worker path. A duplicate is an idempotent no-op and - a conflicting redelivery for an already-accepted unit is refused so the stored score/leaderboard - is never mutated. - - Effective tier uses IMAGE_PIN only (max tier 1). Claimed attestation fields never elevate and - never block score finalization (Prism has no TEE-required scoring path). + leaves the submission untouched (eligible for retry). + + **P1 fail-closed (todo 22):** no valid constation bundle ⇒ **no** ``final_score`` row is + written at all. Failures carry ``miner_fault:*`` or ``infra_fault:*`` reason codes. + Infra-fault runs may be admitted only via an audited operator break-glass (todo 23); + miner-fault runs can never be override-admitted. Bounded retry applies to infra faults + before discard. + + Effective tier is granted only via ``constation_ok`` (M14 / todo 21); max tier is 1. + Claimed attestation / TEE fields never elevate. Self-reported ``PRISM_IMAGE_DIGEST`` is + telemetry only. """ + del submission_ref # reserved for future cross-checks if not isinstance(result, Mapping): raise ResultIngestionError("result_malformed", "result must be an object") @@ -229,15 +273,11 @@ async def ingest_work_unit_result( manifest = raw_manifest if isinstance(raw_manifest, Mapping) else None verify_proof_integrity(proof, unit_id=work_unit_id, manifest=manifest, verify=verify) - tier = effective_tier(proof, pinned_image_digest=pinned_image_digest) claimed_tier = int(proof.tier) - downgraded = claimed_tier != tier submission_id = work_unit_id - # Replication at acceptance (R=1 degraded or R=2 reconciled), forwarded by base for - # observability. It never affects audit eligibility: R=1 results are audited at their - # effective-tier rate exactly like R=2 ones (VAL-PRISM-026). replication = _as_int(result.get("replication"), 2) repository = worker.repository + att_mode = attestation_mode_of(proof) or ATTESTATION_MODE_V1 existing = await repository.get_work_unit_result(work_unit_id) if existing is not None: @@ -248,11 +288,13 @@ async def ingest_work_unit_result( work_unit_id=work_unit_id, submission_id=submission_id, claimed_tier=_as_int(existing.get("claimed_tier"), claimed_tier), - effective_tier=_as_int(existing.get("effective_tier"), tier), - tier_downgraded=bool(existing.get("tier_downgraded", downgraded)), + effective_tier=_as_int(existing.get("effective_tier"), 0), + tier_downgraded=bool(existing.get("tier_downgraded", True)), idempotent=True, finalized=False, submission_status=await repository.submission_status(submission_id), + attestation_mode=att_mode, + score_written=True, ) logger.warning( "rejecting conflicting result delivery for finalized work unit %s " @@ -266,14 +308,81 @@ async def ingest_work_unit_result( work_unit_id=work_unit_id, submission_id=submission_id, claimed_tier=claimed_tier, - effective_tier=tier, - tier_downgraded=downgraded, + effective_tier=0, + tier_downgraded=True, idempotent=False, finalized=False, submission_status=await repository.submission_status(submission_id), reason="manifest_conflict", + attestation_mode=att_mode, + score_written=True, ) + # --- Constation gate (P1) ----------------------------------------------------------------- + gate = _evaluate_constation_gate( + bundle=constation_bundle, + check_allowlist=check_allowlist, + check_nonce=check_nonce, + verify_constation_signature=verify_constation_signature, + constation_infra_fault=constation_infra_fault, + constation_attempt=constation_attempt, + max_constation_attempts=max_constation_attempts, + ) + break_glass_admitted = False + if not gate.admit: + if gate.retryable: + raise ResultIngestionError( + gate.reason or "infra_fault:constation_retry", + gate.message or "constation infra fault; retryable", + ) + # Optional break-glass for infra_fault only. + if break_glass is not None and fault_class_of(gate.reason or "") == "infra_fault": + decision = evaluate_break_glass( + break_glass, + fault_reason=gate.reason or infra_fault_reason(INFRA_FAULT_CONSTATION_UNAVAILABLE), + audit_log=break_glass_audit_log, + ) + if decision.admitted: + break_glass_admitted = True + logger.warning( + "break-glass admitted infra-fault run %s by operator %s", + work_unit_id, + break_glass.operator_id, + ) + else: + return await _reject_no_score( + work_unit_id=work_unit_id, + submission_id=submission_id, + claimed_tier=claimed_tier, + reason=gate.reason or miner_fault_reason("missing_constation_bundle"), + attestation_mode=att_mode, + repository=repository, + ) + else: + if break_glass is not None and fault_class_of(gate.reason or "") == "miner_fault": + # Explicit refuse path for attempted miner_fault override (todo 23). + evaluate_break_glass( + break_glass, + fault_reason=gate.reason or miner_fault_reason("unknown"), + audit_log=break_glass_audit_log, + ) + return await _reject_no_score( + work_unit_id=work_unit_id, + submission_id=submission_id, + claimed_tier=claimed_tier, + reason=gate.reason or miner_fault_reason("missing_constation_bundle"), + attestation_mode=att_mode, + repository=repository, + ) + + # Break-glass admits a score at tier 0 only — elevation still requires real constation_ok. + tier = effective_tier( + proof, + pinned_image_digest=pinned_image_digest, + constation_ok_result=bool(gate.constation_ok) and not break_glass_admitted, + ) + downgraded = claimed_tier != tier + if downgraded: logger.warning( "downgrading unverifiable tier claim for work unit %s: claimed %d -> effective %d", @@ -289,8 +398,6 @@ async def ingest_work_unit_result( ) if worker.settings.worker_plane.enabled: - # Worker plane: finalize from the forwarded, verified+reconciled manifest WITHOUT - # re-executing the evaluator (the heavy GPU work already ran on the miner-funded worker). if manifest is None: raise ResultIngestionError( "manifest_missing", @@ -302,15 +409,11 @@ async def ingest_work_unit_result( dict(manifest), ) except WorkerFinalizationError as exc: - # An internal/transient derivation failure is NOT a clean finalize: nothing is recorded - # (so a redelivery is genuinely retried, not idempotent-skipped) and the submission was - # reverted to pending. Surface it with a distinct, retryable reason. raise ResultIngestionError( "finalization_failed", f"worker-plane finalization failed transiently and is retryable: {exc}", ) from exc else: - # Flag OFF: legacy in-process re-execution finalization, byte-for-byte unchanged. result_id = await worker.process_submission(submission_id) submission_status = await repository.submission_status(submission_id) await repository.record_work_unit_result( @@ -326,10 +429,6 @@ async def ingest_work_unit_result( audit_unit_id: str | None = None if audit_sampler is not None: audit_sampled = audit_sampler.should_sample(work_unit_id=work_unit_id, effective_tier=tier) - # A sampled accepted result gets a validator audit unit on the existing dispatch path with a - # DISTINCT id; the audited submission is NOT reverted to pending (VAL-PRISM-012). R=1 - # (replication-degraded) results are sampled and audited at their effective-tier rate just - # like R=2-reconciled ones -- they are never exempted (VAL-PRISM-026). if audit_sampled: audit_unit_id = await repository.create_audit_unit( submission_id=submission_id, @@ -350,10 +449,131 @@ async def ingest_work_unit_result( submission_status=submission_status, audit_sampled=audit_sampled, audit_unit_id=audit_unit_id, + attestation_mode=att_mode, + break_glass_admitted=break_glass_admitted, + score_written=result_id is not None, + reason=gate.reason if break_glass_admitted else None, + ) + + +@dataclass(frozen=True) +class _ConstationGate: + admit: bool + constation_ok: bool + reason: str | None = None + message: str | None = None + retryable: bool = False + + +def _evaluate_constation_gate( + *, + bundle: ConstationBundle | None, + check_allowlist: AllowlistChecker | None, + check_nonce: NonceChecker | None, + verify_constation_signature: ConstationSignatureVerifier | None, + constation_infra_fault: str | None, + constation_attempt: int, + max_constation_attempts: int, +) -> _ConstationGate: + """Decide whether scoring may proceed under P1 fail-closed policy.""" + if constation_infra_fault: + reason = infra_fault_reason(constation_infra_fault) + if constation_attempt < max_constation_attempts: + return _ConstationGate( + admit=False, + constation_ok=False, + reason=reason, + message=f"infra fault attempt {constation_attempt}/{max_constation_attempts}", + retryable=True, + ) + return _ConstationGate( + admit=False, + constation_ok=False, + reason=infra_fault_reason(INFRA_FAULT_RETRY_EXHAUSTED) + if constation_attempt >= max_constation_attempts + else reason, + message="infra fault retry budget exhausted; no score", + retryable=False, + ) + + if bundle is None: + return _ConstationGate( + admit=False, + constation_ok=False, + reason=miner_fault_reason("missing_constation_bundle"), + message="no constation bundle; no score (P1)", + ) + + if check_allowlist is None or check_nonce is None or verify_constation_signature is None: + # Bundle present but checkers unavailable → infra (cannot evaluate). + reason = infra_fault_reason(INFRA_FAULT_CONSTATION_UNAVAILABLE) + if constation_attempt < max_constation_attempts: + return _ConstationGate( + admit=False, + constation_ok=False, + reason=reason, + message="constation checkers unavailable; retryable", + retryable=True, + ) + return _ConstationGate( + admit=False, + constation_ok=False, + reason=infra_fault_reason(INFRA_FAULT_RETRY_EXHAUSTED), + message="constation checkers unavailable; retries exhausted", + ) + + result: ConstationResult = constation_ok( + bundle, + check_allowlist=check_allowlist, + check_nonce=check_nonce, + verify_signature=verify_constation_signature, + ) + if result.ok: + return _ConstationGate(admit=True, constation_ok=True, reason="ok") + + fault = classify_constation_fault(result) + return _ConstationGate( + admit=False, + constation_ok=False, + reason=fault, + message=f"constation_ok failed: {result.reason.value}", + ) + + +async def _reject_no_score( + *, + work_unit_id: str, + submission_id: str, + claimed_tier: int, + reason: str, + attestation_mode: str, + repository: Any, +) -> IngestionOutcome: + """Return a rejected outcome without writing a score row (P1).""" + logger.warning( + "fail-closed: refusing score for work unit %s reason=%s", + work_unit_id, + reason, + ) + status = await repository.submission_status(submission_id) + return IngestionOutcome( + status="rejected", + work_unit_id=work_unit_id, + submission_id=submission_id, + claimed_tier=claimed_tier, + effective_tier=0, + tier_downgraded=claimed_tier != 0, + idempotent=False, + finalized=False, + submission_status=status, + reason=reason, + attestation_mode=attestation_mode, + score_written=False, ) __all__ = [ + "DEFAULT_MAX_CONSTATION_ATTEMPTS", "IngestionOutcome", "ResultIngestionError", "ingest_work_unit_result", diff --git a/src/prism_challenge/models.py b/src/prism_challenge/models.py index 3369e7e..0641123 100644 --- a/src/prism_challenge/models.py +++ b/src/prism_challenge/models.py @@ -139,8 +139,13 @@ class ArchitectureSummary(BaseModel): arch_hash: str name: str | None owner_hotkey: str + # Emission crown (held-out-primary q_arch_best). May stay 0 under skip_heldout. best_final_score: float best_submission_id: str + # Lab/inventory display: max scores.final_score in the family (includes + # worker-plane skip_heldout inventory that never advances the crown). + inventory_best_score: float | None = None + inventory_best_submission_id: str | None = None variant_count: int submission_count: int updated_at: datetime @@ -156,8 +161,13 @@ class ArchitectureDetailResponse(BaseModel): arch_hash: str name: str | None owner_hotkey: str + # Emission crown (held-out-primary q_arch_best). May stay 0 under skip_heldout. best_final_score: float best_submission_id: str + # Lab/inventory display: max scores.final_score in the family (includes + # worker-plane skip_heldout inventory that never advances the crown). + inventory_best_score: float | None = None + inventory_best_submission_id: str | None = None variant_count: int submission_count: int first_seen_at: datetime @@ -169,7 +179,10 @@ class TrainingVariantEntry(BaseModel): training_hash: str owner_hotkey: str submission_id: str + # Crown recipe scalar (q_recipe). May stay 0 under skip_heldout. final_score: float + # Inventory scalar from the linked submission's scores.final_score/q_arch. + inventory_final_score: float | None = None metric_mean: float metric_std: float is_current_best: bool diff --git a/src/prism_challenge/proof.py b/src/prism_challenge/proof.py index 188a53d..3c994b6 100644 --- a/src/prism_challenge/proof.py +++ b/src/prism_challenge/proof.py @@ -10,13 +10,18 @@ ``{manifest_sha256}:{unit_id}`` -- so a proof prism emits verifies with the same code as one the base worker plane emits, and a proof cannot be replayed across units. -Tiers (architecture.md 3.4): +Tiers (architecture.md 3.4; no-TEE residual): * tier 0 -- mandatory, all backends: canonical manifest hash + worker signature. -* tier 1 -- BOTH a pinned ``image_digest`` AND pod metadata (``provider.pod_id``). -* tier 2 -- CLAIMED when a closed structured attestation is present; EFFECTIVE tier 2 is granted - only after Prism TEE verification succeeds (LOCAL-FIXTURE PASS today). Opaque non-empty - ``tdx_quote_b64`` / ``gpu_eat_jwt`` alone never elevate effective tier. +* tier 1 -- CLAIMED when a pinned ``image_digest`` AND pod metadata are present; EFFECTIVE tier 1 + is granted **only** when :func:`~prism_challenge.constation.constation_ok` is True (M14). + Self-reported ``PRISM_IMAGE_DIGEST`` is telemetry only and never elevates. +* tier 2 -- may still be *claimed* for wire compatibility when a closed structured attestation + shape is present. EFFECTIVE tier is **never** 2: Prism has no TEE verifier path. Claimed tier + >= 2 collapses to effective 0. Opaque ``tdx_quote_b64`` / ``gpu_eat_jwt`` never elevate. + +Every emitted proof carries ``attestation_mode=miner_rent_image_pin_evidence_v1`` (image-identity +tamper-evidence on a miner-rented pod). Never ``lium_attested``, never any TEE-implying mode. Security invariant (VAL-PRISM-008): proof construction reads ONLY the manifest, the work unit id, the worker signer, and a FIXED ALLOWLIST of non-secret provider env vars. It never reads the @@ -75,9 +80,25 @@ ATTESTATION_ENV, ) -#: Documented attestation payload keys (architecture.md 3.4). +#: Documented attestation payload keys (architecture.md 3.4) — claim-shape only, never elevation. ATTESTATION_KEYS: tuple[str, ...] = ("tdx_quote_b64", "gpu_eat_jwt") +#: Sole honest attestation_mode value (todo 20). Never TEE-named, never "lium_attested". +ATTESTATION_MODE_V1 = "miner_rent_image_pin_evidence_v1" +ATTESTATION_MODE_KEY = "attestation_mode" +#: Forbidden mode strings that would overclaim independent/TEE verification. +FORBIDDEN_ATTESTATION_MODES: frozenset[str] = frozenset( + { + "lium_attested", + "tee", + "tee_attested", + "tdx", + "sev", + "cvm", + "hardware_root_of_trust", + } +) + @runtime_checkable class WorkerSigner(Protocol): @@ -157,27 +178,22 @@ def has_attestation(attestation: Any) -> bool: WARNING: this is ONLY a shape/presence hint for CLAIMED-tier emission compatibility. It is NEVER cryptographic verification and NEVER elevates effective tier. Prism has no - TEE verifier; effective tier is IMAGE_PIN only (max 1) via - :func:`~prism_challenge.audit.effective_tier`. + TEE verifier; effective tier is granted only via + :func:`~prism_challenge.constation.constation_ok` (max 1). """ if not isinstance(attestation, Mapping): return False tdx = attestation.get("tdx_quote_b64") gpu = attestation.get("gpu_eat_jwt") - return ( - isinstance(tdx, str) - and bool(tdx.strip()) - and isinstance(gpu, str) - and bool(gpu.strip()) - ) + return isinstance(tdx, str) and bool(tdx.strip()) and isinstance(gpu, str) and bool(gpu.strip()) def has_structured_attestation_claim(attestation: Any) -> bool: """Whether attestation claims the closed prism.tee.v1 shape (compat claim only; unverified). Used only to decide the CLAIMED emission tier for wire compatibility. Effective tier is - never elevated from this claim (max effective tier is 1 via IMAGE_PIN). + never elevated from this claim (max effective tier is 1 via constation_ok). """ if not has_attestation(attestation): @@ -194,6 +210,48 @@ def has_structured_attestation_claim(attestation: Any) -> bool: ) +def normalize_attestation_mode(mode: str | None) -> str: + """Return the sole honest mode; reject TEE/overclaim labels.""" + value = (mode or ATTESTATION_MODE_V1).strip() + if not value: + value = ATTESTATION_MODE_V1 + lowered = value.lower() + if lowered in FORBIDDEN_ATTESTATION_MODES or "tee" in lowered or "tdx" in lowered: + raise ValueError( + f"attestation_mode {value!r} is forbidden (no TEE / no independent Lium claim)" + ) + if value != ATTESTATION_MODE_V1: + raise ValueError(f"attestation_mode must be {ATTESTATION_MODE_V1!r}, got {value!r}") + return ATTESTATION_MODE_V1 + + +def attach_attestation_mode( + attestation: dict[str, Any] | None = None, + *, + mode: str = ATTESTATION_MODE_V1, +) -> dict[str, Any]: + """Return an attestation dict carrying the honest ``attestation_mode`` field.""" + out: dict[str, Any] = dict(attestation or {}) + out[ATTESTATION_MODE_KEY] = normalize_attestation_mode(mode) + return out + + +def attestation_mode_of(proof_or_attestation: Any) -> str | None: + """Read ``attestation_mode`` from a proof or attestation mapping, if present.""" + if proof_or_attestation is None: + return None + att = getattr(proof_or_attestation, "attestation", None) + if att is None and isinstance(proof_or_attestation, Mapping): + if ATTESTATION_MODE_KEY in proof_or_attestation: + raw = proof_or_attestation.get(ATTESTATION_MODE_KEY) + return str(raw) if raw is not None else None + att = proof_or_attestation.get("attestation") + if isinstance(att, Mapping): + raw = att.get(ATTESTATION_MODE_KEY) + return str(raw) if raw is not None else None + return None + + def compute_tier( *, image_digest: str | None, @@ -204,9 +262,10 @@ def compute_tier( tier 2 may still be *claimed* for a closed structured attestation shape (wire compat; never mere non-empty opaque strings). The claimed tier is NOT trusted at verification: - :func:`~prism_challenge.audit.effective_tier` recomputes from IMAGE_PIN only and never - elevates above tier 1. tier 1 iff BOTH a pinned image digest AND pod metadata - (``provider.pod_id``) are present; else tier 0. + :func:`~prism_challenge.audit.effective_tier` grants tier 1 only via constation_ok and + never elevates above tier 1. tier 1 is *claimed* iff BOTH an image digest AND pod + metadata (``provider.pod_id``) are present; else tier 0. Self-reported env digests do + not elevate. """ if has_structured_attestation_claim(attestation): @@ -232,12 +291,26 @@ def provider_from_env(env: Mapping[str, str] | None = None) -> ProviderInfo | No def image_digest_from_env(env: Mapping[str, str] | None = None) -> str | None: - """Read the evaluator image digest from the injected provider env.""" + """Read self-reported ``PRISM_IMAGE_DIGEST`` for **telemetry only**. + + This value must never be used as an elevation source. Elevation digests come + exclusively from a constation record (todo 20 / B5). + """ env = os.environ if env is None else env return _clean(env.get(IMAGE_DIGEST_ENV)) +def elevation_image_digest( + *, + constation_digest: str | None, + env_digest: str | None = None, +) -> str | None: + """Digest used for elevation: constation record only. ``env_digest`` is ignored.""" + del env_digest # telemetry only — never elevates + return _clean(constation_digest) + + def attestation_from_env(env: Mapping[str, str] | None = None) -> dict[str, Any] | None: """Read + parse the attestation payload (JSON) from the injected provider env, if any.""" @@ -261,15 +334,29 @@ def build_execution_proof( image_digest: str | None = None, attestation: dict[str, Any] | None = None, tier: ExecutionProofTier | None = None, + constation_digest: str | None = None, + attestation_mode: str = ATTESTATION_MODE_V1, ) -> ExecutionProof: """Build and sign an ExecutionProof binding ``manifest_sha256`` to ``unit_id`` under ``signer``. The tier is computed from the provenance unless explicitly overridden. ``signer`` is the WORKER keypair; its public identity becomes ``worker_signature.worker_pubkey``. + + When ``constation_digest`` is provided it is the elevation digest written on the proof; + otherwise ``image_digest`` is stored as telemetry only (still may appear on the wire for + observability). Every proof carries ``attestation_mode=miner_rent_image_pin_evidence_v1``. """ - effective_tier: ExecutionProofTier = ( - compute_tier(image_digest=image_digest, provider=provider, attestation=attestation) + digest_for_proof = elevation_image_digest( + constation_digest=constation_digest, env_digest=image_digest + ) + if digest_for_proof is None: + digest_for_proof = image_digest # telemetry / claim shape only + + att = attach_attestation_mode(attestation, mode=attestation_mode) + + claimed_tier: ExecutionProofTier = ( + compute_tier(image_digest=digest_for_proof, provider=provider, attestation=att) if tier is None else tier ) @@ -278,12 +365,12 @@ def build_execution_proof( ) return ExecutionProof( version=cast(ExecutionProofVersion, EXECUTION_PROOF_VERSION), - tier=effective_tier, + tier=claimed_tier, manifest_sha256=manifest_sha256, - image_digest=image_digest, + image_digest=digest_for_proof, provider=provider, worker_signature=WorkerSignature(worker_pubkey=signer.worker_pubkey, sig=signature), - attestation=attestation, + attestation=att, ) @@ -295,23 +382,28 @@ def build_execution_proof_from_manifest( manifest_bytes: bytes | None = None, manifest_path: str | os.PathLike[str] | None = None, env: Mapping[str, str] | None = None, + constation_digest: str | None = None, ) -> ExecutionProof: """Build a signed proof from a manifest source + the injected provider env. Exactly ONE manifest source must be given. Prefer ``manifest_path`` at emission time so the hash is taken from the exact on-disk bytes of ``prism_run_manifest.v2.json``. The provider provenance is read ONLY from the non-secret provider env allowlist. + + ``PRISM_IMAGE_DIGEST`` from env is telemetry; pass ``constation_digest`` for elevation. """ digest = _resolve_manifest_sha256( manifest=manifest, manifest_bytes=manifest_bytes, manifest_path=manifest_path ) + env_digest = image_digest_from_env(env) return build_execution_proof( signer=signer, manifest_sha256=digest, unit_id=unit_id, provider=provider_from_env(env), - image_digest=image_digest_from_env(env), + image_digest=env_digest, + constation_digest=constation_digest, attestation=attestation_from_env(env), ) @@ -361,8 +453,11 @@ def _clean(value: Any) -> str | None: __all__ = [ "ATTESTATION_ENV", "ATTESTATION_KEYS", + "ATTESTATION_MODE_KEY", + "ATTESTATION_MODE_V1", "EXECUTION_PROOF_VERSION", "EXECUTOR_ID_ENV", + "FORBIDDEN_ATTESTATION_MODES", "IMAGE_DIGEST_ENV", "MINER_HOTKEY_ENV", "POD_ID_ENV", @@ -375,17 +470,21 @@ def _clean(value: Any) -> str | None: "ProviderInfo", "WorkerSignature", "WorkerSigner", + "attach_attestation_mode", "attestation_from_env", + "attestation_mode_of", "build_execution_proof", "build_execution_proof_from_manifest", "canonical_manifest_json", "compute_manifest_sha256", "compute_tier", + "elevation_image_digest", "execution_proof_signing_payload", "has_attestation", "has_structured_attestation_claim", "image_digest_from_env", "manifest_sha256_from_bytes", + "normalize_attestation_mode", "provider_from_env", "read_manifest_sha256", "verify_execution_proof", diff --git a/src/prism_challenge/queue.py b/src/prism_challenge/queue.py index 6390aa5..2e59ff0 100644 --- a/src/prism_challenge/queue.py +++ b/src/prism_challenge/queue.py @@ -56,7 +56,46 @@ CONTAINER_EXECUTION_BACKENDS = frozenset( {"base_container", "base_gpu", "container_gpu", "docker_gpu"} ) +#: Always-on backends (no constation bundle required at worker construction). SUPPORTED_EXECUTION_BACKENDS = CONTAINER_EXECUTION_BACKENDS +#: Lium is gated: permitted only when a full constation bundle is present (todo 19). +LIUM_EXECUTION_BACKEND = "lium" +GATED_EXECUTION_BACKENDS = frozenset({LIUM_EXECUTION_BACKEND}) + + +def is_execution_backend_supported( + backend: str, + *, + constation_bundle: object | None = None, +) -> bool: + """Whether ``backend`` may be used under the current constation gate. + + Container backends (``base_gpu``, …) are always allowed. ``lium`` is allowed + **only** when a full constation bundle object is supplied — bare Lium without + a bundle stays rejected. This does not evaluate ``constation_ok``; that is the + ingestion elevation path (todos 21–22). + """ + if backend in SUPPORTED_EXECUTION_BACKENDS: + return True + if backend == LIUM_EXECUTION_BACKEND: + return constation_bundle is not None + return False + + +def require_execution_backend( + backend: str, + *, + constation_bundle: object | None = None, +) -> None: + """Raise ``ValueError`` when ``backend`` is not permitted under the gate.""" + if is_execution_backend_supported(backend, constation_bundle=constation_bundle): + return + if backend == LIUM_EXECUTION_BACKEND: + raise ValueError( + f"Unsupported execution backend: {backend}: constation bundle required for lium" + ) + raise ValueError(f"Unsupported execution backend: {backend}") + logger = logging.getLogger(__name__) @@ -140,15 +179,16 @@ def __init__( settings: PrismSettings | None = None, evaluator_factory: EvaluatorFactory | None = None, checkpoint_publisher: CheckpointPublisher | None = None, + constation_bundle: object | None = None, ) -> None: - if execution_backend not in SUPPORTED_EXECUTION_BACKENDS: - raise ValueError(f"Unsupported execution backend: {execution_backend}") + require_execution_backend(execution_backend, constation_bundle=constation_bundle) self.repository = repository self.ctx = ctx self.execution_backend = execution_backend self.settings = settings or PrismSettings() self._evaluator_factory = evaluator_factory or _default_evaluator_factory self._checkpoint_publisher = checkpoint_publisher + self._constation_bundle = constation_bundle async def process_next(self) -> str | None: submission = await self.repository.claim_next() @@ -262,7 +302,10 @@ async def replay_audit_manifest_sha256( repeated (the honest run is deterministic, so an honest worker's hash reproduces). Returns ``None`` on any replay failure, resolving the audit inconclusive rather than confirming it. """ - if self.execution_backend not in CONTAINER_EXECUTION_BACKENDS: + # Lium runs are miner-side; validator audit replay uses the same container path. + if self.execution_backend not in CONTAINER_EXECUTION_BACKENDS and ( + self.execution_backend != LIUM_EXECUTION_BACKEND + ): return None submission = await self.repository.submission_execution_row(submission_id) if submission is None: @@ -351,7 +394,10 @@ async def _process_claimed( metadata = cast(dict[str, Any], raw_metadata) if isinstance(raw_metadata, dict) else {} hotkey = str(submission.get("hotkey") or "") code_hash = str(submission.get("code_hash") or sha256(code.encode()).hexdigest()) - if self.execution_backend in CONTAINER_EXECUTION_BACKENDS: + if ( + self.execution_backend in CONTAINER_EXECUTION_BACKENDS + or self.execution_backend == LIUM_EXECUTION_BACKEND + ): return await self._process_container( submission_id, code, diff --git a/src/prism_challenge/routes.py b/src/prism_challenge/routes.py index 029c7cf..0a40ee6 100644 --- a/src/prism_challenge/routes.py +++ b/src/prism_challenge/routes.py @@ -16,6 +16,7 @@ from pydantic import ValidationError from .admission import enforce_admission +from .attestation_routes import build_attestation_public_router from .auth import authenticate_miner from .evaluator.train_series import downsample_train_series_for_api from .models import ( @@ -46,6 +47,27 @@ router = APIRouter(prefix="/v1") +# Public attestation challenge/answer (published via BASE proxy as +# /challenges/prism/v1/attestation/*). Lives on the challenge app, not master. +router.include_router(build_attestation_public_router()) + + +def _optional_float(value: object | None) -> float | None: + if value is None: + return None + try: + number = float(cast(SupportsFloat, value)) + except (TypeError, ValueError): + return None + return number if number == number else None # NaN → None + + +def _optional_str(value: object | None) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + def repo_from_request(request: Request) -> PrismRepository: return request.app.state.repository @@ -208,6 +230,8 @@ async def list_architectures( owner_hotkey=str(row["owner_hotkey"]), best_final_score=float(cast(SupportsFloat, row["best_final_score"])), best_submission_id=str(row["best_submission_id"]), + inventory_best_score=_optional_float(row.get("inventory_best_score")), + inventory_best_submission_id=_optional_str(row.get("inventory_best_submission_id")), variant_count=int(cast(SupportsInt, row["variant_count"])), submission_count=int(cast(SupportsInt, row["submission_count"])), updated_at=datetime.fromisoformat(str(row["updated_at"])), @@ -232,6 +256,8 @@ async def get_architecture( owner_hotkey=str(row["owner_hotkey"]), best_final_score=float(cast(SupportsFloat, row["best_final_score"])), best_submission_id=str(row["best_submission_id"]), + inventory_best_score=_optional_float(row.get("inventory_best_score")), + inventory_best_submission_id=_optional_str(row.get("inventory_best_submission_id")), variant_count=int(cast(SupportsInt, row["variant_count"])), submission_count=int(cast(SupportsInt, row["submission_count"])), first_seen_at=datetime.fromisoformat(str(row["first_seen_at"])), @@ -255,6 +281,7 @@ async def list_architecture_variants( owner_hotkey=str(row["owner_hotkey"]), submission_id=str(row["submission_id"]), final_score=float(cast(SupportsFloat, row["final_score"])), + inventory_final_score=_optional_float(row.get("inventory_final_score")), metric_mean=float(cast(SupportsFloat, row["metric_mean"])), metric_std=float(cast(SupportsFloat, row["metric_std"])), is_current_best=bool(row["is_current_best"]), diff --git a/tests/conftest.py b/tests/conftest.py index 47e6d62..5587796 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -101,3 +101,67 @@ def client(tmp_path: Path) -> TestClient: ) with TestClient(create_app(settings)) as test_client: yield test_client + + +# --- Wave 4 constation auto-inject for legacy tests (todo 22) ------------------------------------ +# Production P1: no bundle ⇒ no score. Legacy tests that never heard of constation omit the +# kwargs entirely; we inject a valid bundle so they keep exercising finalization/audit. +# Tests that pass ``constation_bundle=None`` (or ``constation_infra_fault=...``) explicitly +# exercise the fail-closed / break-glass paths and are left alone. + + +@pytest.fixture(autouse=True) +def _auto_constation_for_legacy_ingestion( + monkeypatch: pytest.MonkeyPatch, request: pytest.FixtureRequest +) -> None: + # Production-path tests must exercise real fail-closed without the legacy seam. + modname = getattr(getattr(request, "module", None), "__name__", "") or "" + if "prod_constation" in modname or request.node.get_closest_marker("no_auto_constation"): + return + + import prism_challenge.ingestion as ingestion_mod + from prism_challenge.constation import CheckOutcome, ConstationBundle + + original = ingestion_mod.ingest_work_unit_result + + def _ok(**_k: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def _bundle() -> ConstationBundle: + man = {"legacy-test-harness.py": "a" * 64} + digest = "sha256:" + ("11" * 32) + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="legacy-wu", + miner_hotkey="legacy-hk", + pod_id="legacy-pod", + nonce="legacy-nonce", + signed_attestation={"legacy": True}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + async def _wrapped(**kwargs): # type: ignore[no-untyped-def] + if ( + "constation_bundle" not in kwargs + and "constation_infra_fault" not in kwargs + and kwargs.get("check_allowlist") is None + ): + kwargs = dict(kwargs) + kwargs["constation_bundle"] = _bundle() + kwargs["check_allowlist"] = _ok + kwargs["check_nonce"] = _ok + kwargs["verify_constation_signature"] = lambda _s: _ok() + return await original(**kwargs) + + monkeypatch.setattr(ingestion_mod, "ingest_work_unit_result", _wrapped) + # Patch the bound name on the requesting test module (from-import sites). + mod = getattr(request, "module", None) + if mod is not None and getattr(mod, "ingest_work_unit_result", None) is not None: + monkeypatch.setattr(mod, "ingest_work_unit_result", _wrapped, raising=False) diff --git a/tests/test_constation_bundle_wire.py b/tests/test_constation_bundle_wire.py new file mode 100644 index 0000000..3b879a0 --- /dev/null +++ b/tests/test_constation_bundle_wire.py @@ -0,0 +1,50 @@ +"""Wire serdes for ConstationBundle (production HTTP path).""" + +from __future__ import annotations + +import pytest + +from prism_challenge.constation import ( + ConstationBundle, + constation_bundle_from_dict, + constation_bundle_to_dict, +) + + +def _bundle() -> ConstationBundle: + man = {"h.py": "a" * 64} + digest = "sha256:" + ("1" * 64) + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="wu-1", + miner_hotkey="hk", + pod_id="pod", + nonce="nonce-1", + signed_attestation={"sig": "x"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + +def test_roundtrip() -> None: + b = _bundle() + again = constation_bundle_from_dict(constation_bundle_to_dict(b)) + assert again == b + + +def test_missing_field_raises() -> None: + raw = constation_bundle_to_dict(_bundle()) + del raw["nonce"] + with pytest.raises(ValueError, match="missing"): + constation_bundle_from_dict(raw) + + +def test_non_object_raises() -> None: + with pytest.raises(ValueError, match="object"): + constation_bundle_from_dict([]) diff --git a/tests/test_constation_ok.py b/tests/test_constation_ok.py new file mode 100644 index 0000000..9eb8beb --- /dev/null +++ b/tests/test_constation_ok.py @@ -0,0 +1,424 @@ +"""TDD tests for constation_ok — sole elevation predicate (checkbox 12 / M14). + +constation_ok is the ONLY path that may authorize tier elevation. Each required +mechanism is tested in isolation: omitting or breaking any one yields False with +a distinct machine reason code. Dependencies are injected so tests never need +live Lium or network. +""" + +from __future__ import annotations + +from dataclasses import FrozenInstanceError, dataclass +from typing import Any + +import pytest + +from prism_challenge.constation import ( + CheckOutcome, + ConstationBundle, + ConstationFailReason, + ConstationResult, + constation_ok, +) + +COMMIT = "a" * 40 +TREE = "c" * 40 +DIGEST = "sha256:" + ("1" * 64) +DIGEST_OTHER = "sha256:" + ("2" * 64) +NONCE = "550e8400-e29b-41d4-a716-446655440000" +POD = "pod_test_001" +HOTKEY = "5FakeHotkeyForUnitTestsOnly000000000000000" +WORK_UNIT = "wu-unit-001" +VARIANT = "cuda" +MANIFEST: dict[str, str] = { + "src/prism_recipe/harness.py": "a" * 64, + "src/prism_recipe/gpu_train.py": "b" * 64, +} +GAP_BUDGET = 30.0 + + +def _ok() -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + +def _miss(reason: str) -> CheckOutcome: + return CheckOutcome(ok=False, reason=reason) + + +def _bundle(**overrides: Any) -> ConstationBundle: + fields: dict[str, Any] = { + "commit_sha": COMMIT, + "tree_sha": TREE, + "variant": VARIANT, + "digest": DIGEST, + "work_unit_id": WORK_UNIT, + "miner_hotkey": HOTKEY, + "pod_id": POD, + "nonce": NONCE, + "signed_attestation": {"schema": "test", "signature": "deadbeef"}, + "expected_sealed_manifest_hashes": dict(MANIFEST), + "reported_sealed_manifest_hashes": dict(MANIFEST), + "lium_declared_digest": DIGEST, + "constation_gap_budget_seconds": GAP_BUDGET, + "constation_observed_max_gap_seconds": 5.0, + } + fields.update(overrides) + return ConstationBundle(**fields) + + +@dataclass(frozen=True, slots=True) +class _Injected: + """Injectable checker outcomes for a single constation_ok call.""" + + allowlist: CheckOutcome = CheckOutcome(ok=True, reason="ok") + nonce: CheckOutcome = CheckOutcome(ok=True, reason="ok") + signature: CheckOutcome = CheckOutcome(ok=True, reason="ok") + + +def _run( + bundle: ConstationBundle, + inj: _Injected | None = None, +) -> ConstationResult: + inj = inj or _Injected() + + def check_allowlist( + *, + digest: str, + commit_sha: str, + tree_sha: str, + variant: str, + ) -> CheckOutcome: + del digest, commit_sha, tree_sha, variant + return inj.allowlist + + def check_nonce( + *, + nonce: str, + work_unit_id: str, + miner_hotkey: str, + pod_id: str, + ) -> CheckOutcome: + del nonce, work_unit_id, miner_hotkey, pod_id + return inj.nonce + + def verify_signature(signed: object) -> CheckOutcome: + del signed + return inj.signature + + return constation_ok( + bundle, + check_allowlist=check_allowlist, + check_nonce=check_nonce, + verify_signature=verify_signature, + ) + + +def test_complete_valid_bundle_returns_true() -> None: + """S1 happy: all six mechanisms pass → ok=True reason=ok.""" + result = _run(_bundle()) + + assert result.ok is True + assert result.reason is ConstationFailReason.OK + assert bool(result) is True + + +def test_allowlist_miss_unknown_digest() -> None: + result = _run( + _bundle(), + _Injected(allowlist=_miss("unknown_digest")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST + assert result.detail == "unknown_digest" + + +def test_allowlist_miss_variant_mismatch() -> None: + result = _run( + _bundle(), + _Injected(allowlist=_miss("variant_mismatch")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.ALLOWLIST_VARIANT_MISMATCH + + +def test_allowlist_miss_commit_mismatch() -> None: + result = _run( + _bundle(), + _Injected(allowlist=_miss("commit_mismatch")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.ALLOWLIST_COMMIT_MISMATCH + + +def test_allowlist_miss_revoked() -> None: + result = _run( + _bundle(), + _Injected(allowlist=_miss("revoked")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.ALLOWLIST_REVOKED + + +def test_nonce_already_consumed() -> None: + result = _run( + _bundle(), + _Injected(nonce=_miss("already_consumed")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.NONCE_ALREADY_CONSUMED + + +def test_nonce_unknown() -> None: + result = _run( + _bundle(), + _Injected(nonce=_miss("unknown_nonce")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.NONCE_UNKNOWN + + +def test_nonce_expired() -> None: + result = _run( + _bundle(), + _Injected(nonce=_miss("expired")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.NONCE_EXPIRED + + +def test_signature_mismatch() -> None: + result = _run( + _bundle(), + _Injected(signature=_miss("signature_mismatch")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.SIGNATURE_INVALID + assert result.detail == "signature_mismatch" + + +def test_sealed_manifest_mismatch() -> None: + bad_manifest = {**MANIFEST, "src/prism_recipe/harness.py": "f" * 64} + result = _run( + _bundle(reported_sealed_manifest_hashes=bad_manifest), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.SEALED_MANIFEST_MISMATCH + + +def test_corroboration_mismatch_fails() -> None: + """Negative-only: Lium-declared digest disagrees with sidecar digest.""" + result = _run(_bundle(lium_declared_digest=DIGEST_OTHER)) + + assert result.ok is False + assert result.reason is ConstationFailReason.CORROBORATION_MISMATCH + + +def test_corroboration_agreement_alone_insufficient() -> None: + """Agreement contributes nothing: allowlist miss still fails.""" + result = _run( + _bundle(lium_declared_digest=DIGEST), + _Injected(allowlist=_miss("unknown_digest")), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST + + +def test_constation_gap_beyond_budget() -> None: + result = _run( + _bundle( + constation_gap_budget_seconds=10.0, + constation_observed_max_gap_seconds=10.0001, + ), + ) + + assert result.ok is False + assert result.reason is ConstationFailReason.CONSTATION_GAP + + +def test_gap_within_budget_ok() -> None: + result = _run( + _bundle( + constation_gap_budget_seconds=10.0, + constation_observed_max_gap_seconds=10.0, + ), + ) + + assert result.ok is True + assert result.reason is ConstationFailReason.OK + + +@pytest.mark.parametrize( + ("label", "bundle_kw", "inj", "expected"), + [ + ( + "allowlist", + {}, + _Injected(allowlist=_miss("unknown_digest")), + ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST, + ), + ( + "nonce", + {}, + _Injected(nonce=_miss("already_consumed")), + ConstationFailReason.NONCE_ALREADY_CONSUMED, + ), + ( + "signature", + {}, + _Injected(signature=_miss("signature_mismatch")), + ConstationFailReason.SIGNATURE_INVALID, + ), + ( + "sealed_manifest", + { + "reported_sealed_manifest_hashes": { + **MANIFEST, + "src/prism_recipe/harness.py": "0" * 64, + } + }, + _Injected(), + ConstationFailReason.SEALED_MANIFEST_MISMATCH, + ), + ( + "corroboration", + {"lium_declared_digest": DIGEST_OTHER}, + _Injected(), + ConstationFailReason.CORROBORATION_MISMATCH, + ), + ( + "constation_gap", + { + "constation_gap_budget_seconds": 1.0, + "constation_observed_max_gap_seconds": 2.0, + }, + _Injected(), + ConstationFailReason.CONSTATION_GAP, + ), + ], + ids=[ + "allowlist", + "nonce", + "signature", + "sealed_manifest", + "corroboration", + "constation_gap", + ], +) +def test_each_single_mechanism_omission_fails_with_distinct_reason( + label: str, + bundle_kw: dict[str, Any], + inj: _Injected, + expected: ConstationFailReason, +) -> None: + """Parameterized: each single-field/mechanism break → False + distinct reason.""" + del label + result = _run(_bundle(**bundle_kw), inj) + + assert result.ok is False + assert result.reason is expected + assert result.reason is not ConstationFailReason.OK + + +def test_omission_reasons_are_pairwise_distinct() -> None: + """The six mechanism failure reasons used by the param table must all differ.""" + reasons = [ + ConstationFailReason.ALLOWLIST_UNKNOWN_DIGEST, + ConstationFailReason.NONCE_ALREADY_CONSUMED, + ConstationFailReason.SIGNATURE_INVALID, + ConstationFailReason.SEALED_MANIFEST_MISMATCH, + ConstationFailReason.CORROBORATION_MISMATCH, + ConstationFailReason.CONSTATION_GAP, + ] + assert len(reasons) == len(set(reasons)) + + +def test_module_docstring_states_sole_elevation_predicate_m14() -> None: + import prism_challenge.constation as mod + + doc = (mod.__doc__ or "") + (constation_ok.__doc__ or "") + lowered = doc.lower() + assert "sole" in lowered or "only" in lowered + assert "tier" in lowered + assert "m14" in lowered or "no other" in lowered + + +def test_module_exposes_no_tier_grant_api() -> None: + import prism_challenge.constation as mod + + forbidden = { + "effective_tier", + "grant_tier", + "elevate_tier", + "set_tier", + "compute_tier", + } + names = {n for n in dir(mod) if not n.startswith("_")} + assert names.isdisjoint(forbidden) + + +def test_checkers_receive_bundle_fields() -> None: + """Injected checkers are called with the bundle's identity fields.""" + seen: dict[str, Any] = {} + + def check_allowlist( + *, + digest: str, + commit_sha: str, + tree_sha: str, + variant: str, + ) -> CheckOutcome: + seen["allowlist"] = (digest, commit_sha, tree_sha, variant) + return _ok() + + def check_nonce( + *, + nonce: str, + work_unit_id: str, + miner_hotkey: str, + pod_id: str, + ) -> CheckOutcome: + seen["nonce"] = (nonce, work_unit_id, miner_hotkey, pod_id) + return _ok() + + def verify_signature(signed: object) -> CheckOutcome: + seen["sig"] = signed + return _ok() + + bundle = _bundle() + result = constation_ok( + bundle, + check_allowlist=check_allowlist, + check_nonce=check_nonce, + verify_signature=verify_signature, + ) + + assert result.ok is True + assert seen["allowlist"] == (DIGEST, COMMIT, TREE, VARIANT) + assert seen["nonce"] == (NONCE, WORK_UNIT, HOTKEY, POD) + assert seen["sig"] == bundle.signed_attestation + + +def test_missing_lium_corroboration_is_not_contradiction() -> None: + """Negative-only: absent Lium channel does not fail (not a positive grant).""" + result = _run(_bundle(lium_declared_digest=None)) + + assert result.ok is True + assert result.reason is ConstationFailReason.OK + + +def test_result_is_structured_and_frozen() -> None: + result = _run(_bundle(), _Injected(allowlist=_miss("revoked"))) + assert isinstance(result, ConstationResult) + assert result.ok is False + with pytest.raises((AttributeError, TypeError, FrozenInstanceError)): + result.ok = True # type: ignore[misc] diff --git a/tests/test_constation_scoring_gate.py b/tests/test_constation_scoring_gate.py new file mode 100644 index 0000000..1236f4c --- /dev/null +++ b/tests/test_constation_scoring_gate.py @@ -0,0 +1,657 @@ +"""Wave 4 todos 20–23: attestation_mode, effective_tier, fail-closed, break-glass.""" + +from __future__ import annotations + +import base64 +import io +import math +import sqlite3 +import zipfile +from pathlib import Path +from typing import Any + +import pytest + +from prism_challenge.app import create_app +from prism_challenge.audit import effective_tier +from prism_challenge.breakglass import ( + BreakGlassAuditLog, + BreakGlassRequest, + evaluate_break_glass, +) +from prism_challenge.config import PrismSettings, WorkerPlaneConfig +from prism_challenge.constation import ( + CheckOutcome, + ConstationBundle, + ConstationFailReason, + constation_ok, +) +from prism_challenge.evaluator.mock_reexec import cpu_reexec_run +from prism_challenge.ingestion import ingest_work_unit_result +from prism_challenge.models import SubmissionCreate +from prism_challenge.proof import ( + ATTESTATION_MODE_V1, + MANIFEST_PAYLOAD_KEY, + PROOF_PAYLOAD_KEY, + ExecutionProof, + ProviderInfo, + WorkerSignature, + attach_attestation_mode, + attestation_mode_of, + build_execution_proof, + compute_manifest_sha256, + elevation_image_digest, + image_digest_from_env, + normalize_attestation_mode, + worker_signer_from_key, +) + +WORKER_KEY = "//WorkerConstationGate" +PINNED = "sha256:" + ("aa" * 32) +OTHER = "sha256:" + ("bb" * 32) +DIGEST = "sha256:" + ("11" * 32) + +TINY_ARCH = """ +import torch +from torch import nn + + +class TinyLM(nn.Module): + def __init__(self, vocab): + super().__init__() + self.emb = nn.Embedding(vocab, 8) + self.head = nn.Linear(8, vocab) + + def forward(self, tokens): + return self.head(self.emb(tokens)) + + +def build_model(ctx): + return TinyLM(ctx.vocab_size) +""" + +TINY_TRAIN = """ +import torch +import torch.nn.functional as F + + +def train(ctx): + model = ctx.build_model() + opt = torch.optim.AdamW(model.parameters(), lr=0.01) + for batch in ctx.iter_train_batches(model, batch_size=1): + opt.zero_grad() + logits = model(batch.tokens) + nv = logits.shape[-1] + loss = F.cross_entropy( + logits[:, :-1, :].reshape(-1, nv), batch.tokens[:, 1:].reshape(-1) % nv + ) + loss.backward() + opt.step() +""" + +_SHARD_LINE = ( + '{{"id": "doc-{i}", "text": "the locked fineweb edu training sample number {i} ' + 'has enough bytes to cover several challenge instrument batches deterministically"}}\n' +) + + +def _stage_train(root: Path, *, lines: int = 64) -> Path: + data_dir = root / "train-data" + data_dir.mkdir(parents=True, exist_ok=True) + (data_dir / "train-00000.jsonl").write_text( + "".join(_SHARD_LINE.format(i=i) for i in range(lines)), encoding="utf-8" + ) + return data_dir + + +def _code_bundle() -> str: + stream = io.BytesIO() + with zipfile.ZipFile(stream, "w") as archive: + archive.writestr("architecture.py", TINY_ARCH) + archive.writestr("training.py", TINY_TRAIN) + return base64.b64encode(stream.getvalue()).decode("ascii") + + +def _settings(tmp_path: Path) -> PrismSettings: + return PrismSettings( + database_url=f"sqlite+aiosqlite:///{tmp_path / 'coord.sqlite3'}", + shared_token="secret", + allow_insecure_signatures=True, + execution_backend="base_gpu", + docker_enabled=True, + docker_backend="broker", + docker_broker_url="http://base-docker-broker:8082", + docker_broker_token="secret", + sequence_length=16, + plagiarism_enabled=False, + distributed_contract_policy="off", + base_eval_artifact_root=tmp_path / "artifacts", + worker_plane=WorkerPlaneConfig(enabled=True, signing_key=WORKER_KEY), + ) + + +def _manifest(marker: str = "v2") -> dict[str, Any]: + covered_bytes = 4096 + online_loss = [10.0, 6.0, 3.0, 2.0] + return { + "schema_version": "prism_run_manifest.v2", + "data": {"covered_bytes": covered_bytes, "single_pass": True}, + "metrics": { + "online_loss": online_loss, + "sum_neg_log_likelihood_nats": 900.0, + "covered_bytes": covered_bytes, + "predicted_tokens": 96, + "step0_loss": online_loss[0], + "consumed_batches": len(online_loss), + "random_init_baseline_nats": math.log(50257), + "prequential_bpb": 1.23, + "marker": marker, + }, + "anti_cheat": { + "step0_anomaly": False, + "nan_inf_detected": False, + "no_learning": False, + "zero_forward": False, + }, + } + + +def _constation_bundle(*, digest: str = DIGEST) -> ConstationBundle: + man = {"src/prism_recipe/harness.py": "a" * 64} + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="wu", + miner_hotkey="hk", + pod_id="pod-1", + nonce="nonce-1", + signed_attestation={"sig": "ok"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + +def _ok_checkers(): + def allow(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def nonce(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def sig(_s: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + return allow, nonce, sig + + +def _fail_manifest_checkers(): + def allow(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def nonce(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def sig(_s: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + return allow, nonce, sig + + +def _tier1_proof(signer, unit_id: str, manifest: dict[str, Any], *, image_digest: str): + digest = compute_manifest_sha256(manifest) + proof = build_execution_proof( + signer=signer, + manifest_sha256=digest, + unit_id=unit_id, + image_digest=image_digest, + constation_digest=image_digest, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + tier=1, # type: ignore[arg-type] + ) + return proof.model_dump(mode="json") + + +def _result(proof_dict: dict[str, Any], manifest: dict[str, Any]) -> dict[str, Any]: + return { + "executed": 1, + "completed_submissions": [], + PROOF_PAYLOAD_KEY: proof_dict, + MANIFEST_PAYLOAD_KEY: manifest, + } + + +async def _make_app(settings: PrismSettings): + app = create_app(settings) + await app.state.database.init() + return app + + +async def _seed(app, hotkey: str = "hk-owner") -> str: + sub = await app.state.repository.create_submission( + hotkey, SubmissionCreate(code=_code_bundle(), filename="project.zip") + ) + return sub.id + + +def _final_score(db_path: Path, submission_id: str) -> float | None: + conn = sqlite3.connect(db_path) + try: + row = conn.execute( + "SELECT final_score FROM scores WHERE submission_id=?", (submission_id,) + ).fetchone() + finally: + conn.close() + return None if row is None else float(row[0]) + + +# --- todo 20 ------------------------------------------------------------------------------------ + + +def test_attestation_mode_is_miner_rent_image_pin_evidence_v1() -> None: + att = attach_attestation_mode(None) + assert att["attestation_mode"] == ATTESTATION_MODE_V1 + assert normalize_attestation_mode(ATTESTATION_MODE_V1) == ATTESTATION_MODE_V1 + with pytest.raises(ValueError, match="forbidden"): + normalize_attestation_mode("lium_attested") + with pytest.raises(ValueError, match="forbidden"): + normalize_attestation_mode("tee_attested") + + +def test_env_digest_is_telemetry_not_elevation() -> None: + assert elevation_image_digest(constation_digest=DIGEST, env_digest=OTHER) == DIGEST + assert elevation_image_digest(constation_digest=None, env_digest=OTHER) is None + # image_digest_from_env still reads env but docstring marks telemetry-only + assert image_digest_from_env({"PRISM_IMAGE_DIGEST": OTHER}) == OTHER + + +def test_build_proof_always_sets_attestation_mode() -> None: + signer = worker_signer_from_key(WORKER_KEY) + proof = build_execution_proof( + signer=signer, + manifest_sha256="c" * 64, + unit_id="u1", + image_digest=PINNED, + provider=ProviderInfo(name="lium", pod_id="p"), + tier=1, # type: ignore[arg-type] + ) + assert attestation_mode_of(proof) == ATTESTATION_MODE_V1 + + +def test_selfreport_digest_match_without_constation_is_tier0() -> None: + """Correct PRISM_IMAGE_DIGEST alone cannot reach tier 1 (todo 20 failure path).""" + proof = ExecutionProof( + version=1, + tier=1, + manifest_sha256="c" * 64, + image_digest=PINNED, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + attestation=attach_attestation_mode(None), + ) + assert effective_tier(proof, pinned_image_digest=PINNED) == 0 + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=False) == 0 + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=True) == 1 + + +# --- todo 21 ------------------------------------------------------------------------------------ + + +def test_tier1_only_when_constation_ok_true() -> None: + proof = ExecutionProof( + version=1, + tier=1, + manifest_sha256="c" * 64, + image_digest=PINNED, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + ) + assert effective_tier(proof, constation_ok_result=True) == 1 + assert effective_tier(proof, constation_ok_result=False) == 0 + assert effective_tier(proof, constation_ok_result=None) == 0 + + +def test_claimed_tier2_always_zero_even_with_constation() -> None: + proof = ExecutionProof( + version=1, + tier=2, + manifest_sha256="c" * 64, + image_digest=PINNED, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + attestation={"tdx_quote_b64": "x", "gpu_eat_jwt": "y"}, + ) + assert effective_tier(proof, constation_ok_result=True) == 0 + + +def test_constation_result_object_drives_tier() -> None: + proof = ExecutionProof( + version=1, + tier=1, + manifest_sha256="c" * 64, + image_digest=DIGEST, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + ) + allow, nonce, sig = _ok_checkers() + ok = constation_ok( + _constation_bundle(), + check_allowlist=allow, + check_nonce=nonce, + verify_signature=sig, + ) + assert ok.ok is True + assert effective_tier(proof, constation_ok_result=ok) == 1 + + bad_bundle = _constation_bundle() + # force sealed mismatch + from dataclasses import replace + + bad = replace( + bad_bundle, + reported_sealed_manifest_hashes={"src/prism_recipe/harness.py": "f" * 64}, + ) + fail = constation_ok(bad, check_allowlist=allow, check_nonce=nonce, verify_signature=sig) + assert fail.ok is False + assert fail.reason == ConstationFailReason.SEALED_MANIFEST_MISMATCH + assert effective_tier(proof, constation_ok_result=fail) == 0 + + +# --- todo 22 ------------------------------------------------------------------------------------ + + +@pytest.mark.asyncio +async def test_valid_bundle_writes_score_with_attestation_mode( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app) + manifest = _manifest("ok") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + allow, nonce, sig = _ok_checkers() + + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-owner", + result=_result(proof, manifest), + pinned_image_digest=DIGEST, + constation_bundle=_constation_bundle(digest=DIGEST), + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, + ) + assert outcome.status == "accepted" + assert outcome.finalized is True + assert outcome.score_written is True + assert outcome.effective_tier == 1 + assert outcome.attestation_mode == ATTESTATION_MODE_V1 + score = _final_score(db_path, submission_id) + assert score is not None and score > 0.0 + + +@pytest.mark.asyncio +async def test_no_bundle_writes_no_score_miner_fault( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app) + manifest = _manifest("nobundle") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=PINNED) + + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-owner", + result=_result(proof, manifest), + pinned_image_digest=PINNED, + constation_bundle=None, + ) + assert outcome.status == "rejected" + assert outcome.finalized is False + assert outcome.score_written is False + assert outcome.reason == "miner_fault:missing_constation_bundle" + assert _final_score(db_path, submission_id) is None + + +@pytest.mark.asyncio +async def test_manifest_mismatch_miner_fault_no_score( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app, hotkey="hk-mm") + manifest = _manifest("mm") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + allow, nonce, sig = _ok_checkers() + from dataclasses import replace + + bad = replace( + _constation_bundle(digest=DIGEST), + reported_sealed_manifest_hashes={"src/prism_recipe/harness.py": "f" * 64}, + ) + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-mm", + result=_result(proof, manifest), + constation_bundle=bad, + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, + ) + assert outcome.status == "rejected" + assert outcome.score_written is False + assert outcome.reason == "miner_fault:manifest_mismatch" + assert _final_score(db_path, submission_id) is None + + +@pytest.mark.asyncio +async def test_infra_fault_constation_unavailable_no_score( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app, hotkey="hk-infra") + manifest = _manifest("infra") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + + # Exhaust retries → no score + from prism_challenge.ingestion import ResultIngestionError + + with pytest.raises(ResultIngestionError) as ei: + await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-infra", + result=_result(proof, manifest), + constation_infra_fault="constation_unavailable", + constation_attempt=1, + max_constation_attempts=3, + ) + assert "infra_fault" in ei.value.reason + + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-infra", + result=_result(proof, manifest), + constation_infra_fault="constation_unavailable", + constation_attempt=3, + max_constation_attempts=3, + ) + assert outcome.status == "rejected" + assert outcome.score_written is False + assert outcome.reason is not None and outcome.reason.startswith("infra_fault:") + assert _final_score(db_path, submission_id) is None + + +@pytest.mark.asyncio +async def test_revoked_digest_no_score(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app, hotkey="hk-rev") + manifest = _manifest("rev") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + + def allow(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=False, reason="revoked") + + def nonce(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def sig(_s: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-rev", + result=_result(proof, manifest), + constation_bundle=_constation_bundle(digest=DIGEST), + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, + ) + assert outcome.status == "rejected" + assert outcome.reason == "miner_fault:revoked_digest" + assert _final_score(db_path, submission_id) is None + + +# --- todo 23 ------------------------------------------------------------------------------------ + + +def test_breakglass_admits_infra_fault_only() -> None: + log = BreakGlassAuditLog() + req = BreakGlassRequest( + operator_id="ops-alice", + reason="constation outage window", + work_unit_id="wu-1", + fault_code="infra_fault:constation_unavailable", + ) + ok = evaluate_break_glass(req, fault_reason="infra_fault:constation_unavailable", audit_log=log) + assert ok.admitted is True + assert log.entries and log.entries[0]["admitted"] is True + assert log.entries[0]["operator_id"] == "ops-alice" + + log2 = BreakGlassAuditLog() + bad = evaluate_break_glass(req, fault_reason="miner_fault:replayed_nonce", audit_log=log2) + assert bad.admitted is False + assert bad.reason == "breakglass_refused_miner_fault" + assert log2.entries and log2.entries[0]["admitted"] is False + + +@pytest.mark.asyncio +async def test_breakglass_admits_infra_run_and_writes_score( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app, hotkey="hk-bg") + manifest = _manifest("bg") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + log = BreakGlassAuditLog() + bg = BreakGlassRequest( + operator_id="ops-bob", + reason="confirmed BASE outage", + work_unit_id=submission_id, + fault_code="infra_fault:constation_unavailable", + ) + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-bg", + result=_result(proof, manifest), + constation_infra_fault="constation_unavailable", + constation_attempt=3, + max_constation_attempts=3, + break_glass=bg, + break_glass_audit_log=log, + ) + assert outcome.status == "accepted" + assert outcome.break_glass_admitted is True + assert outcome.effective_tier == 0 # no elevation without real constation + assert outcome.score_written is True + assert _final_score(db_path, submission_id) is not None + assert any(e.get("admitted") for e in log.entries) + + +@pytest.mark.asyncio +async def test_breakglass_refuses_miner_fault( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + submission_id = await _seed(app, hotkey="hk-bgm") + manifest = _manifest("bgm") + proof = _tier1_proof(signer, submission_id, manifest, image_digest=DIGEST) + log = BreakGlassAuditLog() + bg = BreakGlassRequest( + operator_id="ops-eve", + reason="please admit anyway", + work_unit_id=submission_id, + fault_code="miner_fault:replayed_nonce", + ) + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-bgm", + result=_result(proof, manifest), + # missing bundle = miner_fault + constation_bundle=None, + break_glass=bg, + break_glass_audit_log=log, + ) + assert outcome.status == "rejected" + assert outcome.break_glass_admitted is False + assert outcome.score_written is False + assert _final_score(db_path, submission_id) is None + assert any(e.get("detail") == "miner_fault_override_refused" for e in log.entries) diff --git a/tests/test_doc_trust_contract.py b/tests/test_doc_trust_contract.py new file mode 100644 index 0000000..db3cfe2 --- /dev/null +++ b/tests/test_doc_trust_contract.py @@ -0,0 +1,102 @@ +"""Doc trust contract (todo 25): behavioural ceiling on effective_tier. + +Docs claim no TEE and max effective tier 1. This test exercises the real +``effective_tier`` function — not a docs grep — and fails if any route returns > 1. +""" + +from __future__ import annotations + +import pytest + +from prism_challenge.audit import effective_tier +from prism_challenge.proof import ExecutionProof, ProviderInfo, WorkerSignature + +PINNED = "sha256:" + ("ab" * 32) + + +def _proof(*, tier: int, image_digest: str | None = None) -> ExecutionProof: + if tier >= 1 and image_digest is None: + image_digest = PINNED + attestation = None + if tier >= 2: + attestation = {"tdx_quote_b64": "opaque", "gpu_eat_jwt": "opaque"} + return ExecutionProof( + version=1, + tier=tier, # type: ignore[arg-type] + manifest_sha256="c" * 64, + image_digest=image_digest, + provider=ProviderInfo(name="lium", pod_id="pod-doc-contract"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + attestation=attestation, + ) + + +@pytest.mark.parametrize( + "claimed,constation,pin", + [ + (0, None, None), + (0, True, PINNED), + (1, None, PINNED), + (1, False, PINNED), + (1, True, PINNED), + (1, True, "sha256:" + ("cd" * 32)), + (2, True, PINNED), + (2, False, PINNED), + (3, True, PINNED), + (99, True, PINNED), + ], +) +def test_effective_tier_never_exceeds_one( + claimed: int, + constation: bool | None, + pin: str | None, +) -> None: + """Behavioural contract: docs may claim tier ceiling 1 only if code enforces it.""" + proof = _proof(tier=min(claimed, 2) if claimed >= 2 else claimed) + # For claimed > 2, still construct a tier-2-shaped proof then override attribute if allowed. + object.__setattr__(proof, "tier", claimed) if hasattr(proof, "__dict__") else None + try: + # Pydantic models may freeze tier; rebuild when needed. + if int(proof.tier) != claimed: + proof = ExecutionProof( + version=1, + tier=min(claimed, 2), # type: ignore[arg-type] + manifest_sha256="c" * 64, + image_digest=PINNED, + provider=ProviderInfo(name="lium", pod_id="pod-doc-contract"), + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + attestation={"tdx_quote_b64": "x", "gpu_eat_jwt": "y"} if claimed >= 2 else None, + ) + # effective_tier reads getattr tier — pass via model_copy if available + if hasattr(proof, "model_copy"): + proof = proof.model_copy(update={"tier": claimed}) + except Exception: + proof = _proof(tier=2 if claimed >= 2 else claimed) + + got = effective_tier( + proof, + pinned_image_digest=pin, + constation_ok_result=constation, + ) + assert isinstance(got, int) + assert got <= 1, ( + f"effective_tier returned {got} > 1 " + f"(claimed={claimed}, constation={constation})" + ) + assert got >= 0 + + +def test_constation_true_claimed_1_is_at_most_one() -> None: + proof = _proof(tier=1) + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=True) == 1 + + +def test_claimed_tier_2_with_constation_still_zero_not_two() -> None: + proof = _proof(tier=2) + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=True) == 0 + + +def test_pin_match_alone_does_not_elevate() -> None: + proof = _proof(tier=1) + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=None) == 0 + assert effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=False) == 0 diff --git a/tests/test_execution_backend_constation_gate.py b/tests/test_execution_backend_constation_gate.py new file mode 100644 index 0000000..4b2cc10 --- /dev/null +++ b/tests/test_execution_backend_constation_gate.py @@ -0,0 +1,66 @@ +"""Lium execution backend is gated on a full constation bundle (todo 19). + +Renamed from the misleading test_lium_client.py (which tested no client). +""" + +from __future__ import annotations + +import pytest + +from prism_challenge.constation import ConstationBundle +from prism_challenge.queue import ( + LIUM_EXECUTION_BACKEND, + SUPPORTED_EXECUTION_BACKENDS, + is_execution_backend_supported, + require_execution_backend, +) + + +def _bundle() -> ConstationBundle: + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest="sha256:" + ("1" * 64), + work_unit_id="wu-1", + miner_hotkey="hk", + pod_id="pod-1", + nonce="n-1", + signed_attestation={"sig": "x"}, + expected_sealed_manifest_hashes={"h.py": "c" * 64}, + reported_sealed_manifest_hashes={"h.py": "c" * 64}, + lium_declared_digest="sha256:" + ("1" * 64), + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + +def test_base_gpu_always_supported_without_bundle() -> None: + assert "base_gpu" in SUPPORTED_EXECUTION_BACKENDS + assert is_execution_backend_supported("base_gpu") is True + assert is_execution_backend_supported("base_gpu", constation_bundle=None) is True + require_execution_backend("base_gpu") # no raise + + +def test_lium_without_bundle_rejected() -> None: + assert is_execution_backend_supported(LIUM_EXECUTION_BACKEND) is False + assert is_execution_backend_supported(LIUM_EXECUTION_BACKEND, constation_bundle=None) is False + with pytest.raises(ValueError, match="constation bundle required for lium"): + require_execution_backend(LIUM_EXECUTION_BACKEND) + with pytest.raises(ValueError, match="constation bundle required for lium"): + require_execution_backend("lium", constation_bundle=None) + + +def test_lium_with_full_bundle_accepted() -> None: + bundle = _bundle() + assert is_execution_backend_supported("lium", constation_bundle=bundle) is True + require_execution_backend("lium", constation_bundle=bundle) # no raise + + +def test_remote_provider_and_local_cpu_still_rejected() -> None: + assert "remote_provider" not in SUPPORTED_EXECUTION_BACKENDS + assert "local_cpu" not in SUPPORTED_EXECUTION_BACKENDS + with pytest.raises(ValueError, match="Unsupported execution backend"): + require_execution_backend("remote_provider") + with pytest.raises(ValueError, match="Unsupported execution backend"): + require_execution_backend("local_cpu", constation_bundle=_bundle()) diff --git a/tests/test_lium_client.py b/tests/test_lium_client.py deleted file mode 100644 index ec6af12..0000000 --- a/tests/test_lium_client.py +++ /dev/null @@ -1,9 +0,0 @@ -from __future__ import annotations - -from prism_challenge.queue import SUPPORTED_EXECUTION_BACKENDS - - -def test_lium_backends_are_not_supported() -> None: - assert "remote_provider" not in SUPPORTED_EXECUTION_BACKENDS - assert "local_cpu" not in SUPPORTED_EXECUTION_BACKENDS - assert "base_gpu" in SUPPORTED_EXECUTION_BACKENDS diff --git a/tests/test_prism_attestation_routes_s8.py b/tests/test_prism_attestation_routes_s8.py new file mode 100644 index 0000000..ae27119 --- /dev/null +++ b/tests/test_prism_attestation_routes_s8.py @@ -0,0 +1,122 @@ +"""S8: Prism public attestation challenge/answer roundtrip (proxy product path).""" + +from __future__ import annotations + +from pathlib import Path + +from fastapi.testclient import TestClient + +from prism_challenge.app import create_app +from prism_challenge.config import PrismSettings, WorkerPlaneConfig + +WORKER_KEY = "//WorkerAttestS8" +DIGEST = "sha256:" + ("11" * 32) +COMMIT = "a" * 40 +TREE = "b" * 40 + + +def _settings(tmp_path: Path) -> PrismSettings: + return PrismSettings( + database_url=f"sqlite+aiosqlite:///{tmp_path / 's8.sqlite3'}", + shared_token="secret", + allow_insecure_signatures=False, + execution_backend="base_gpu", + docker_enabled=True, + docker_backend="broker", + docker_broker_url="http://base-docker-broker:8082", + docker_broker_token="secret", + sequence_length=16, + plagiarism_enabled=False, + distributed_contract_policy="off", + base_eval_artifact_root=tmp_path / "artifacts", + worker_plane=WorkerPlaneConfig(enabled=True, signing_key=WORKER_KEY), + # No constation_base_url → in-process SoT on prism app.state + ) + + +def test_s8_challenge_answer_roundtrip_on_prism(tmp_path: Path) -> None: + settings = _settings(tmp_path) + with TestClient(create_app(settings)) as client: + # Public surface exactly as published through BASE proxy under + # /challenges/prism/v1/attestation/* (challenge app owns the path). + ch = client.get( + "/v1/attestation/challenge", + params={ + "phase": "start", + "work_unit_id": "wu-s8", + "miner_hotkey": "hk-s8", + "pod_id": "pod-s8", + }, + ) + assert ch.status_code == 200, ch.text + body = ch.json() + assert body["nonce"] + assert body["phase"] == "start" + assert body["work_unit_id"] == "wu-s8" + assert body["challenge_id"] == body["nonce"] + + ans = client.post( + "/v1/attestation/answer", + json={"nonce": body["nonce"], "phase": "start"}, + ) + assert ans.status_code == 200, ans.text + assert ans.json()["status"] == "accepted" + + +def test_s8_inprocess_register_check_and_nonce_consume(tmp_path: Path) -> None: + settings = _settings(tmp_path) + headers = {"Authorization": "Bearer secret"} + with TestClient(create_app(settings)) as client: + reg = client.post( + "/internal/v1/constation/register_digest", + headers=headers, + json={ + "commit_sha": COMMIT, + "tree_sha": TREE, + "variant": "cuda", + "digest": DIGEST, + }, + ) + assert reg.status_code == 200, reg.text + + hit = client.post( + "/internal/v1/constation/check_allowlist", + headers=headers, + json={ + "digest": DIGEST, + "commit_sha": COMMIT, + "tree_sha": TREE, + "variant": "cuda", + }, + ) + assert hit.status_code == 200 + assert hit.json() == {"ok": True, "reason": "ok"} + + ch = client.get( + "/v1/attestation/challenge", + params={ + "phase": "interval", + "work_unit_id": "wu-s8b", + "miner_hotkey": "hk-s8b", + "pod_id": "pod-s8b", + }, + ) + nonce = ch.json()["nonce"] + body = { + "nonce": nonce, + "work_unit_id": "wu-s8b", + "miner_hotkey": "hk-s8b", + "pod_id": "pod-s8b", + } + first = client.post("/internal/v1/constation/check_nonce", headers=headers, json=body) + second = client.post("/internal/v1/constation/check_nonce", headers=headers, json=body) + assert first.json() == {"ok": True, "reason": "ok"} + assert second.json()["ok"] is False + assert second.json()["reason"] == "already_consumed" + + +def test_s8_missing_binding_query_422(tmp_path: Path) -> None: + settings = _settings(tmp_path) + with TestClient(create_app(settings)) as client: + r = client.get("/v1/attestation/challenge", params={"phase": "start"}) + assert r.status_code == 422 diff --git a/tests/test_prism_audit_effective_tier.py b/tests/test_prism_audit_effective_tier.py index 7a80673..b5cce64 100644 --- a/tests/test_prism_audit_effective_tier.py +++ b/tests/test_prism_audit_effective_tier.py @@ -66,19 +66,33 @@ def test_tier2_claim_downgrades_without_verified_attestation() -> None: assert is_tier_downgraded(with_digest, pinned_image_digest=PINNED) is True -def test_tier1_claim_requires_matching_pinned_digest() -> None: +def test_tier1_claim_requires_constation_ok() -> None: matching = _proof(tier=1, image_digest=PINNED) mismatched = _proof(tier=1, image_digest=OTHER) - # Schema forbids tier-1 without image_digest; uncovered digest=empty via mismatched. - emptyish = _proof(tier=1, image_digest=OTHER) - # Tier 1 also requires provider pod binding (workload identity). - assert effective_tier(matching, pinned_image_digest=PINNED) == 1 - assert is_tier_downgraded(matching, pinned_image_digest=PINNED) is False + # Pin match alone never elevates (todo 21 / M14). + assert effective_tier(matching, pinned_image_digest=PINNED) == 0 assert effective_tier(mismatched, pinned_image_digest=PINNED) == 0 - assert effective_tier(emptyish, pinned_image_digest=PINNED) == 0 - # With no pinned digest configured, no tier-1 claim is verifiable. - assert effective_tier(matching, pinned_image_digest=None) == 0 + # constation_ok is the sole elevation predicate. + assert ( + effective_tier( + matching, + pinned_image_digest=PINNED, + constation_ok_result=True, + ) + == 1 + ) + assert ( + is_tier_downgraded( + matching, + pinned_image_digest=PINNED, + constation_ok_result=True, + ) + is False + ) + assert effective_tier(matching, constation_ok_result=False) == 0 + # Digest mismatch is irrelevant when constation_ok is True (digest already gated upstream). + assert effective_tier(mismatched, constation_ok_result=True) == 1 def test_tier0_claim_stays_tier0() -> None: @@ -89,10 +103,19 @@ def test_tier0_claim_stays_tier0() -> None: # --- Sampling follows the EFFECTIVE tier (VAL-PRISM-019 statistical) ----------------------------- -def _sampled_fraction(sampler: AuditSampler, proof: ExecutionProof, n: int) -> float: +def _sampled_fraction( + sampler: AuditSampler, + proof: ExecutionProof, + n: int, + *, + constation_ok_result: bool | None = None, +) -> float: hits = sum( sampler.decide( - work_unit_id=f"{proof.tier}-{i}", proof=proof, pinned_image_digest=PINNED + work_unit_id=f"{proof.tier}-{i}", + proof=proof, + pinned_image_digest=PINNED, + constation_ok_result=constation_ok_result, ).sampled for i in range(n) ) @@ -117,8 +140,14 @@ def _bound(p: float) -> float: # Unverified tier-2 claims are sampled at tier-0 rate (fail-closed TEE). assert abs(_sampled_fraction(sampler, opaque_t2, n) - 0.10) < _bound(0.10) - # Honest tier-1 claims are sampled at their tier-1 rate. - assert abs(_sampled_fraction(sampler, honest_t1, n) - 0.05) < _bound(0.05) + # Honest tier-1 + constation_ok are sampled at their tier-1 rate. + assert abs(_sampled_fraction(sampler, honest_t1, n, constation_ok_result=True) - 0.05) < _bound( + 0.05 + ) + # Without constation_ok, tier-1 claims are effective 0. + assert abs( + _sampled_fraction(sampler, honest_t1, n, constation_ok_result=False) - 0.10 + ) < _bound(0.10) # Unverifiable claims are sampled at the EFFECTIVE (tier-0) rate, NOT the claimed rate. assert abs(_sampled_fraction(sampler, fake_t2, n) - 0.10) < _bound(0.10) assert abs(_sampled_fraction(sampler, fake_t1, n) - 0.10) < _bound(0.10) diff --git a/tests/test_prism_no_tee_absence.py b/tests/test_prism_no_tee_absence.py index c317931..36c89b0 100644 --- a/tests/test_prism_no_tee_absence.py +++ b/tests/test_prism_no_tee_absence.py @@ -1,4 +1,8 @@ -"""Prism NO TEE residual: package absence + score finalize without tee (VAL-NOTEE-001..008).""" +"""Prism NO TEE residual: package absence + score finalize without tee (VAL-NOTEE-001..008). + +Extended by todo 24: image-attestation path, attestation_mode never TEE, tier cannot exceed 1. +Never delete, skip, or xfail this file. +""" from __future__ import annotations @@ -16,22 +20,28 @@ from prism_challenge.app import create_app from prism_challenge.audit import effective_tier from prism_challenge.config import PrismSettings, WorkerPlaneConfig +from prism_challenge.constation import CheckOutcome, ConstationBundle from prism_challenge.ingestion import ResultIngestionError, ingest_work_unit_result from prism_challenge.models import SubmissionCreate from prism_challenge.proof import ( + ATTESTATION_MODE_V1, MANIFEST_PAYLOAD_KEY, PROOF_PAYLOAD_KEY, ExecutionProof, ProviderInfo, WorkerSignature, + attach_attestation_mode, + attestation_mode_of, build_execution_proof, compute_manifest_sha256, + normalize_attestation_mode, worker_signer_from_key, ) WORKER_KEY = "//WorkerNoTee" PINNED = "sha256:" + ("ab" * 32) OTHER = "sha256:" + ("cd" * 32) +DIGEST = "sha256:" + ("11" * 32) TINY_ARCH = """ import torch @@ -94,8 +104,6 @@ def test_config_has_no_tee_block_or_capability() -> None: docker_backend="cli", database_url="sqlite+aiosqlite:////tmp/prism-notee-cfg.sqlite3", ) - # Nested TeeConfig / settings.tee / PRISM_TEE gone; capability not advertised. - # Base ChallengeSettings may still expose an inert tee_verification_enabled flag. assert not hasattr(settings, "tee") assert "challenge.tee_verification" not in settings.capabilities config_mod = __import__("prism_challenge.config", fromlist=["*"]) @@ -127,12 +135,99 @@ def test_max_effective_tier_is_one_never_two() -> None: provider=ProviderInfo(name="lium", pod_id="pod-1"), worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), ) - assert effective_tier(proof_t2, pinned_image_digest=PINNED) == 0 - assert effective_tier(proof_t1, pinned_image_digest=PINNED) == 1 - assert effective_tier(proof_t1, pinned_image_digest=OTHER) == 0 + # Claimed tier 2 never elevates, even with constation_ok. + assert effective_tier(proof_t2, pinned_image_digest=PINNED, constation_ok_result=True) == 0 + # Tier 1 requires constation_ok (not pin match alone). + assert effective_tier(proof_t1, pinned_image_digest=PINNED) == 0 + assert effective_tier(proof_t1, pinned_image_digest=PINNED, constation_ok_result=True) == 1 + assert effective_tier(proof_t1, pinned_image_digest=OTHER, constation_ok_result=False) == 0 + + +def test_attestation_mode_never_implies_tee() -> None: + """Todo 24: attestation_mode is miner_rent_image_pin_evidence_v1; TEE labels forbidden.""" + assert ATTESTATION_MODE_V1 == "miner_rent_image_pin_evidence_v1" + assert "tee" not in ATTESTATION_MODE_V1.lower() + assert "tdx" not in ATTESTATION_MODE_V1.lower() + assert normalize_attestation_mode(ATTESTATION_MODE_V1) == ATTESTATION_MODE_V1 + for bad in ("lium_attested", "tee", "tee_attested", "tdx", "sev", "cvm"): + with pytest.raises(ValueError): + normalize_attestation_mode(bad) + att = attach_attestation_mode({"tdx_quote_b64": "x", "gpu_eat_jwt": "y"}) + assert att["attestation_mode"] == ATTESTATION_MODE_V1 + + +def test_image_attestation_path_exists_and_functions() -> None: + """Todo 24: image-attestation path (constation_ok + attestation_mode) is present.""" + from prism_challenge.constation import constation_ok + from prism_challenge.queue import LIUM_EXECUTION_BACKEND, is_execution_backend_supported + + man = {"h.py": "a" * 64} + bundle = ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=DIGEST, + work_unit_id="wu", + miner_hotkey="hk", + pod_id="pod-1", + nonce="n", + signed_attestation={"s": "1"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=DIGEST, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + def ok(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + result = constation_ok( + bundle, check_allowlist=ok, check_nonce=ok, verify_signature=lambda _s: ok() + ) + assert result.ok is True + assert is_execution_backend_supported(LIUM_EXECUTION_BACKEND, constation_bundle=bundle) + signer = worker_signer_from_key(WORKER_KEY) + proof = build_execution_proof( + signer=signer, + manifest_sha256="c" * 64, + unit_id="u", + image_digest=DIGEST, + constation_digest=DIGEST, + provider=ProviderInfo(name="lium", pod_id="pod-1"), + tier=1, # type: ignore[arg-type] + ) + assert attestation_mode_of(proof) == ATTESTATION_MODE_V1 + assert effective_tier(proof, constation_ok_result=result) == 1 + + +def test_effective_tier_cannot_exceed_one_by_any_route() -> None: + """Todo 24: no route yields effective tier > 1.""" + for claimed in (0, 1, 2): + proof = ExecutionProof( + version=1, + tier=claimed, # type: ignore[arg-type] + manifest_sha256="c" * 64, + image_digest=PINNED if claimed >= 1 else None, + provider=ProviderInfo(name="lium", pod_id="pod-1") if claimed >= 1 else None, + worker_signature=WorkerSignature(worker_pubkey="wk", sig="0xab"), + attestation={ + "version": 1, + "provider": "local_fixture", + "evidence_type": "prism.tee.v1", + "tdx_quote_b64": "Q", + "gpu_eat_jwt": "J", + "attestation_mode": ATTESTATION_MODE_V1, + } + if claimed == 2 + else attach_attestation_mode(None), + ) + for cok in (True, False, None): + tier = effective_tier(proof, pinned_image_digest=PINNED, constation_ok_result=cok) + assert tier <= 1, f"claimed={claimed} cok={cok} -> {tier}" -def _bundle() -> str: +def _code_bundle() -> str: stream = io.BytesIO() with zipfile.ZipFile(stream, "w") as archive: archive.writestr("architecture.py", TINY_ARCH) @@ -203,6 +298,7 @@ def _proof_payload( unit_id=unit_id, provider=ProviderInfo(name="lium", pod_id="pod-1"), image_digest=image_digest, + constation_digest=image_digest, attestation=attestation, tier=tier, # type: ignore[arg-type] ) @@ -219,6 +315,39 @@ def _result(proof_dict: dict[str, Any], manifest: dict[str, Any]) -> dict[str, A } +def _ok_checkers(): + def allow(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def nonce(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def sig(_s: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + return allow, nonce, sig + + +def _constation_bundle(digest: str = DIGEST) -> ConstationBundle: + man = {"h.py": "a" * 64} + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="wu", + miner_hotkey="hk", + pod_id="pod-1", + nonce="n", + signed_attestation={"s": "1"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + async def _make_app(settings: PrismSettings): app = create_app(settings) await app.state.database.init() @@ -227,7 +356,7 @@ async def _make_app(settings: PrismSettings): async def _seed(app, hotkey: str = "hk-notee") -> str: sub = await app.state.repository.create_submission( - hotkey, SubmissionCreate(code=_bundle(), filename="project.zip") + hotkey, SubmissionCreate(code=_code_bundle(), filename="project.zip") ) return sub.id @@ -245,19 +374,24 @@ def _score(db_path: Path, submission_id: str): @pytest.mark.asyncio async def test_score_finalize_works_without_tee_package(tmp_path: Path) -> None: - """Worker-plane finalize succeeds with no tee package / no tee_required (VAL-NOTEE-003).""" + """Worker-plane finalize succeeds with no tee package; requires constation (P1).""" settings = _settings(tmp_path) app = await _make_app(settings) signer = worker_signer_from_key(WORKER_KEY) submission_id = await _seed(app) manifest = _manifest() - proof = _proof_payload(signer, submission_id, manifest, tier=1, image_digest=PINNED) + proof = _proof_payload(signer, submission_id, manifest, tier=1, image_digest=DIGEST) + allow, nonce, sig = _ok_checkers() outcome = await ingest_work_unit_result( worker=app.state.worker, work_unit_id=submission_id, submission_ref="hk-notee", result=_result(proof, manifest), - pinned_image_digest=PINNED, + pinned_image_digest=DIGEST, + constation_bundle=_constation_bundle(DIGEST), + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, ) assert outcome.status == "accepted" assert outcome.finalized is True @@ -265,11 +399,13 @@ async def test_score_finalize_works_without_tee_package(tmp_path: Path) -> None: assert outcome.claimed_tier == 1 assert outcome.tier_downgraded is False assert outcome.reason is None + assert outcome.attestation_mode == ATTESTATION_MODE_V1 assert _score(tmp_path / "notee.sqlite3", submission_id) is not None @pytest.mark.asyncio -async def test_pin_mismatch_downgrades_but_still_finalizes(tmp_path: Path) -> None: +async def test_pin_mismatch_without_constation_writes_no_score(tmp_path: Path) -> None: + """NEW contract (todo 22): missing constation ⇒ no score (was: still finalizes).""" settings = _settings(tmp_path) app = await _make_app(settings) signer = worker_signer_from_key(WORKER_KEY) @@ -282,18 +418,18 @@ async def test_pin_mismatch_downgrades_but_still_finalizes(tmp_path: Path) -> No submission_ref="hk-mismatch", result=_result(proof, manifest), pinned_image_digest=PINNED, + constation_bundle=None, ) - assert outcome.status == "accepted" - assert outcome.finalized is True - assert outcome.claimed_tier == 1 + assert outcome.status == "rejected" + assert outcome.finalized is False + assert outcome.score_written is False assert outcome.effective_tier == 0 - assert outcome.tier_downgraded is True - assert _score(tmp_path / "notee.sqlite3", submission_id) is not None + assert _score(tmp_path / "notee.sqlite3", submission_id) is None @pytest.mark.asyncio async def test_ingestion_never_raises_tee_required(tmp_path: Path) -> None: - """Attestation-claiming tier-2 proof finalizes without tee_required (max effective=0).""" + """Attestation-claiming tier-2 proof never raises tee_required (max effective=0).""" settings = _settings(tmp_path) app = await _make_app(settings) signer = worker_signer_from_key(WORKER_KEY) @@ -311,21 +447,26 @@ async def test_ingestion_never_raises_tee_required(tmp_path: Path) -> None: submission_id, manifest, tier=2, - image_digest=PINNED, + image_digest=DIGEST, attestation=attestation, ) + allow, nonce, sig = _ok_checkers() try: outcome = await ingest_work_unit_result( worker=app.state.worker, work_unit_id=submission_id, submission_ref="hk-t2", result=_result(proof, manifest), - pinned_image_digest=PINNED, + pinned_image_digest=DIGEST, + constation_bundle=_constation_bundle(DIGEST), + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, ) except ResultIngestionError as exc: assert exc.reason != "tee_required" raise - assert outcome.finalized is True + # With constation, score may write but effective tier stays 0 for claimed tier 2. assert outcome.effective_tier == 0 assert outcome.tier_downgraded is True - assert _score(tmp_path / "notee.sqlite3", submission_id) is not None + assert outcome.attestation_mode == ATTESTATION_MODE_V1 diff --git a/tests/test_prism_proof.py b/tests/test_prism_proof.py index 57b0642..6a6dd8a 100644 --- a/tests/test_prism_proof.py +++ b/tests/test_prism_proof.py @@ -107,7 +107,8 @@ def test_tier0_when_no_provider_metadata() -> None: assert p.tier == 0 assert p.image_digest is None assert p.provider is None - assert p.attestation is None + # Every proof carries honest attestation_mode (constation wire); no TEE payload. + assert p.attestation == {"attestation_mode": proof.ATTESTATION_MODE_V1} assert p.manifest_sha256 == digest @@ -164,7 +165,7 @@ def test_opaque_nonempty_attestation_does_not_claim_tier2() -> None: ) assert p.tier != 2 assert p.tier == 1 - assert p.attestation == attestation + assert p.attestation == {**attestation, "attestation_mode": proof.ATTESTATION_MODE_V1} def test_structured_attestation_claims_tier2_but_is_unverified() -> None: @@ -182,7 +183,7 @@ def test_structured_attestation_claims_tier2_but_is_unverified() -> None: ) # Emission may claim tier 2 for wire compat; effective elevation never exceeds IMAGE_PIN tier-1. assert p.tier == 2 - assert p.attestation == attestation + assert p.attestation == {**attestation, "attestation_mode": proof.ATTESTATION_MODE_V1} def test_tier_never_2_without_attestation() -> None: @@ -190,7 +191,7 @@ def test_tier_never_2_without_attestation() -> None: p = build_execution_proof_from_manifest( signer=signer, unit_id=UNIT_ID, manifest=MANIFEST_A, env=dict(_FULL_ENV) ) - assert p.attestation is None + assert p.attestation == {"attestation_mode": proof.ATTESTATION_MODE_V1} assert p.tier != 2 diff --git a/tests/test_prism_result_ingestion.py b/tests/test_prism_result_ingestion.py index c2b1b6f..ddf27a8 100644 --- a/tests/test_prism_result_ingestion.py +++ b/tests/test_prism_result_ingestion.py @@ -440,6 +440,7 @@ def _eval_job_count(db_path: Path, submission_id: str) -> int: async def test_ingestion_records_downgraded_effective_tier(tmp_path, monkeypatch) -> None: + """Claimed tier 2 never elevates; with valid constation, score writes at effective 0.""" data_dir = _stage_train(tmp_path) monkeypatch.setattr( "prism_challenge.evaluator.container.DockerExecutor.run", @@ -454,24 +455,31 @@ async def test_ingestion_records_downgraded_effective_tier(tmp_path, monkeypatch submission_id = await _seed(app) manifest = _manifest() - # A tier-1 claim whose image_digest does not match the pinned digest is - # unverifiable under SDK-strict proof shape -> effective tier 0. + # Claimed tier 2 (structured attestation shape) collapses to effective 0 even when + # constation_ok is True (auto-injected by conftest for legacy callers). proof = _proof_dict( signer, submission_id, manifest, - tier=1, + tier=2, image_digest="sha256:" + ("ab" * 32), - attestation=None, + attestation={ + "version": 1, + "provider": "local_fixture", + "evidence_type": "prism.tee.v1", + "tdx_quote_b64": "QUOTE", + "gpu_eat_jwt": "JWT", + }, ) outcome = await ingest_work_unit_result( worker=app.state.worker, work_unit_id=submission_id, submission_ref="hk-owner", result=_result(proof, manifest), - pinned_image_digest="sha256:" + ("cd" * 32), + pinned_image_digest="sha256:" + ("ab" * 32), ) - assert outcome.claimed_tier == 1 + assert outcome.status == "accepted" + assert outcome.claimed_tier == 2 assert outcome.effective_tier == 0 assert outcome.tier_downgraded is True @@ -484,7 +492,7 @@ async def test_ingestion_records_downgraded_effective_tier(tmp_path, monkeypatch ).fetchone() finally: conn.close() - assert row == (1, 0, 1) + assert row == (2, 0, 1) # --- HTTP route body contract + status codes (VAL-PRISM-017/018) --------------------------------- diff --git a/tests/test_prism_scoring_characterization_baseline.py b/tests/test_prism_scoring_characterization_baseline.py new file mode 100644 index 0000000..4261b4e --- /dev/null +++ b/tests/test_prism_scoring_characterization_baseline.py @@ -0,0 +1,295 @@ +"""Post-fail-closed scoring contract (todo 18 baseline rewritten by todo 22). + +Todo 18 pinned PRE-change reality (pin mismatch still scored). Todo 22 deliberately +broke that contract (P1: no valid constation bundle ⇒ no score row). This file now +documents the NEW contract — tests are rewritten, not deleted. +""" + +from __future__ import annotations + +import base64 +import io +import math +import sqlite3 +import zipfile +from pathlib import Path +from typing import Any + +import pytest + +from prism_challenge.app import create_app +from prism_challenge.audit import effective_tier +from prism_challenge.config import PrismSettings, WorkerPlaneConfig +from prism_challenge.constation import CheckOutcome, ConstationBundle +from prism_challenge.evaluator.mock_reexec import cpu_reexec_run +from prism_challenge.ingestion import ingest_work_unit_result +from prism_challenge.models import SubmissionCreate +from prism_challenge.proof import ( + MANIFEST_PAYLOAD_KEY, + PROOF_PAYLOAD_KEY, + ProviderInfo, + build_execution_proof, + compute_manifest_sha256, + worker_signer_from_key, +) + +WORKER_KEY = "//WorkerCharBaseline" +PINNED = "sha256:" + ("aa" * 32) +OTHER = "sha256:" + ("bb" * 32) +DIGEST = "sha256:" + ("11" * 32) + +TINY_ARCH = """ +import torch +from torch import nn + + +class TinyLM(nn.Module): + def __init__(self, vocab): + super().__init__() + self.emb = nn.Embedding(vocab, 8) + self.head = nn.Linear(8, vocab) + + def forward(self, tokens): + return self.head(self.emb(tokens)) + + +def build_model(ctx): + return TinyLM(ctx.vocab_size) +""" + +TINY_TRAIN = """ +import torch +import torch.nn.functional as F + + +def train(ctx): + model = ctx.build_model() + opt = torch.optim.AdamW(model.parameters(), lr=0.01) + for batch in ctx.iter_train_batches(model, batch_size=1): + opt.zero_grad() + logits = model(batch.tokens) + nv = logits.shape[-1] + loss = F.cross_entropy( + logits[:, :-1, :].reshape(-1, nv), batch.tokens[:, 1:].reshape(-1) % nv + ) + loss.backward() + opt.step() +""" + +_SHARD_LINE = ( + '{{"id": "doc-{i}", "text": "the locked fineweb edu training sample number {i} ' + 'has enough bytes to cover several challenge instrument batches deterministically"}}\n' +) + + +def _stage_train(root: Path, *, lines: int = 64) -> Path: + data_dir = root / "train-data" + data_dir.mkdir(parents=True, exist_ok=True) + (data_dir / "train-00000.jsonl").write_text( + "".join(_SHARD_LINE.format(i=i) for i in range(lines)), encoding="utf-8" + ) + return data_dir + + +def _bundle() -> str: + stream = io.BytesIO() + with zipfile.ZipFile(stream, "w") as archive: + archive.writestr("architecture.py", TINY_ARCH) + archive.writestr("training.py", TINY_TRAIN) + return base64.b64encode(stream.getvalue()).decode("ascii") + + +def _settings(tmp_path: Path) -> PrismSettings: + return PrismSettings( + database_url=f"sqlite+aiosqlite:///{tmp_path / 'coord.sqlite3'}", + shared_token="secret", + allow_insecure_signatures=True, + execution_backend="base_gpu", + docker_enabled=True, + docker_backend="broker", + docker_broker_url="http://base-docker-broker:8082", + docker_broker_token="secret", + sequence_length=16, + plagiarism_enabled=False, + distributed_contract_policy="off", + base_eval_artifact_root=tmp_path / "artifacts", + worker_plane=WorkerPlaneConfig(enabled=True, signing_key=WORKER_KEY), + ) + + +def _manifest(marker: str = "v2") -> dict[str, Any]: + covered_bytes = 4096 + online_loss = [10.0, 6.0, 3.0, 2.0] + return { + "schema_version": "prism_run_manifest.v2", + "data": {"covered_bytes": covered_bytes, "single_pass": True}, + "metrics": { + "online_loss": online_loss, + "sum_neg_log_likelihood_nats": 900.0, + "covered_bytes": covered_bytes, + "predicted_tokens": 96, + "step0_loss": online_loss[0], + "consumed_batches": len(online_loss), + "random_init_baseline_nats": math.log(50257), + "prequential_bpb": 1.23, + "marker": marker, + }, + "anti_cheat": { + "step0_anomaly": False, + "nan_inf_detected": False, + "no_learning": False, + "zero_forward": False, + }, + } + + +def _tier1_proof_dict(signer, unit_id: str, manifest: dict[str, Any], *, image_digest: str): + digest = compute_manifest_sha256(manifest) + proof = build_execution_proof( + signer=signer, + manifest_sha256=digest, + unit_id=unit_id, + image_digest=image_digest, + constation_digest=image_digest, + provider=ProviderInfo(name="lium", pod_id="pod-char-1"), + tier=1, # type: ignore[arg-type] + ) + return proof.model_dump(mode="json") + + +def _result(proof_dict: dict[str, Any], manifest: dict[str, Any]) -> dict[str, Any]: + return { + "executed": 1, + "completed_submissions": [], + PROOF_PAYLOAD_KEY: proof_dict, + MANIFEST_PAYLOAD_KEY: manifest, + } + + +def _constation_bundle(digest: str = DIGEST) -> ConstationBundle: + man = {"h.py": "a" * 64} + return ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="wu", + miner_hotkey="hk", + pod_id="pod-char-1", + nonce="n", + signed_attestation={"s": "1"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + + +def _ok_checkers(): + def allow(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def nonce(**_k: Any) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + def sig(_s: object) -> CheckOutcome: + return CheckOutcome(ok=True, reason="ok") + + return allow, nonce, sig + + +async def _make_app(settings: PrismSettings): + app = create_app(settings) + await app.state.database.init() + return app + + +async def _seed(app, hotkey: str = "hk-owner") -> str: + sub = await app.state.repository.create_submission( + hotkey, SubmissionCreate(code=_bundle(), filename="project.zip") + ) + return sub.id + + +def _final_score(db_path: Path, submission_id: str) -> float | None: + conn = sqlite3.connect(db_path) + try: + row = conn.execute( + "SELECT final_score FROM scores WHERE submission_id=?", (submission_id,) + ).fetchone() + finally: + conn.close() + return None if row is None else float(row[0]) + + +async def test_pin_mismatch_without_constation_writes_no_final_score( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """NEW (todo 22): IMAGE_PIN mismatch / missing constation ⇒ no score row (P1).""" + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + + submission_id = await _seed(app) + manifest = _manifest("pin-mismatch") + proof = _tier1_proof_dict(signer, submission_id, manifest, image_digest=OTHER) + + from prism_challenge.proof import ExecutionProof + + assert effective_tier(ExecutionProof.model_validate(proof), pinned_image_digest=PINNED) == 0 + + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-owner", + result=_result(proof, manifest), + pinned_image_digest=PINNED, + constation_bundle=None, + ) + + assert outcome.status == "rejected" + assert outcome.finalized is False + assert outcome.score_written is False + assert outcome.reason == "miner_fault:missing_constation_bundle" + assert _final_score(db_path, submission_id) is None + + +async def test_constation_ok_writes_score_and_sets_effective_tier( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """NEW: valid constation bundle ⇒ score written; effective_tier follows constation_ok.""" + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + app = await _make_app(_settings(tmp_path)) + signer = worker_signer_from_key(WORKER_KEY) + db_path = tmp_path / "coord.sqlite3" + allow, nonce, sig = _ok_checkers() + + submission_id = await _seed(app, hotkey="hk-ok") + manifest = _manifest("constation-ok") + proof = _tier1_proof_dict(signer, submission_id, manifest, image_digest=DIGEST) + outcome = await ingest_work_unit_result( + worker=app.state.worker, + work_unit_id=submission_id, + submission_ref="hk-ok", + result=_result(proof, manifest), + pinned_image_digest=DIGEST, + constation_bundle=_constation_bundle(DIGEST), + check_allowlist=allow, + check_nonce=nonce, + verify_constation_signature=sig, + ) + assert outcome.status == "accepted" + assert outcome.effective_tier == 1 + assert outcome.finalized is True + score = _final_score(db_path, submission_id) + assert score is not None and score > 0.0 diff --git a/tests/test_prod_constation_http_ingest.py b/tests/test_prod_constation_http_ingest.py new file mode 100644 index 0000000..fc4d633 --- /dev/null +++ b/tests/test_prod_constation_http_ingest.py @@ -0,0 +1,287 @@ +"""Production HTTP constation path (S1/S3) — no legacy auto-inject (module name gate).""" + +from __future__ import annotations + +import io +import sqlite3 +import zipfile +from pathlib import Path +from typing import Any + +import httpx +import pytest +from fastapi.testclient import TestClient + +from prism_challenge.app import create_app +from prism_challenge.config import PrismSettings, WorkerPlaneConfig +from prism_challenge.constation import ConstationBundle, constation_bundle_to_dict +from prism_challenge.evaluator.mock_reexec import cpu_reexec_run +from prism_challenge.proof import ( + MANIFEST_PAYLOAD_KEY, + PROOF_PAYLOAD_KEY, + build_execution_proof, + compute_manifest_sha256, + worker_signer_from_key, +) + +WORKER_KEY = "//WorkerProdConstation" +DIGEST = "sha256:" + ("11" * 32) + +TINY_ARCH = """ +import torch +from torch import nn + + +class TinyLM(nn.Module): + def __init__(self, vocab): + super().__init__() + self.emb = nn.Embedding(vocab, 8) + self.head = nn.Linear(8, vocab) + + def forward(self, tokens): + return self.head(self.emb(tokens)) + + +def build_model(ctx): + return TinyLM(ctx.vocab_size) +""" + +TINY_TRAIN = """ +import torch +import torch.nn.functional as F + + +def train(ctx): + model = ctx.build_model() + opt = torch.optim.AdamW(model.parameters(), lr=0.01) + for batch in ctx.iter_train_batches(model, batch_size=1): + opt.zero_grad() + logits = model(batch.tokens) + nv = logits.shape[-1] + loss = F.cross_entropy( + logits[:, :-1, :].reshape(-1, nv), batch.tokens[:, 1:].reshape(-1) % nv + ) + loss.backward() + opt.step() +""" + +_SHARD = ( + '{{"id": "doc-{i}", "text": "the locked fineweb edu training sample number {i} ' + 'has enough bytes to cover several challenge instrument batches deterministically"}}\n' +) + + +def _stage_train(root: Path) -> Path: + data_dir = root / "train-data" + data_dir.mkdir(parents=True, exist_ok=True) + (data_dir / "train-00000.jsonl").write_text( + "".join(_SHARD.format(i=i) for i in range(64)), encoding="utf-8" + ) + return data_dir + + +def _zip_b64() -> bytes: + stream = io.BytesIO() + with zipfile.ZipFile(stream, "w") as archive: + archive.writestr("architecture.py", TINY_ARCH) + archive.writestr("training.py", TINY_TRAIN) + return stream.getvalue() + + +def _settings(tmp_path: Path, **extra: Any) -> PrismSettings: + kw: dict[str, Any] = dict( + database_url=f"sqlite+aiosqlite:///{tmp_path / 'coord.sqlite3'}", + shared_token="secret", + allow_insecure_signatures=False, + execution_backend="base_gpu", + docker_enabled=True, + docker_backend="broker", + docker_broker_url="http://base-docker-broker:8082", + docker_broker_token="secret", + sequence_length=16, + plagiarism_enabled=False, + distributed_contract_policy="off", + base_eval_artifact_root=tmp_path / "artifacts", + worker_plane=WorkerPlaneConfig(enabled=True, signing_key=WORKER_KEY), + constation_base_url="http://base-constation.test", + constation_internal_token="constation-tok", + ) + kw.update(extra) + return PrismSettings(**kw) + + +def _manifest() -> dict[str, Any]: + return { + "schema_version": "prism_run_manifest.v2", + "metrics": { + "token_accuracy": 0.5, + "loss": 1.0, + "step": 1, + }, + "timing": {"wall_seconds": 1.0}, + } + + +def _bundle_for(submission_id: str) -> dict[str, Any]: + man = {"route-test-harness.py": "a" * 64} + return constation_bundle_to_dict( + ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=DIGEST, + work_unit_id=submission_id, + miner_hotkey="hk-owner", + pod_id="pod-1", + nonce="nonce-prod-1", + signed_attestation={"sig": "fixture"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=DIGEST, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + ) + + +def _ok_transport() -> httpx.MockTransport: + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(200, json={"ok": True, "reason": "ok"}) + + return httpx.MockTransport(handler) + + +@pytest.fixture(autouse=True) +def _patch_http_checkers(monkeypatch: pytest.MonkeyPatch) -> None: + """Route BaseHttpConstationClient to always-ok MockTransport (S1 checkers).""" + from prism_challenge import constation_checkers as mod + + orig = mod.BaseHttpConstationClient.__init__ + + def _init(self, *args, **kwargs): # type: ignore[no-untyped-def] + kwargs = dict(kwargs) + kwargs.setdefault("transport", _ok_transport()) + orig(self, *args, **kwargs) + + monkeypatch.setattr(mod.BaseHttpConstationClient, "__init__", _init) + + +def test_s3_missing_bundle_fail_closed_via_http(tmp_path: Path, monkeypatch) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + settings = _settings(tmp_path) + headers = {"Authorization": "Bearer secret"} + signer = worker_signer_from_key(WORKER_KEY) + + with TestClient(create_app(settings)) as client: + seed = client.post( + "/internal/v1/bridge/submissions", + content=_zip_b64(), + headers={ + "Authorization": "Bearer secret", + "X-Base-Verified-Hotkey": "hk-owner", + "X-Submission-Filename": "project.zip", + "Content-Type": "application/octet-stream", + }, + ) + assert seed.status_code == 200, seed.text + sid = seed.json()["id"] + manifest = _manifest() + proof = build_execution_proof( + signer=signer, + manifest_sha256=compute_manifest_sha256(manifest), + unit_id=sid, + image_digest=DIGEST, + constation_digest=DIGEST, + ).model_dump(mode="json") + body = { + "api_version": "1.0", + "work_unit_id": sid, + "assignment_id": sid, + "submission_ref": "hk-owner", + "challenge_slug": settings.slug, + "result": { + "executed": 1, + PROOF_PAYLOAD_KEY: proof, + MANIFEST_PAYLOAD_KEY: manifest, + }, + "proof": proof, + } + resp = client.post("/internal/v1/work_units/result", json=body, headers=headers) + assert resp.status_code == 422, resp.text + detail = resp.json()["detail"] + code = detail.get("code") if isinstance(detail, dict) else None + assert code in { + "miner_fault:missing_constation_bundle", + "constation_rejected", + } or (isinstance(detail, dict) and "missing_constation" in str(detail.get("code", ""))), ( + detail + ) + + db_path = tmp_path / "coord.sqlite3" + conn = sqlite3.connect(db_path) + try: + score = conn.execute( + "SELECT final_score FROM scores WHERE submission_id=?", (sid,) + ).fetchone() + finally: + conn.close() + assert score is None + + +def test_s1_honest_bundle_scores_via_http(tmp_path: Path, monkeypatch) -> None: + data_dir = _stage_train(tmp_path) + monkeypatch.setattr( + "prism_challenge.evaluator.container.DockerExecutor.run", + cpu_reexec_run(train_data_dir=data_dir), + ) + settings = _settings(tmp_path) + headers = {"Authorization": "Bearer secret"} + signer = worker_signer_from_key(WORKER_KEY) + + with TestClient(create_app(settings)) as client: + seed = client.post( + "/internal/v1/bridge/submissions", + content=_zip_b64(), + headers={ + "Authorization": "Bearer secret", + "X-Base-Verified-Hotkey": "hk-owner", + "X-Submission-Filename": "project.zip", + "Content-Type": "application/octet-stream", + }, + ) + assert seed.status_code == 200, seed.text + sid = seed.json()["id"] + manifest = _manifest() + proof = build_execution_proof( + signer=signer, + manifest_sha256=compute_manifest_sha256(manifest), + unit_id=sid, + image_digest=DIGEST, + constation_digest=DIGEST, + tier=1, # type: ignore[arg-type] + ).model_dump(mode="json") + body = { + "api_version": "1.0", + "work_unit_id": sid, + "assignment_id": sid, + "submission_ref": "hk-owner", + "challenge_slug": settings.slug, + "result": { + "executed": 1, + PROOF_PAYLOAD_KEY: proof, + MANIFEST_PAYLOAD_KEY: manifest, + "constation_bundle": _bundle_for(sid), + }, + "proof": proof, + } + resp = client.post("/internal/v1/work_units/result", json=body, headers=headers) + assert resp.status_code == 200, resp.text + data = resp.json() + assert data["status"] == "accepted", data + assert data.get("score_written") is True, data + assert data.get("effective_tier") == 1, data + assert data.get("attestation_mode"), data diff --git a/tests/test_prod_constation_kwargs.py b/tests/test_prod_constation_kwargs.py new file mode 100644 index 0000000..0361e6f --- /dev/null +++ b/tests/test_prod_constation_kwargs.py @@ -0,0 +1,58 @@ +"""Production constation kwargs: deserialize bundle + attach checkers.""" + +from __future__ import annotations + +from prism_challenge.app import _constation_ingest_kwargs +from prism_challenge.config import PrismSettings +from prism_challenge.constation import ConstationBundle, constation_bundle_to_dict + + +def _bundle_dict() -> dict: + man = {"h.py": "a" * 64} + digest = "sha256:" + ("1" * 64) + b = ConstationBundle( + commit_sha="a" * 40, + tree_sha="b" * 40, + variant="cuda", + digest=digest, + work_unit_id="wu-1", + miner_hotkey="hk", + pod_id="pod", + nonce="n", + signed_attestation={"sig": "x"}, + expected_sealed_manifest_hashes=dict(man), + reported_sealed_manifest_hashes=dict(man), + lium_declared_digest=digest, + constation_gap_budget_seconds=30.0, + constation_observed_max_gap_seconds=1.0, + ) + return constation_bundle_to_dict(b) + + +def test_prod_missing_bundle_returns_empty() -> None: + settings = PrismSettings( + allow_insecure_signatures=False, + constation_base_url="http://base.test", + constation_internal_token="tok", + ) + assert _constation_ingest_kwargs(settings, {"executed": 1}) == {} + + +def test_prod_with_bundle_and_checkers() -> None: + settings = PrismSettings( + allow_insecure_signatures=False, + constation_base_url="http://base.test", + constation_internal_token="tok", + ) + kwargs = _constation_ingest_kwargs(settings, {"constation_bundle": _bundle_dict()}) + assert "constation_bundle" in kwargs + assert kwargs["check_allowlist"] is not None + assert kwargs["check_nonce"] is not None + assert kwargs["verify_constation_signature"] is not None + + +def test_insecure_seam_still_injects_without_bundle() -> None: + settings = PrismSettings(allow_insecure_signatures=True) + kwargs = _constation_ingest_kwargs(settings, {}) + # Test seam injects synthetic bundle + assert "constation_bundle" in kwargs diff --git a/tests/test_sdk_release_boundary.py b/tests/test_sdk_release_boundary.py index a9e422d..1896c73 100644 --- a/tests/test_sdk_release_boundary.py +++ b/tests/test_sdk_release_boundary.py @@ -96,10 +96,15 @@ def test_prism_wheel_contains_no_vendored_sdk(tmp_path: Path) -> None: def test_prism_service_build_uses_only_the_immutable_base_wheel() -> None: repository = Path(__file__).resolve().parents[1] dockerfile = (repository / "Dockerfile").read_text(encoding="utf-8") - assert "base @ https://github.com/BaseIntelligence/base/releases/download/v3.1.2/" in ( - repository / "pyproject.toml" - ).read_text(encoding="utf-8") - assert "git" not in dockerfile.lower() + pyproject = (repository / "pyproject.toml").read_text(encoding="utf-8") + # Immutable release wheel ships attestation modules (DigestAllowlist, + # AttestationNonceService, payload) from base v3.2.0+. + release_pin_prefix = ( + "base @ https://github.com/BaseIntelligence/base/releases/download/" + ) + assert release_pin_prefix in pyproject + # Dockerfile must not clone arbitrary git; dependency resolution stays in pyproject. + assert "git clone" not in dockerfile.lower() def test_clean_artifacts_resolve_one_base_sdk(tmp_path: Path) -> None: @@ -172,9 +177,10 @@ def test_clean_artifacts_resolve_one_base_sdk(tmp_path: Path) -> None: assert evidence["base_version"] == evidence["manifest"]["artifact_version"] assert evidence["sdk_version"] == evidence["manifest"]["sdk_contract_version"] assert evidence["prism_sdk"] is None + release_pin_prefix = ( + "base @ https://github.com/BaseIntelligence/base/releases/download/" + ) assert any( - requirement.startswith( - "base @ https://github.com/BaseIntelligence/base/releases/download/v3.1.2/" - ) + requirement.startswith(release_pin_prefix) for requirement in evidence["base_requirement"] ) diff --git a/uv.lock b/uv.lock index a36ab27..6d7e8a0 100644 --- a/uv.lock +++ b/uv.lock @@ -294,8 +294,8 @@ wheels = [ [[package]] name = "base" -version = "3.1.2" -source = { url = "https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl" } +version = "3.2.0" +source = { url = "https://github.com/BaseIntelligence/base/releases/download/v3.2.0/base-3.2.0-py3-none-any.whl" } dependencies = [ { name = "aiosqlite" }, { name = "alembic" }, @@ -318,7 +318,7 @@ dependencies = [ { name = "uvicorn", extra = ["standard"] }, ] wheels = [ - { url = "https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl", hash = "sha256:3a61c2d3a343ed6de55e80215486e3de0c9639276443d08f2ed316bc807f2ff0" }, + { url = "https://github.com/BaseIntelligence/base/releases/download/v3.2.0/base-3.2.0-py3-none-any.whl", hash = "sha256:0fdd79b8eacda77132d01173df2057730350f3e82fb5a8a4478e7a53e61852c3" }, ] [package.metadata] @@ -2142,7 +2142,7 @@ dev = [ [package.metadata] requires-dist = [ { name = "aiosqlite", specifier = ">=0.20" }, - { name = "base", url = "https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl" }, + { name = "base", url = "https://github.com/BaseIntelligence/base/releases/download/v3.2.0/base-3.2.0-py3-none-any.whl" }, { name = "bittensor", marker = "extra == 'bittensor'", specifier = ">=9" }, { name = "bittensor", marker = "extra == 'dev'", specifier = ">=9" }, { name = "cryptography", specifier = ">=49.0.0" },