Joysafeter v2#141
Open
yuzzjj wants to merge 575 commits into
Open
Conversation
…dundant workspace file services across backend and frontend
…o NotFoundError and InvalidRequestError
…ill permissions, skill version, task activity, task, thread, workspace services, and utility functions. These deletions streamline the test suite by eliminating outdated or redundant tests, ensuring a more maintainable codebase.
…ect documentation and focus on finalized design specifications.
- Removed the draft boundary design document and the draft to initial version terminology design document as they are no longer needed. - Updated the login form to improve error handling for authentication errors, utilizing a centralized error message mapping. - Enhanced OAuth button component to handle errors more effectively, including specific error messages for various OAuth-related issues. - Improved Copilot error boundary to differentiate between network and WebSocket errors using error codes. - Refactored useCopilotActions hook to streamline error handling and improve user feedback for authentication and network errors. - Updated English and Chinese localization files to include new error messages related to OAuth provider issues.
…and structured responses
…services and components - Reordered import statements for consistency in `execution_service.py`, `mcp_server_service.py`, `oauth_service.py`, `organization_service.py`, `platform_token_service.py`, `sandbox_manager.py`, `skill_collaborator_service.py`, `skill_service.py`, `skill_version_service.py`, `task_activity_service.py`, `task_service.py`, `thread_service.py`, `workspace_service.py`, and `execution_subscription_handler.py`. - Consolidated error messages in `organization_service.py`, `skill_collaborator_service.py`, `task_activity_service.py`, and others for better clarity and maintainability. - Updated error handling to use `createApiError` in various frontend services and hooks for consistent error structure. - Added tests for `use-test-model-stream` and `api-client` to ensure proper error handling and response validation. - Removed unnecessary blank lines and improved formatting for better readability.
… structured error payloads and specific error codes
- Updated InternalServiceError raising in live_read_file to include detailed error context. - Introduced normalize_app_error function to standardize error normalization across the application. - Refactored normalize_exception to utilize normalize_app_error for better consistency. - Added error_payload fields in CLIResult for ClaudeCodeProvider and CodexProvider to provide structured error information. - Enhanced ExecutionRunner to return structured error payloads using normalize_app_error. - Updated OpenClawProvider to handle error payloads and emit structured error messages. - Modified GraphEngine and CopilotEngine to utilize normalize_app_error for error handling. - Improved error handling in orchestrator methods to include specific error codes and data. - Added tests for structured error payloads in CLIMessage and OpenClawProvider. - Updated frontend types to accommodate structured error payloads in error handling.
…ion-specific error classes across core services and engine components
Designs a Langfuse-aligned observation system for in-product agent debugging: unified ObservationCollector across all four engines (Graph/CLI/Code/Copilot) with WebSocket streaming of structured trace trees to the Builder UI. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
- Rename Trace.metadata/Observation.metadata attribute to meta to avoid SQLAlchemy Base.metadata collision (column name unchanged) - Clarify collector.flush delegates to writer.flush, distinguish from finalize - Make broadcaster seq counter explicit - Document finalize behavior preserves ERROR level over auto-closed WARNING - Scope span_update to v1 Copilot streaming only - Note v1 Copilot ships without cost calculation - Gate trace_context.py deletion on external caller check Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
16-task TDD plan covering Phase 1 (core layer: types/model/writer/broadcaster/collector), Phase 2 (engine integration: all 4 engines + orchestrator + instrumentation), Phase 3 (API + frontend outline). Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
…aligned with Langfuse Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Introduces Trace and Observation ORM models aligned with the Langfuse schema. Uses `meta` as the Python attribute name mapped to a `metadata` column to avoid collision with SQLAlchemy's Base.metadata. All required fields are NOT NULL; server_default values are set for status and level. 14/14 tests passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e files - Simplified object key handling in agent-diff.ts - Enhanced formatting and readability in errors.ts - Improved filter logic in filters.ts - Streamlined ID stripping logic in id.ts - Refined role options function in roles.ts - Consolidated secret key definitions in secret-keys.ts - Enhanced draft zip file naming in skill-draft-zip.ts - Cleaned up skill import utilities for better clarity - Optimized skill import handling in skill-import.ts - Improved skill version diffing logic in skill-version-diff.ts - Enhanced SSE handling in sse.ts - Refined URL trimming logic in url-trim.ts - Streamlined permissions provider rendering in permissions-provider.tsx - Improved proxy response handling in proxy.ts - Simplified agent type definitions in agent.ts - Refined managed types for better clarity and consistency in managed.ts
…ing) Closes the residual zombie race left by the running-task lease: an instance that stalls without crashing (GC pause / partition) can wake after its lease lapsed and its task was reclaimed + re-run under a new owner, then make a stale mutating write that corrupts the task the new owner now runs. Each →RUNNING claim stamps a globally-monotonic owner_epoch (a Postgres SEQUENCE — the only durable monotonic source; Redis has none and a task-local counter can't survive a cross-instance requeue). Mutating writes for a running task (status transition, error, output, usage) and failover_or_fail_task are conditioned on the epoch the caller was granted; a stale-epoch write matches zero rows and is dropped. The token is threaded through both terminal-write paths — the in-process TaskRunner and the gRPC completion handlers (_handle_result / _handle_reconnect_result via bridge.current_owner_epoch). expected_epoch=None preserves the prior status-only behavior for callers that hold no grant (pre-RUNNING scheduler/watchdog). Independently verified: 28/28 tests green against real Postgres (7 new fencing tests incl. end-to-end stale-write and stale-failover drops), ruff + format clean, mypy clean on the changed files, alembic chain linear with reversible up/down. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
GLei16
approved these changes
Jul 2, 2026
GLei16
previously approved these changes
Jul 2, 2026
…reads
Cross-instance input (a tool confirmation / interrupt that lands on an
orchestrator instance which is NOT the sandbox owner, routed to the owner via
Redis joysafeter:cmd:{owner}) was silently dropped. CommandListener._dispatch
enqueued the input into bridge.runner_tx — an asyncio.Queue with no consumer
anywhere in the repo — and logged success, but the runner loops deliver input
by draining bridge._control_queue on confirmation, so the agent resumed from
its HITL wait with no input. Remote cancel only worked incidentally because it
also set _cancel_event.
Route input via bridge.send_control_input(content) (-> _control_queue +
confirmation_event, matching the local path) and cancel via
bridge.request_cancel() (-> _cancel_event). This fixes both the gRPC-runner and
in-process TaskRunner execution modes, since both consume _control_queue /
_cancel_event, not runner_tx.
Independently verified PASS: 31/31 tests green (3 new routing tests, RED-proven),
ruff/format/mypy clean; whole-repo grep confirms runner_tx has no consumer.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…stream grpc.aio forbids concurrent writes to one stream, but the orchestrator wrote to a sandbox's runner stream from many coroutines (the gRPC handler loop plus API/relay/shutdown paths) — a latent interleave/corruption hazard. Separately a `bridge.runner_tx` asyncio.Queue had no consumer, so writes enqueued there were silently dropped (graceful shutdown, agent-cancel, memory-sync). SandboxBridge.write_to_runner(msg) is now the ONE way to send to the runner: it holds a per-bridge asyncio.Lock so writes can never overlap, re-checks the stream for None under the lock (returns False/drops instead of raising on a concurrent disconnect), and every producer — the ~10 gRPC handler-loop writes and all external writers — funnels through it. The dead runner_tx queue and the send_to_runner shim are removed. Because it's a direct locked write (not a task-loop-dependent drain), shutdown/memory-sync/cancel now reach even idle-but-connected sandboxes. Independently verified PASS: 34/34 tests green (incl. a serialization test and adversarial probes — 200 concurrent writers stay max_in_flight==1, and a disconnect that nulls the stream mid-flight is caught by the under-lock re-check), ruff/format/mypy clean, grep-confirmed sole writer (zero context.write / runner_stream.write / runner_tx references remain). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ion 3) A single tenant (project) could occupy unbounded orchestrator/sandbox capacity and starve every other tenant on the shared HA fleet. Gate task submission on the project's live (non-terminal) task count. - Project.max_concurrent_tasks (nullable) overrides the global max_concurrent_per_project default; migration 20260703_000005. - TaskService.count_active_tasks_for_project + resolve_project_task_limit hold the load-bearing logic (Postgres-tested); the API is thin glue. - create_task returns a structured 429 (code/limit/active/project_id + Retry-After) placed AFTER the idempotency short-circuit, so an idempotent retry is never rejected. - Soft limit: count-then-create can slightly over-admit under concurrent submits — acceptable for a fairness quota (idempotency stays strict). Independently verified PASS: ruff/mypy clean, 40 tests pass (incl. 6 new + fresh-container migration chain), single alembic head. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
sandbox_cpu/sandbox_memory_mb were defined but never passed to the provider, so every Docker sandbox ran unbounded — one tenant's agent could exhaust host CPU/RAM on the shared fleet. Wire the limits through and enforce safe defaults. - Defaults now enforced: sandbox_cpu=2.0 cores, sandbox_memory_mb=4096 MB (still Optional; None disables). Overridable per-deployment via env. - Per-project override: nullable Project.max_cpu / max_memory_mb columns; migration 20260703_000006. Resolved per-field (a project may override only one); NULL falls back to the global default. - resolve_project_sandbox_limits (Postgres-tested) holds the decision; the Docker provider translates it to HostConfig NanoCpus / Memory. A project-agnostic warm-pool sandbox uses global defaults with no DB read. Independently verified PASS: ruff/format/mypy/pyright clean, 47 tests pass (incl. 7 new + fresh-container migration chain), single alembic head. Adversarial probe confirmed a 0.0 override is honored (is-not-None, not truthiness). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…undation 2) Make session-status events durable and observable instead of best-effort: - Batch writer no longer silently drops on a wedged queue; it writes the event synchronously and records a lost-event counter for /health. - Session status events are routed through the persist-then-broadcast path and skipped by the async batch/stream persisters to avoid double-writes. - EventBus and EventBatchSender expose health_snapshot(); /health surfaces event_bus / event_buffer / event_stream backlog + dead-letter state. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…depend on them Prevent pulling dependencies out from under a running task (which for pentest workloads means irreproducible, externally-visible side effects): - agent: reject secret_ref/environment_ref change, delete, archive, and session-archive while the agent has active tasks; validate environment_ref exists and is not archived on create/update. - environment / secret / memory-store: reject update (name/config), delete, and archive while referenced by an active task, agent, or live session. - project: reject archive while it has active tasks. Endpoints translate the new service-layer ValueErrors into 409/503. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…asks (Foundation 3) Add nullable, non-FK-constrained user_id/org_id columns to joysafeter_tasks for attribution/audit and per-user admission control (survive user/org deletion). Composite index (user_id, status) backs the per-user active-task count. Schema only; the writer and admission gate land in later commits. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…re (Foundation 1) Make session-state ownership explicit so a stale task can no longer flip a session that a newer task now owns, and stop stale sandboxes from writing terminal state after failover: - New SessionService.update_session_status_for_task_event(session_id, status, task_id) under pg_advisory_xact_lock + FOR UPDATE, rejecting status events whose task no longer owns the session. All orchestrator/runner/gRPC status transitions route through it and carry task_id end-to-end. - Propagate expected_epoch to every terminal task write (error/status/output/ usage/retry); CAS conflicts now abort the write instead of overwriting a reowned task. - failover_or_fail_task returns Optional[int]; callers check "is not None" so the first failover (retry_count == 0) is actually requeued. - Startup recovery respects max_retries for stale scheduling tasks; sandbox cleanup/orphan rescue go through failover; cancel() is a single CAS UPDATE; runner idle-before-result and non-terminal result are handled defensively. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…istener The command listener now runs each dispatched command through _dispatch_inner and RPUSHes an ack (ok/error, keyed by command_id, TTL 30s) to the caller's ack_key, refusing ack_keys without the expected prefix. This lets the API side confirm a control input/cancel/shutdown was actually delivered to the local bridge instead of assuming success on publish. The API-side requester lands in the session-events commit. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…TL delivery Harden the task/session write endpoints: - user.message is idempotent via Idempotency-Key: a partial unique index on (session_id, payload->>'_idempotency_key') plus IntegrityError replay makes a retried submit return the original event/task instead of double-dispatching. - Per-user concurrent-task admission (structured 429) alongside the existing per-project gate; task rows now carry the submitter's user_id/org_id. - Task/session dispatch marks the session running/idle through the task-scoped primitive and compensates to idle on enqueue failure; create-task and send-event reject archived/terminated/rescheduling/running/active-task sessions. - HITL control inputs wait for the command listener's ack before returning, falling back to a retry task only on real non-delivery; unified enqueue_joysafeter_task() works in both monolith and split-service modes. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Generate a per-submit UUID (crypto.randomUUID with a timestamp fallback) and pass it as the Idempotency-Key header so a retried "send message" returns the original task/event instead of dispatching a duplicate agent run. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…ce keys
The per-user concurrent-task quota keyed on auth_ctx.user_id, which for API-key
auth is the key's creator — so every task from one service key shared a single
human's budget and could be silently clamped (default 5) below the project
limit. Add principal_type to the auth context ("api_key" for key auth) and
apply the per-user fairness gate only to human principals; service keys remain
bounded by the per-project quota. Attribution (task.user_id/org_id) is
unchanged.
Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
The provisioning-timeout path reset scheduling tasks to pending unconditionally (reset_sandbox_tasks_to_pending), ignoring max_retries — a task whose sandbox kept failing to provision could be requeued forever. Route it through TaskController.failover_or_fail_task like sandbox-death recovery, so exhausted tasks fail instead of looping, and only retryable ones are requeued. Removes the now-dead reset_sandbox_tasks_to_pending / reset_sandbox_scheduling_to_pending. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
The archive endpoint archived sessions and the agent in two separate service calls, each committing on its own — a failure of the second left sessions archived but the agent live (or the reverse). Replace with a single archive_agent_with_sessions() that validates active tasks once and commits both writes together, so archival is all-or-nothing. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…sage The result handler, reconnect-result handler, and failover path each inlined their own "if no agent.message exists, synthesize one from the task output" logic. Consolidate into SessionService.repair_missing_agent_message(), which no-ops on empty output or when the task already has an agent.message (a task legitimately emits many), so the recovery paths can't drift and the emit stays idempotent. Removes the duplicated gRPC _task_has_agent_output helper. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
The resumed-task and main streaming loops each carried an identical ~55-line block to persist and fan out a session.status_running/idle event (event-bus path vs. direct DB write + task WS broadcast + cross-instance Redis + SSE). Collapse both into AgentBridgeServicer._emit_task_scoped_status_event so the fan-out logic has a single definition and cannot drift between the two loops. Behavior is unchanged; each call site passes its own session/task/coordinator. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Both call sites of _emit_task_scoped_status_event resolved coordinator to get_redis_coordinator(); fetch it inside the helper so the two call sites are identical and the parameter list is smaller. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
… types bus.py, event_persist.py, and stream_publisher.py each defined their own identical _STATUS_EVENT_TYPES set to decide which events the two-phase bus routes persist-then-broadcast (and the async persisters skip). Hoist the one canonical set to envelope.py as STATUS_EVENT_TYPES so the routing layer cannot silently drift out of sync. No behavior change (all three sets were identical). Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
SessionService (4 sites) and the worker batch writer both derived the pg_advisory_xact_lock key inline as int.from_bytes(session_id.bytes[8:], ...). These acquirers MUST derive the key identically to mutually exclude (a documented lock-ordering contract), yet it was enforced by copy-paste. Extract session_advisory_lock_key() in joysafeter_shared.utils.locks so the derivation can't drift between the two lock acquirers. No behavior change. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
…estricted-syntax Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
Map FastAPI HTTPException to typed AppError (auth/access/notfound/conflict/ validation/ratelimit/unavailable) and inject the OTel trace_id into error payloads. On the frontend, surface trace_id (payload or x-trace-id header), truncate non-JSON error bodies, and fall back to the raw server message. Adds api-client and managed/errors unit tests. Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.