Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Copy to .env and fill in values. For Docker Compose, do NOT use 'export' prefix.
#
# Auto-generated by prepare-env.sh: GOCLAW_GATEWAY_TOKEN, GOCLAW_ENCRYPTION_KEY.
# LLM providers and channels are configured via the web dashboard setup wizard.
# LLM provider API keys: configure via the web dashboard setup wizard.

# --- Gateway (required) ---
GOCLAW_GATEWAY_TOKEN=
Expand Down
16 changes: 8 additions & 8 deletions api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@
| POST | `/v1/chat/completions` | OpenAI-compatible chat API |
| POST | `/v1/responses` | Responses protocol |
| POST | `/v1/tools/invoke` | Tool invocation |
| GET/POST | `/v1/agents/*` | Agent management (managed mode) |
| GET/POST | `/v1/skills/*` | Skills management (managed mode) |
| GET/POST/PUT/DELETE | `/v1/tools/custom/*` | Custom tool CRUD (managed mode) |
| GET/POST/PUT/DELETE | `/v1/mcp/*` | MCP server + grants management (managed mode) |
| GET | `/v1/traces/*` | Trace viewer (managed mode) |
| GET/POST | `/v1/agents/*` | Agent management |
| GET/POST | `/v1/skills/*` | Skills management |
| GET/POST/PUT/DELETE | `/v1/tools/custom/*` | Custom tool CRUD |
| GET/POST/PUT/DELETE | `/v1/mcp/*` | MCP server + grants management |
| GET | `/v1/traces/*` | Trace viewer |

## Custom Tools (Managed Mode)
## Custom Tools

Define shell-based tools at runtime via HTTP API — no recompile or restart needed. The LLM can invoke custom tools identically to built-in tools.

Expand Down Expand Up @@ -67,7 +67,7 @@ Connect external [Model Context Protocol](https://modelcontextprotocol.io) serve

**Supported transports:** `stdio`, `sse`, `streamable-http`

**Standalone mode** — configure in `config.json`:
**Static config** — configure in `config.json` (deprecated; use HTTP API for dynamic management):

```json
{
Expand All @@ -87,7 +87,7 @@ Connect external [Model Context Protocol](https://modelcontextprotocol.io) serve
}
```

**Managed mode** — full CRUD via HTTP API with per-agent and per-user access grants:
**HTTP API** — full CRUD with per-agent and per-user access grants:

| Method | Path | Description |
|---|---|---|
Expand Down
1 change: 0 additions & 1 deletion cmd/gateway.go
Original file line number Diff line number Diff line change
Expand Up @@ -909,7 +909,6 @@ func runGateway() {
slog.Info("goclaw gateway starting",
"version", Version,
"protocol", protocol.ProtocolVersion,
"mode", "managed",
"agents", agentRouter.List(),
"tools", toolsReg.Count(),
"channels", channelMgr.GetEnabledChannels(),
Expand Down
14 changes: 7 additions & 7 deletions cmd/onboard.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,25 +37,25 @@ func runOnboard() {
}
}

// ── Step 1: Postgres DSN ──
// ── Step 1: Postgres connection ──
postgresDSN := os.Getenv("GOCLAW_POSTGRES_DSN")
if postgresDSN == "" {
postgresDSN = cfg.Database.PostgresDSN
}
if postgresDSN == "" {
var err error
postgresDSN, err = promptString("Postgres DSN", "Connection string (e.g. postgres://user:pass@host:5432/dbname)", "")
fmt.Println("── Database Connection ──")
fmt.Println(" Enter your PostgreSQL connection details (press Enter for defaults).")
fmt.Println()

dsn, err := promptPostgresFields()
if err != nil {
fmt.Println("Cancelled.")
return
}
postgresDSN = dsn
} else {
fmt.Printf(" Using Postgres DSN from environment\n")
}
if postgresDSN == "" {
fmt.Println(" Error: Postgres DSN is required.")
return
}

// ── Step 2: Test connection ──
fmt.Print(" Testing Postgres connection... ")
Expand Down
39 changes: 39 additions & 0 deletions cmd/onboard_helpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"crypto/rand"
"encoding/hex"
"fmt"
"net/url"
"os"
"strings"
)
Expand All @@ -14,6 +15,44 @@ func onboardGenerateToken(bytes int) string {
return hex.EncodeToString(b)
}

// promptPostgresFields prompts for individual database fields and builds a DSN.
func promptPostgresFields() (string, error) {
host, err := promptString("Host", "", "localhost")
if err != nil {
return "", err
}
port, err := promptString("Port", "", "5432")
if err != nil {
return "", err
}
dbName, err := promptString("Database name", "", "goclaw")
if err != nil {
return "", err
}
user, err := promptString("Username", "", "postgres")
if err != nil {
return "", err
}
password, err := promptPassword("Password", "Leave empty if no password")
if err != nil {
return "", err
}
sslMode, err := promptString("SSL mode", "", "disable")
if err != nil {
return "", err
}

// Build DSN with proper escaping
var userInfo *url.Userinfo
if password != "" {
userInfo = url.UserPassword(user, password)
} else {
userInfo = url.User(user)
}
dsn := fmt.Sprintf("postgres://%s@%s:%s/%s?sslmode=%s", userInfo.String(), host, port, dbName, sslMode)
return dsn, nil
}

// onboardWriteEnvFile writes the minimal .env.local with only the 3 required secrets.
func onboardWriteEnvFile(path, postgresDSN, gatewayToken, encryptionKey string) {
var lines []string
Expand Down
6 changes: 3 additions & 3 deletions docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ set -e

case "${1:-serve}" in
serve)
# Managed mode: auto-upgrade (schema migrations + data hooks) before starting.
if [ "$GOCLAW_MODE" = "managed" ] && [ -n "$GOCLAW_POSTGRES_DSN" ]; then
echo "Managed mode: running upgrade..."
# Auto-upgrade (schema migrations + data hooks) before starting.
if [ -n "$GOCLAW_POSTGRES_DSN" ]; then
echo "Running database upgrade..."
/app/goclaw upgrade || \
echo "Upgrade warning (may already be up-to-date)"
fi
Expand Down
6 changes: 3 additions & 3 deletions docs/00-architecture-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ sequenceDiagram

---

## 7. Managed Mode Wiring
## 7. Database Wiring

The `wireManagedExtras()` function in `cmd/gateway_managed.go` wires multi-tenant components:

Expand Down Expand Up @@ -393,7 +393,7 @@ flowchart TD
|------|---------|
| `cmd/root.go` | Cobra CLI entry point, flag parsing |
| `cmd/gateway.go` | Gateway startup orchestrator (`runGateway()`) |
| `cmd/gateway_managed.go` | Managed mode wiring (`wireManagedExtras()`, `wireManagedHTTP()`) |
| `cmd/gateway_managed.go` | Database wiring (`wireManagedExtras()`, `wireManagedHTTP()`) |
| `cmd/gateway_callbacks.go` | Shared callbacks (user seeding, context file loading) |
| `cmd/gateway_consumer.go` | Inbound message consumer (subagent, delegate, teammate, handoff routing) |
| `cmd/gateway_providers.go` | Provider registration (config-based + DB-based) |
Expand Down Expand Up @@ -423,7 +423,7 @@ flowchart TD
| [02-providers.md](./02-providers.md) | LLM providers, retry logic, schema cleaning |
| [03-tools-system.md](./03-tools-system.md) | Tool registry, policy engine, interceptors, custom tools, MCP grants |
| [04-gateway-protocol.md](./04-gateway-protocol.md) | WebSocket protocol v3, HTTP API, RBAC, identity propagation |
| [05-channels-messaging.md](./05-channels-messaging.md) | Channel adapters, Telegram formatting, pairing, managed-mode user scoping |
| [05-channels-messaging.md](./05-channels-messaging.md) | Channel adapters, Telegram formatting, pairing, per-user scoping |
| [06-store-data-model.md](./06-store-data-model.md) | Store interfaces, PostgreSQL schema, session caching, custom tool store |
| [07-bootstrap-skills-memory.md](./07-bootstrap-skills-memory.md) | Bootstrap files, skills system, memory, skills grants |
| [08-scheduling-cron.md](./08-scheduling-cron.md) | Scheduler lanes, cron lifecycle |
Expand Down
6 changes: 3 additions & 3 deletions docs/01-agent-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ flowchart TD

- Increment the `activeRuns` atomic counter (no mutex -- true concurrency, especially in group chats with `maxConcurrent = 3`).
- Emit a `run.started` event to notify connected clients.
- Create a trace record (managed mode) with a generated trace UUID.
- Create a trace record with a generated trace UUID.
- Propagate context values: `WithAgentID()`, `WithUserID()`, `WithAgentType()`. Downstream tools and interceptors rely on these.
- Compute per-user workspace: `base + "/" + sanitize(userID)`. Inject via `WithToolWorkspace(ctx)` so all filesystem and shell tools use the correct directory.
- Ensure per-user files exist. A `sync.Map` cache guarantees the seeding function runs at most once per user.
Expand Down Expand Up @@ -381,7 +381,7 @@ flowchart TD

---

## 10. Resolver (Managed Mode)
## 10. Resolver

The `ManagedResolver` lazy-creates Loop instances from PostgreSQL data when the Router encounters a cache miss.

Expand All @@ -392,7 +392,7 @@ flowchart TD
PROV --> BOOT["Step 3: Load bootstrap files<br/>bootstrap.LoadFromStore(agentID)"]
BOOT --> DEFAULTS["Step 4: Apply defaults<br/>contextWindow <= 0 then 200K<br/>maxIterations <= 0 then 20"]
DEFAULTS --> CREATE["Step 5: Create Loop<br/>NewLoop(LoopConfig)"]
CREATE --> WIRE["Step 6: Wire managed-mode hooks<br/>EnsureUserFilesFunc, ContextFileLoaderFunc"]
CREATE --> WIRE["Step 6: Wire hooks<br/>EnsureUserFilesFunc, ContextFileLoaderFunc"]
WIRE --> DONE["Return Loop to Router for caching"]
```

Expand Down
4 changes: 2 additions & 2 deletions docs/02-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,9 +202,9 @@ The Anthropic provider calls `CleanSchemaForProvider("anthropic", ...)` when con

---

## 7. Managed Mode -- Providers from Database
## 7. Providers from Database

In managed mode, providers are loaded from the `llm_providers` table in addition to the config file. Database providers override config providers with the same name.
Providers are loaded from the `llm_providers` table in addition to the config file. Database providers override config providers with the same name.

### Loading Flow

Expand Down
8 changes: 4 additions & 4 deletions docs/03-tools-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ Context keys ensure each tool call receives the correct per-call values without

## 3. Filesystem Tools and Virtual FS Routing

In managed mode, filesystem operations are intercepted before hitting the host disk. Two interceptor layers route specific paths to the database instead.
Filesystem operations are intercepted before hitting the host disk. Two interceptor layers route specific paths to the database instead.

```mermaid
flowchart TD
Expand Down Expand Up @@ -570,9 +570,9 @@ GoClaw integrates with Model Context Protocol (MCP) servers via `internal/mcp/`.
- Tools are registered with a prefix (e.g., `mcp_servername_toolname`)
- Dynamic tool group registration: `mcp` and `mcp:{serverName}` groups

### Access Control (Managed Mode)
### Access Control

In managed mode, MCP server access is controlled through per-agent and per-user grants stored in PostgreSQL.
MCP server access is controlled through per-agent and per-user grants stored in PostgreSQL.

```mermaid
flowchart TD
Expand Down Expand Up @@ -602,7 +602,7 @@ flowchart LR

---

## 13. Custom Tools (Managed Mode)
## 13. Custom Tools

Define shell-based tools at runtime via the HTTP API -- no recompile or restart needed. Custom tools are stored in the `custom_tools` PostgreSQL table and loaded dynamically into the agent's tool registry.

Expand Down
16 changes: 8 additions & 8 deletions docs/04-gateway-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ flowchart TD

Token comparison uses `crypto/subtle.ConstantTimeCompare` to prevent timing attacks.

In managed mode, `user_id` in the connect parameters is required for per-user session scoping and context file routing. GoClaw uses the **Identity Propagation** pattern — it trusts the upstream service to provide accurate user identity. The `user_id` is opaque (VARCHAR 255); multi-tenant deployments use the compound format `tenant.{tenantId}.user.{userId}`. See [00-architecture-overview.md Section 5](./00-architecture-overview.md) for details.
The `user_id` in the connect parameters is required for per-user session scoping and context file routing. GoClaw uses the **Identity Propagation** pattern — it trusts the upstream service to provide accurate user identity. The `user_id` is opaque (VARCHAR 255); multi-tenant deployments use the compound format `tenant.{tenantId}.user.{userId}`. See [00-architecture-overview.md Section 5](./00-architecture-overview.md) for details.

### Three Roles

Expand Down Expand Up @@ -161,7 +161,7 @@ flowchart TD
| `agent.wait` | Wait for an agent to become available |
| `agent.identity.get` | Get agent identity (name, description) |
| `agents.list` | List all accessible agents |
| `agents.create` | Create a new agent (managed mode) |
| `agents.create` | Create a new agent |
| `agents.update` | Update agent configuration |
| `agents.delete` | Soft-delete an agent |
| `agents.files.list` | List agent context files |
Expand Down Expand Up @@ -301,7 +301,7 @@ flowchart TD

- `Authorization: Bearer <token>` -- timing-safe comparison via `crypto/subtle.ConstantTimeCompare`
- No token configured: all requests allowed
- `X-GoClaw-User-Id`: required in managed mode for per-user scoping
- `X-GoClaw-User-Id`: required for per-user scoping
- `X-GoClaw-Agent-Id`: specify target agent for the request

### Endpoints
Expand Down Expand Up @@ -334,9 +334,9 @@ Direct tool invocation without the agent loop. Supports `dryRun: true` to return

Returns `{"status":"ok","protocol":3}`.

#### Managed Mode CRUD Endpoints
#### CRUD Endpoints

All managed endpoints require `Authorization: Bearer <token>` and `X-GoClaw-User-Id` header for per-user scoping.
All CRUD endpoints require `Authorization: Bearer <token>` and `X-GoClaw-User-Id` header for per-user scoping.

**Agents** (`/v1/agents`):

Expand Down Expand Up @@ -482,9 +482,9 @@ Error responses include `retryable` (boolean) and `retryAfterMs` (integer) field
| `internal/http/chat_completions.go` | POST /v1/chat/completions (OpenAI-compatible) |
| `internal/http/responses.go` | POST /v1/responses (OpenResponses protocol) |
| `internal/http/tools_invoke.go` | POST /v1/tools/invoke (direct tool execution) |
| `internal/http/agents.go` | Agent CRUD HTTP handlers (managed mode) |
| `internal/http/skills.go` | Skills HTTP handlers (managed mode) |
| `internal/http/traces.go` | Traces HTTP handlers (managed mode) |
| `internal/http/agents.go` | Agent CRUD HTTP handlers |
| `internal/http/skills.go` | Skills HTTP handlers |
| `internal/http/traces.go` | Traces HTTP handlers |
| `internal/http/delegations.go` | Delegation history HTTP handlers |
| `internal/http/summoner.go` | LLM-powered agent setup (XML parsing, context file generation) |
| `internal/http/auth.go` | Bearer token authentication, timing-safe comparison |
Expand Down
6 changes: 3 additions & 3 deletions docs/05-channels-messaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ flowchart LR

Internal channels (`cli`, `system`, `subagent`) are silently skipped by the outbound dispatcher and never forwarded to external platforms.

### Handoff Routing (Managed Mode)
### Handoff Routing

Before normal agent routing, the consumer checks the `handoff_routes` table for an active routing override. If a handoff route exists for the incoming channel + chat ID, the message is redirected to the target agent instead of the original.

Expand Down Expand Up @@ -504,7 +504,7 @@ flowchart TD
WS2 --> USER3["user_charlie/"]
```

In managed mode, channel instances are loaded from the database with their assigned agent ID. The agent key is resolved and propagated through the message pipeline, ensuring all filesystem tools, context files, and memory operations use the correct workspace.
Channel instances are loaded from the database with their assigned agent ID. The agent key is resolved and propagated through the message pipeline, ensuring all filesystem tools, context files, and memory operations use the correct workspace.

---

Expand Down Expand Up @@ -570,7 +570,7 @@ flowchart TD
|------|---------|
| `internal/channels/channel.go` | Channel interface, BaseChannel, extended interfaces, HandleMessage |
| `internal/channels/manager.go` | Manager: registration, StartAll, StopAll, outbound dispatch, webhook collection |
| `internal/channels/instance_loader.go` | DB-based channel instance loading (managed mode) |
| `internal/channels/instance_loader.go` | DB-based channel instance loading |
| `internal/channels/telegram/channel.go` | Telegram core: long polling, mention gating, typing indicators |
| `internal/channels/telegram/handlers.go` | Message handling, media processing, forum topic detection |
| `internal/channels/telegram/topic_config.go` | Per-topic config layering and resolution |
Expand Down
4 changes: 2 additions & 2 deletions docs/06-store-data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ flowchart TD

## 4. Agent Access Control

In managed mode, agent access is checked via a 4-step pipeline.
Agent access is checked via a 4-step pipeline.

```mermaid
flowchart TD
Expand Down Expand Up @@ -477,7 +477,7 @@ flowchart TD
| Key | Type | Purpose |
|-----|------|---------|
| `goclaw_user_id` | string | External user ID (e.g., Telegram user ID) |
| `goclaw_agent_id` | uuid.UUID | Agent UUID (managed mode) |
| `goclaw_agent_id` | uuid.UUID | Agent UUID |
| `goclaw_agent_type` | string | Agent type: `"open"` or `"predefined"` |
| `goclaw_sender_id` | string | Original individual sender ID (in group chats, `user_id` is group-scoped but `sender_id` preserves the actual person) |

Expand Down
14 changes: 7 additions & 7 deletions docs/07-bootstrap-skills-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ This ensures resolver-injected virtual files (`DELEGATION.md`, `TEAM.md`) surviv

---

## 7. Agent Summoning (Managed Mode)
## 7. Agent Summoning

Creating a predefined agent requires 4 context files (SOUL.md, IDENTITY.md, AGENTS.md, TOOLS.md) with specific formatting conventions. Agent summoning generates all 4 files from a natural language description in a single LLM call.

Expand Down Expand Up @@ -260,9 +260,9 @@ IDF is computed as: `log((N - df + 0.5) / (df + 0.5) + 1)`

---

## 11. Skills -- Embedding Search (Managed Mode)
## 11. Skills -- Embedding Search

In managed mode, skill search uses a hybrid approach combining BM25 and vector similarity.
Skill search uses a hybrid approach combining BM25 and vector similarity.

```mermaid
flowchart TD
Expand All @@ -283,9 +283,9 @@ flowchart TD

---

## 12. Skills Grants & Visibility (Managed Mode)
## 12. Skills Grants & Visibility

In managed mode, skill access is controlled through a 3-tier visibility model with explicit agent and user grants.
Skill access is controlled through a 3-tier visibility model with explicit agent and user grants.

```mermaid
flowchart TD
Expand Down Expand Up @@ -315,7 +315,7 @@ flowchart TD

**Resolution**: `ListAccessible(agentID, userID)` performs a DISTINCT join across `skills`, `skill_agent_grants`, and `skill_user_grants` with the visibility filter, returning only active skills the caller can access.

**Managed-mode Tier 4**: In managed mode, global skills (Tier 4 in the hierarchy) are loaded from the `skills` PostgreSQL table instead of the filesystem.
**Tier 4**: Global skills (Tier 4 in the hierarchy) are loaded from the `skills` PostgreSQL table instead of the filesystem.

---

Expand Down Expand Up @@ -489,7 +489,7 @@ The flush is idempotent per compaction cycle -- it will not run again until the

| Document | Relevant Content |
|----------|-----------------|
| [00-architecture-overview.md](./00-architecture-overview.md) | Startup sequence, managed mode wiring |
| [00-architecture-overview.md](./00-architecture-overview.md) | Startup sequence, database wiring |
| [01-agent-loop.md](./01-agent-loop.md) | Agent loop calls BuildSystemPrompt, compaction flow |
| [03-tools-system.md](./03-tools-system.md) | ContextFileInterceptor routing read_file/write_file to DB |
| [06-store-data-model.md](./06-store-data-model.md) | memory_documents, memory_chunks tables |
Loading