diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 00000000..ec83af76 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,49 @@ +name: Deploy docs to GitHub Pages + +on: + push: + branches: [main, docs-v1] + paths: + - 'docs/**' + - '.github/workflows/docs.yml' + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 20 + cache: npm + cache-dependency-path: docs/package-lock.json + - name: Install dependencies + working-directory: docs + run: npm ci + - name: Build site + working-directory: docs + run: npm run build + - uses: actions/configure-pages@v5 + - uses: actions/upload-pages-artifact@v3 + with: + path: docs/build + + deploy: + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 00000000..54631bd3 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,15 @@ +# Dependencies +node_modules/ + +# Production +build/ + +# Generated files +.docusaurus/ +.cache-loader/ + +# Misc +.DS_Store +npm-debug.log* +yarn-debug.log* +yarn-error.log* diff --git a/docs/docs/configure/analytics.md b/docs/docs/configure/analytics.md new file mode 100644 index 00000000..ee5b1fad --- /dev/null +++ b/docs/docs/configure/analytics.md @@ -0,0 +1,42 @@ +--- +title: Analytics +description: What anonymous usage events Anton sends, what they contain, and how to opt out. +--- + +# Analytics + +Anton collects anonymous usage events — for example "session started" or +"first query" — to help the MindsDB team understand how the product is used. + +## What is sent + +Each event is a single HTTP GET request carrying only: + +- the action name (e.g. `anton_started`), +- a timestamp, +- an anonymous installation ID. + +**No personal data or query content is ever sent** — no prompts, no file +contents, no hostnames. The installation ID is a one-way SHA-256 hash of the +machine's network adapter address, truncated to 16 hex characters; the raw +address never leaves your device. Events are fire-and-forget: they never +block Anton and failures are silently ignored. + +## Opting out + +Set the environment variable: + +```bash +export ANTON_ANALYTICS_ENABLED=false +``` + +Or add it to your workspace config (`.anton/.env`): + +```text +ANTON_ANALYTICS_ENABLED=false +``` + +To turn it off everywhere, put the same line in the global `~/.anton/.env`. +See [Environment variables](/configure/env-vars) for how the config files are +loaded, and [Security model](/configure/security) for the full picture of +what leaves your machine. diff --git a/docs/docs/configure/env-vars.md b/docs/docs/configure/env-vars.md new file mode 100644 index 00000000..4b6aa083 --- /dev/null +++ b/docs/docs/configure/env-vars.md @@ -0,0 +1,82 @@ +--- +title: Environment variables +description: The authoritative reference for ANTON_* configuration variables and the .env loading chain. +--- + +# Environment variables + +Every Anton setting can be set as an environment variable with the `ANTON_` +prefix. Most users never touch these directly — the `anton setup` and +`anton setup-search` flows write them to the right `.env` file for you — but +they are all available for scripting and overrides. + +## The .env loading chain + +Anton reads configuration from three locations: + +1. `.env` in the current directory +2. `.anton/.env` in the workspace (project-local config) +3. `~/.anton/.env` (global config — LLM keys, search keys) + +Variables set in your actual shell environment always take precedence over +values from any file. Setup flows write LLM and search provider keys to the +global file so they carry across workspaces, and workspace-specific settings +to the local one. + +## Providers and models + +| Variable | Default | What it does | +| --- | --- | --- | +| `ANTON_PLANNING_PROVIDER` | `anthropic` | Provider for the planning model (`anthropic`, `openai`, `openai-compatible`) | +| `ANTON_PLANNING_MODEL` | `claude-sonnet-4-6` | Model used for planning and conversation | +| `ANTON_CODING_PROVIDER` | `anthropic` | Provider for the coding model | +| `ANTON_CODING_MODEL` | `claude-haiku-4-5-20251001` | Model used for code generation in the scratchpad | +| `ANTON_MAX_TOKENS` | `8192` | Max output tokens per LLM call | +| `ANTON_ANTHROPIC_API_KEY` | unset | Anthropic API key | +| `ANTON_OPENAI_API_KEY` | unset | OpenAI (or OpenAI-compatible endpoint) API key | +| `ANTON_OPENAI_BASE_URL` | unset | Base URL for an OpenAI-compatible endpoint | +| `ANTON_OPENAI_API_VERSION` | unset | Azure OpenAI `api-version` query parameter | +| `ANTON_MINDS_ENABLED` | `true` | Allow using a Minds server as LLM provider | +| `ANTON_MINDS_API_KEY` | unset | Minds API key | +| `ANTON_MINDS_URL` | Minds cloud URL | Minds server URL | +| `ANTON_MINDS_MIND_NAME` | unset | Mind name to use on the Minds server | +| `ANTON_MINDS_SSL_VERIFY` | `true` | Verify SSL certificates when talking to the Minds server | + +## Web tools + +| Variable | Default | What it does | +| --- | --- | --- | +| `ANTON_WEB_SEARCH_ENABLED` | `true` | Enable the `web_search` tool — see [Web search](/connect/web-search) | +| `ANTON_WEB_FETCH_ENABLED` | `true` | Enable the `web_fetch` tool — see [Web fetch](/connect/web-fetch) | +| `ANTON_EXTERNAL_SEARCH_PROVIDER` | unset | External search provider for generic endpoints: `exa` or `brave` | +| `ANTON_EXA_API_KEY` | unset | Exa.ai API key | +| `ANTON_BRAVE_API_KEY` | unset | Brave Search API key | + +The search provider variables are normally written by `anton setup-search` — +see [Search providers](/configure/search-providers). + +## Memory + +| Variable | Default | What it does | +| --- | --- | --- | +| `ANTON_MEMORY_ENABLED` | `true` | Master switch for the memory system | +| `ANTON_MEMORY_MODE` | `autopilot` | How lessons are saved: `autopilot`, `copilot`, or `off` — see [Memory overview](/teach/memory-overview) | +| `ANTON_EPISODIC_MEMORY` | `true` | Keep an episode archive of past sessions — see [Episodes and recall](/teach/episodes-and-recall) | + +## Behavior + +| Variable | Default | What it does | +| --- | --- | --- | +| `ANTON_THEME` | `auto` | Terminal color theme | +| `ANTON_DISABLE_AUTOUPDATES` | `false` | Skip automatic update checks — see [Updating](/start/updating) | +| `ANTON_ANALYTICS_ENABLED` | `true` | Anonymous usage events — see [Analytics](/configure/analytics) | +| `ANTON_LANGFUSE_HEADERS` | unset | Set to `1` to attach Langfuse trace headers on any OpenAI-compatible endpoint — see [Trace headers](/configure/trace-headers) | +| `ANTON_PROACTIVE_DASHBOARDS` | `false` | When `true`, Anton builds HTML dashboards proactively; when `false`, CLI output only | +| `ANTON_BACKEND` | `local` | Scratchpad backend: `local` or `remote` (remote requires Minds URL and API key) | +| `ANTON_PUBLISH_URL` | `https://4nton.ai` | Publish service used when sharing artifacts | +| `ANTON_TERMS_CONSENT` | `false` | Records that you accepted the terms screen (set by the first-run flow) | +| `ANTON_FIRST_RUN_DONE` | `false` | Records that first-run onboarding completed (set by the setup flow) | + +A few additional internal tuning variables exist (tool-round limits, cell +timeouts); they are intentionally undocumented here as they are not meant for +everyday configuration. diff --git a/docs/docs/configure/search-providers.md b/docs/docs/configure/search-providers.md new file mode 100644 index 00000000..2d802fad --- /dev/null +++ b/docs/docs/configure/search-providers.md @@ -0,0 +1,66 @@ +--- +title: Search providers +description: Set up Exa.ai or Brave Search with anton setup-search for generic OpenAI-compatible endpoints. +--- + +# Search providers + +`anton setup-search` configures an external search provider so the +`web_search` tool works on LLM endpoints that don't ship native search. + +## When you need it + +Only when your LLM provider is a **generic OpenAI-compatible endpoint** +(Together, Groq, Ollama, vLLM, OpenRouter, and similar). Anthropic, OpenAI, +and Minds (mdb.ai) all execute web search natively on your existing LLM key — +no extra setup, and `setup-search` is unnecessary. See +[Web search](/connect/web-search) for the full provider matrix. + +When `anton setup` finishes configuring a custom OpenAI-compatible endpoint, +it offers this step automatically. You can run it again at any time: + +```bash +anton setup-search +``` + +## Choosing a provider + +| Provider | Character | Get a key at | +| --- | --- | --- | +| Exa.ai | AI-native semantic search | dashboard.exa.ai/api-keys | +| Brave Search | privacy-focused web search | api.search.brave.com/app/keys | + +The setup screen shows which provider (if any) is currently configured, with +a masked tail of the active key so you can recognize it without exposing it. + +## Key validation + +Anton validates the key before saving it: it makes a small probe call to the +provider's search API and checks the response. If authentication fails or the +service errors, you can: + +- **retry** — re-enter the key for the same provider (fix a typo without + re-picking from the menu), +- **switch** — go back to the picker and try the other provider, +- **skip** — disable `web_search` for now. + +## Switching providers + +Re-run `anton setup-search` and pick the other provider. The previously +stored key for the old provider is left in place; only the active provider +selection changes. + +## Skipping + +Choosing **Skip** disables `web_search` — the tool stays unavailable until +you run `anton setup-search` again. If a working provider is already +configured, Anton asks for confirmation before clearing it, so a stray +keystroke can't wipe a working setup. + +## Where the key lives + +The provider choice and key are persisted to the **global** `~/.anton/.env` +(as `ANTON_EXTERNAL_SEARCH_PROVIDER` plus `ANTON_EXA_API_KEY` or +`ANTON_BRAVE_API_KEY`), so they survive across sessions and workspaces — +the same scope as your LLM keys. See +[Environment variables](/configure/env-vars). diff --git a/docs/docs/configure/security.md b/docs/docs/configure/security.md new file mode 100644 index 00000000..8b10980f --- /dev/null +++ b/docs/docs/configure/security.md @@ -0,0 +1,75 @@ +--- +title: Security model +description: How the credential vault, scratchpad isolation, and local-first design keep your secrets and data under your control. +--- + +# Security model + +Anton runs on your machine. Your code, files, and queries stay local unless +you explicitly ask Anton to send them somewhere. This page explains the +mechanisms behind that. + +## The credential vault + +Connection credentials are stored in a local vault at `~/.anton/data_vault/` +— one JSON file per connection, with the directory restricted to your user +account and each file readable only by you. + +The key property: **secrets are never placed in LLM prompts.** + +- At run time, credentials are injected as `DS_*` environment variables into + the scratchpad process (for example `DS_PASSWORD`, or namespaced forms like + `DS_POSTGRES_MYDB__PASSWORD` when several connections are loaded). +- The model writes code that reads `os.environ['DS_PASSWORD']` — it sees the + variable *name*, never the value. +- Scratchpad output is scrubbed before it reaches the model: values of + registered secret variables are redacted, so a stray `print` can't leak a + password into the conversation. + +See [Connecting things: overview](/connect/overview) for how credentials get +into the vault in the first place. + +## Scratchpad isolation + +All code Anton writes runs in scratchpads — not in your shell. With the local +backend, each scratchpad is a persistent subprocess running inside its own +dedicated Python virtual environment (created per scratchpad under the +workspace's `.anton/scratchpad-venvs/`, or `~/.anton/scratchpad-venvs/` when +no workspace applies). Packages Anton installs for one task don't pollute +your system Python or other scratchpads, and a scratchpad can be reset to a +clean state at any time. + +## What leaves your machine + +Three things, and only three: + +1. **Prompts to your LLM provider.** Whatever you type, plus the context + Anton assembles, goes to the provider you configured (Anthropic, OpenAI, + Minds, or your own endpoint) and is governed by that provider's terms. +2. **Anonymous analytics.** Event names and timestamps only — no query + content, no personal data. Opt out any time: see + [Analytics](/configure/analytics). +3. **Whatever you ask Anton to send.** Emails, API calls, published + dashboards — actions you request. + +In the other direction, remember that **fetched web content is untrusted +input**: a page Anton reads can contain text designed to manipulate the +model. See [Web fetch](/connect/web-fetch). + +## Windows firewall + +On Windows, the scratchpad's Python needs outbound network access. The +installer offers to add a firewall rule; if you skipped it, run this in an +elevated PowerShell: + +```powershell +netsh advfirewall firewall add rule name="Anton Scratchpad" dir=out action=allow program="$env:USERPROFILE\.anton\scratchpad-venv\Scripts\python.exe" +``` + +## Your responsibility + +Anton acts on your behalf — sending emails, modifying data, calling APIs. As +the first-run terms screen puts it: you're responsible for what Anton does on +your behalf, so review proposed actions before authorizing them. This matters +most for [custom integrations](/connect/custom-integrations), where Anton +writes its own integration code. diff --git a/docs/docs/configure/trace-headers.md b/docs/docs/configure/trace-headers.md new file mode 100644 index 00000000..c1d55402 --- /dev/null +++ b/docs/docs/configure/trace-headers.md @@ -0,0 +1,37 @@ +--- +title: Trace headers +description: Attach Langfuse trace headers to LLM requests on OpenAI-compatible endpoints. +--- + +# Trace headers + +When the planning provider is OpenAI-compatible, Anton can attach trace +headers to its LLM requests so a router or observability proxy can attribute +traces to sessions: + +- `Langfuse-Session-Id` +- `Langfuse-Tags` +- `Langfuse-Metadata` + +This is useful when your requests pass through a gateway that records traces +— for example a self-hosted Langfuse proxy sitting in front of Ollama or +vLLM. + +## Enabling + +To emit the headers against any OpenAI-compatible endpoint, set: + +```bash +export ANTON_LANGFUSE_HEADERS=1 +``` + +Or add it to your workspace config (`.anton/.env`): + +```text +ANTON_LANGFUSE_HEADERS=1 +``` + +The headers only carry session attribution metadata — they don't change what +prompts are sent or where. See +[Environment variables](/configure/env-vars) for the config file loading +order. diff --git a/docs/docs/connect/custom-integrations.md b/docs/docs/connect/custom-integrations.md new file mode 100644 index 00000000..c3d6e51c --- /dev/null +++ b/docs/docs/connect/custom-integrations.md @@ -0,0 +1,59 @@ +--- +title: Custom integrations +description: When there is no connector for something, ask Anton — it builds the integration itself. +--- + +# Custom integrations + +Anton doesn't wait for someone to build a connector. If you need to talk to a +service it has no built-in support for, ask — Anton writes the integration +code itself in the scratchpad, configures it, and gets it running. + +## Example: WhatsApp + +```text +Set up a WhatsApp integration so I can message you from my phone. +``` + +Anton figures out what the integration needs, writes the code, sets it up, +and keeps it running — so you can chat with it from WhatsApp, Telegram, or +whatever channel you need. The same pattern works for any API or tool: you +describe the outcome, Anton builds the plumbing. + +## Custom datasources via `/connect` + +The [connect flow](/connect/overview) also handles services that aren't in +the [built-in catalog](/connect/data-sources). Name any tool or service at the +`/connect` prompt: + +```text +/connect + +(anton) What would you like to connect? +> github +``` + +Anton asks how the service authenticates (a short description, no secrets), +then works out the connection spec itself: the credential fields, the pip +package to use, and a test snippet that verifies the connection. It collects +the credentials conversationally — type `help` on any field for guidance, or +paste several values at once — then tests the connection and saves it. + +The generated engine definition is written to `~/.anton/datasources.md`, so +the new engine behaves like a built-in from then on: it shows up in `/list`, +and `/edit`, `/remove`, and `/test` all work. Removing the last connection +for a custom engine also removes its definition. You can edit the definitions +in that file by hand too — see +[Adding a datasource](/developer/adding-a-datasource) for the format. + +## Set expectations honestly + +- **This is agent-built code.** Anton writes the integration on the fly, + guided by what it knows about the service. Review what the code does — + especially anything that sends messages or modifies data on your behalf. +- **Credentials still go through the vault.** Custom connections use the same + credential vault as built-in engines: secrets are stored locally and + injected as `DS_*` environment variables at run time, never placed in LLM + prompts. See [Security model](/configure/security). +- **Test snippets may be imperfect.** If a generated connection test fails, + Anton shows the error and lets you correct the details or retry. diff --git a/docs/docs/connect/data-sources.md b/docs/docs/connect/data-sources.md new file mode 100644 index 00000000..12718395 --- /dev/null +++ b/docs/docs/connect/data-sources.md @@ -0,0 +1,83 @@ +--- +title: Data sources +description: Catalog of databases, warehouses, vector stores, SaaS apps, and email accounts Anton can connect to out of the box. +--- + +# Data sources + +These are the engines Anton knows how to connect out of the box. For each one +it knows the required fields, how to test the connection, and common failure +modes. Connect any of them with `/connect` — see +[Connecting things: overview](/connect/overview) for the flow. + +## Databases + +| Engine | What you need | +| --- | --- | +| PostgreSQL | host, port (5432), database, user, password; optional schema and SSL | +| MySQL | host, port (3306), database, user, password; optional SSL certificates | +| MariaDB | same as MySQL (wire-compatible, same driver) | +| Microsoft SQL Server | host, port (1433), database, user, password; for Azure SQL use the `server` field (e.g. `myserver.database.windows.net`) | +| Oracle Database | user, password, host, port (1521), and a service name, SID, or full DSN | +| DuckDB | path to a `.duckdb` file or `:memory:`; optional MotherDuck token for cloud databases | +| TimescaleDB | host, port (5432), database, user, password — a PostgreSQL server with the `timescaledb` extension installed | + +## Warehouses + +| Engine | What you need | +| --- | --- | +| Snowflake | account identifier, user, database, plus either a password or a key pair (PEM private key); optional warehouse, schema, role | +| Google BigQuery | GCP project ID and dataset; a service account JSON key (pasted or a file path) with BigQuery Data Viewer and Job User roles | +| Amazon Redshift | cluster endpoint host, port (5439), database, user, password; SSL mode defaults to `require` | +| Databricks | server hostname, HTTP path, and a personal access token (User Settings → Developer → Access Tokens) | + +## Vector stores + +| Engine | What you need | +| --- | --- | +| pgvector | PostgreSQL connection details for a server with the `vector` extension installed (Supabase, Neon, and RDS all support it) | +| ChromaDB | server host and port (8000) for HTTP mode, or a local persist directory; in-memory mode needs nothing | + +## SaaS and CRM + +| Engine | What you need | +| --- | --- | +| Salesforce | username, password, plus consumer key and secret from a connected app (Setup → App Manager) | +| HubSpot | a Private App token (recommended; starts with `pat-na1-`), or OAuth2 client ID and secret | +| Shopify | store URL plus the client ID and client secret of a custom app (Settings → Apps → Develop apps) | +| NetSuite | account ID plus OAuth 1.0a consumer key/secret and token ID/secret from an integration record | +| BigCommerce | API base URL and an API token (Advanced Settings → API Accounts) | +| Comarch Optima (WebArm API) | API base URL and an API key; optional company selector | + +## Email + +| Engine | What you need | +| --- | --- | +| Gmail | your address and a 16-character app password from myaccount.google.com/apppasswords — requires 2-Factor Authentication on the Google account; no OAuth setup | +| Email (generic IMAP/SMTP) | address and password (or app-specific password); IMAP/SMTP server hostnames for non-Gmail providers | + +## Connect, edit, remove, test + +```text +/connect # in chat: pick an engine and enter credentials +anton connect # same flow from your shell +anton connect postgres-mydb # reconnect a saved connection + +/list # show saved connections and their status +/edit postgres-mydb # update credentials (Enter keeps current value) +/remove postgres-mydb # delete from the vault (with confirmation) +/test postgres-mydb # re-run the connection test +``` + +Every engine with a test snippet is verified at connect time: Anton installs +the driver in a scratchpad, runs the test, and reports the exact error if it +fails. Credentials are stored in the local vault and injected as `DS_*` +environment variables at run time — see [Security model](/configure/security). + +## Your own engine definitions + +The built-in catalog is a markdown registry, and you can extend it. Add your +own engine definitions at `~/.anton/datasources.md` — Anton merges them over +the built-ins at startup. Engines you create through the custom-datasource +flow are written there automatically. For the definition format, see +[Adding a datasource](/developer/adding-a-datasource). diff --git a/docs/docs/connect/overview.md b/docs/docs/connect/overview.md new file mode 100644 index 00000000..2bfedebc --- /dev/null +++ b/docs/docs/connect/overview.md @@ -0,0 +1,73 @@ +--- +title: "Connecting things: overview" +description: How Anton connects to databases, warehouses, and apps — and how the credential vault keeps secrets away from the model. +--- + +# Connecting things: overview + +Anton is an open-source AI coworker that can execute tasks, connect to tools +and data, remember lessons, and improve its workflows over time. To work with +your data, it first needs a connection — and the whole flow is conversational. + +## How `/connect` works + +Type `/connect` in a chat session (or run `anton connect` from your shell): + +```text +/connect + +(anton) What would you like to connect? + Examples: PostgreSQL, MySQL, Snowflake, BigQuery, Gmail, GitHub, HubSpot, Salesforce, Jira, REST API. +``` + +1. **Pick an engine.** Name any supported engine from the + [data sources catalog](/connect/data-sources). If Anton doesn't recognize + the name, it offers to set it up as a custom datasource — see + [Custom integrations](/connect/custom-integrations). +2. **Anton collects the fields conversationally.** It shows what the engine + needs, then asks for each value one at a time. You can also paste a + connection string or several `key=value` pairs in one go and Anton extracts + the fields for you. Type `help` for guidance on where to find a credential, + or `skip` to save a partial connection and finish later with `/edit`. +3. **Secrets go into the local credential vault.** Passwords, tokens, and keys + are stored on your machine and are never placed in LLM prompts. At run + time they are injected as environment variables into the scratchpad — the + model only ever sees variable names like `DS_PASSWORD`, never the values. + See [Security model](/configure/security). +4. **Anton tests the connection.** It runs a short test snippet in an isolated + scratchpad (installing the driver if needed). If the test fails, Anton + shows the error and offers to let you re-enter credentials. + +On success the connection is saved under a slug like `postgres-3f2a9c1b` +(engine plus a short generated name) and Anton is ready to query it. + +## The five connection commands + +| In chat | From your shell | What it does | +| --- | --- | --- | +| `/connect` | `anton connect` | Connect a new data source, or pass a saved slug to reconnect without re-entering credentials | +| `/list` | `anton list` | List all saved connections with their status | +| `/edit` | `anton edit NAME` | Update credentials for an existing connection (Enter keeps the current value) | +| `/remove` | `anton remove NAME` | Delete a connection from the vault (asks for confirmation) | +| `/test` | `anton test NAME` | Re-run the connection test for a saved connection | + +`NAME` is the connection slug in `engine-name` format, e.g. `postgres-mydb`. +Running `/remove` with no argument shows a numbered list to pick from. + +## Reconnecting + +Saved connections survive across sessions. To reattach one in a new session, +pass its slug: + +```text +/connect postgres-3f2a9c1b +``` + +or just ask in plain language — "connect to my Gmail and find unanswered +emails" — and Anton finds the credentials in the vault itself. + +## Next steps + +- Browse the full engine catalog: [Data sources](/connect/data-sources) +- Understand where secrets live: [Security model](/configure/security) +- No connector for your tool? [Custom integrations](/connect/custom-integrations) diff --git a/docs/docs/connect/web-fetch.md b/docs/docs/connect/web-fetch.md new file mode 100644 index 00000000..8a295700 --- /dev/null +++ b/docs/docs/connect/web-fetch.md @@ -0,0 +1,51 @@ +--- +title: Web fetch +description: How Anton retrieves the contents of a URL, what to expect from different page types, and why fetched content is untrusted input. +--- + +# Web fetch + +Anton can retrieve the contents of a URL through the `web_fetch` tool, which +is on by default. Like [web search](/connect/web-search), execution depends on +your LLM provider: + +| Provider | `web_fetch` | +| --- | --- | +| Anthropic BYOK | Anthropic native server tool | +| OpenAI BYOK | covered by `web_search` | +| Minds-Enterprise-Cloud (mdb.ai) | mdb.ai passthrough | +| Generic OpenAI-compatible | built-in HTTP GET fallback (no key needed) | + +The fallback needs no API key at all: Anton performs a plain HTTP GET, +follows redirects, and strips HTML down to readable plain text before handing +it to the model. + +## What to expect + +- **30-second timeout.** Slow pages fail rather than hang the conversation. +- **HTML is stripped to plain text.** Works best on article-style pages — + blog posts, documentation, news. +- **Paywalls and JS-heavy single-page apps return little.** The fallback does + not execute JavaScript, so pages that render client-side may come back + nearly empty. + +:::warning Treat fetched page content as untrusted input + +Anything a web page says ends up in the model's context. A malicious page can +embed instructions intended for the model (prompt injection). Anton's +credential vault keeps secrets out of prompts, but you should still be +deliberate about which URLs you ask Anton to fetch, and review any actions it +proposes after reading external content. See +[Security model](/configure/security). + +::: + +## Opting out + +To disable web fetch, set: + +```bash +export ANTON_WEB_FETCH_ENABLED=false +``` + +or add the line to your workspace config (`.anton/.env`). diff --git a/docs/docs/connect/web-search.md b/docs/docs/connect/web-search.md new file mode 100644 index 00000000..9fb1d134 --- /dev/null +++ b/docs/docs/connect/web-search.md @@ -0,0 +1,56 @@ +--- +title: Web search +description: How Anton searches the live web — provider-native when possible, Exa.ai or Brave for generic endpoints. +--- + +# Web search + +Anton can query the live web through the `web_search` tool, which is on by +default. How it executes depends on your LLM provider: + +| Provider | `web_search` | Setup | +| --- | --- | --- | +| Anthropic BYOK | Anthropic native server tool | None — billed on your Anthropic key | +| OpenAI BYOK | OpenAI Responses API native | None — billed on your OpenAI key | +| Minds-Enterprise-Cloud (mdb.ai) | mdb.ai passthrough | None — billed on your Minds key | +| Generic OpenAI-compatible (Together, Groq, Ollama, vLLM, …) | Exa.ai or Brave (you choose) | Run `anton setup-search` once | + +For the first three rows there is nothing to configure — the LLM provider +executes the search server-side and folds the results directly into its +response, billed on the key you already set up. + +## Generic OpenAI-compatible endpoints + +Generic endpoints don't ship a native search capability, so Anton uses an +external search provider — Exa.ai or Brave Search. After `anton setup` +finishes configuring a custom OpenAI-compatible endpoint, Anton offers to set +one up on the spot. You can also run (or re-run) that step at any time: + +```bash +anton setup-search +``` + +You pick a provider, paste an API key, and Anton validates the key with a +probe call before saving it. The key is persisted to `~/.anton/.env`, so it +carries across sessions and workspaces — exactly like your LLM key. See +[Search providers](/configure/search-providers) for the full setup walkthrough +and where to get keys. + +If you skip the step, `web_search` is unavailable on that endpoint until you +run `anton setup-search`. + +## Opting out + +To disable web search entirely, set: + +```bash +export ANTON_WEB_SEARCH_ENABLED=false +``` + +or add the line to your workspace config (`.anton/.env`). + +## Caveats + +- Provider rate limits apply, on both LLM-native and external providers. +- For retrieving the contents of a specific URL, Anton uses a separate tool — + see [Web fetch](/connect/web-fetch). diff --git a/docs/docs/developer/adding-a-datasource.md b/docs/docs/developer/adding-a-datasource.md new file mode 100644 index 00000000..0912e4bb --- /dev/null +++ b/docs/docs/developer/adding-a-datasource.md @@ -0,0 +1,143 @@ +--- +title: Adding a data source +description: How datasource engines are defined in datasources.md — the YAML schema, auth method variants, test snippets, user overrides, and a worked example. +--- + +# Adding a data source + +Anton's datasource catalog is **markdown, not code**. Built-in engines are +defined in `anton/core/datasources/datasources.md`; the parser is +`anton/core/datasources/datasource_registry.py`. Each engine is a level-2 +heading followed by a fenced `yaml` block, with free prose around it that Anton +reads for auth flows, common errors, and where to get tokens. + +To add or override an engine, you usually don't touch the repo at all: put the +same format in `~/.anton/datasources.md` and Anton merges it **over** the +built-in registry at startup (`DatasourceRegistry._load()` parses built-ins +first, then user entries win by engine slug). Fields in user-defined engines +are treated as non-required so partial definitions still work. See +[Custom integrations](/connect/custom-integrations) for the user-facing flow. + +## The YAML block schema + +Parsed into `DatasourceEngine` / `DatasourceField` / `AuthMethod` dataclasses: + +| Key | Type | Meaning | +|---|---|---| +| `engine` | string (required) | Unique slug, e.g. `postgres`. The registry key | +| `display_name` | string | Human name shown in `/connect` | +| `pip` | string | Package to install in the scratchpad before the test snippet runs | +| `name_from` | string or list | Which credential field(s) derive the default connection name (e.g. `database`, or `[account, database]` joined with `_`) | +| `popular` | bool | Surfaces the engine at the top of pickers | +| `fields` | list | The credentials to collect (when there's no auth choice) | +| `auth_method` | `choice` or empty | `choice` means the user picks from `auth_methods` first | +| `auth_methods` | list | Named variants, each with `name`, `display`, and its own `fields` | +| `test_snippet` | multiline string | Python that proves the connection works, reading only `DS_*` env vars | + +Each entry in `fields`: + +| Key | Type | Meaning | +|---|---|---| +| `name` | string | Field name; becomes the `DS_` env var | +| `required` | bool (default true) | Whether collection insists on it | +| `secret` | bool (default false) | Stored in the vault, masked in UIs | +| `description` | string | Shown to the user at collection time | +| `default` | string | Pre-filled value (e.g. `"5432"`) | + +### OAuth2 variants + +An auth method can carry an `oauth2` spec (see the HubSpot entry for the +canonical example): + +```yaml +auth_methods: + - name: oauth2 + display: "OAuth2 (for multi-account or publishable apps)" + fields: + - { name: client_id, required: true, secret: false, description: "OAuth2 client ID" } + - { name: client_secret, required: true, secret: true, description: "OAuth2 client secret" } + oauth2: + auth_url: https://app.hubspot.com/oauth/authorize + token_url: https://api.hubapi.com/oauth/v1/token + scopes: [crm.objects.contacts.read, crm.objects.deals.read] + store_fields: [access_token, refresh_token] +``` + +The prose below the block then tells Anton how to drive the flow with the +scratchpad: build the authorization URL, start a local HTTP server on port 8099 +to catch the callback, open the browser with `webbrowser.open()`, exchange the +`code` at `token_url`, and store the fields listed in `store_fields` in the +vault. + +## The one hard rule: secrets stay in env vars + +Credentials are injected as `DS_*` environment variables before any scratchpad +code runs (namespaced per connection, e.g. `DS_POSTGRES_PROD_DB__HOST`, with +flat `DS_HOST`-style vars during active connection tests — see +`data_vault.py`). **Never embed raw secret values in code strings** — not in +`test_snippet`, not in prose examples. The test snippet must read everything +from `os.environ`. This keeps secrets out of LLM transcripts and episodic logs +(see [Security](/configure/security)). + +## Prose guidance conventions + +After the YAML block, write short prose that Anton will actually use: + +- **Where to get credentials** — e.g. "HubSpot → Settings → Integrations → + Private Apps → Create", or "generate an App Password at + myaccount.google.com/apppasswords". +- **Common errors and what they mean** — e.g. `"password authentication + failed"` → wrong password or user; `"could not connect to server"` → wrong + host/port or firewall. +- **Auth-flow steps** for anything non-trivial (OAuth, key-pair auth). + +## Worked example: a REST API source + +Append this to `~/.anton/datasources.md` (or to the built-in file in a PR): + +````markdown +## OpenWeather + +```yaml +engine: openweather +display_name: OpenWeather +pip: requests +popular: false +fields: + - { name: api_key, required: true, secret: true, description: "OpenWeather API key from home.openweathermap.org/api_keys" } + - { name: units, required: false, secret: false, description: "metric or imperial", default: "metric" } +test_snippet: | + import requests, os + r = requests.get( + "https://api.openweathermap.org/data/2.5/weather", + params={ + "q": "London", + "appid": os.environ['DS_API_KEY'], + "units": os.environ.get('DS_UNITS', 'metric'), + }, + timeout=15, + ) + r.raise_for_status() + print("ok") +``` + +Get a key at home.openweathermap.org/api_keys — free tier allows 60 calls/min. +Common errors: 401 → key not yet activated (takes ~10 minutes after signup); +429 → rate limit exceeded. +```` + +Checklist for the example (and for any new engine): + +1. `engine` slug is unique and lowercase. +2. Every secret field has `secret: true`. +3. `test_snippet` is self-contained, reads only `DS_*` env vars, ends with a + `print("ok")` so success is unambiguous, and fails loudly otherwise. +4. `pip` names the one package the snippet imports (the scratchpad installs it + before running the test). +5. Prose covers token acquisition and the two or three most common errors. + +Validate by restarting Anton (the registry parses both files at startup; +malformed YAML blocks are skipped with a warning on stderr) and running +`/connect` — your engine appears by `display_name`, with fuzzy matching for +typos via `DatasourceRegistry.fuzzy_find()`. See also the +[data sources overview](/connect/data-sources). diff --git a/docs/docs/developer/adding-a-tool.md b/docs/docs/developer/adding-a-tool.md new file mode 100644 index 00000000..3f1c3e35 --- /dev/null +++ b/docs/docs/developer/adding-a-tool.md @@ -0,0 +1,148 @@ +--- +title: Adding a tool +description: Step-by-step walkthrough for adding a built-in tool — ToolDef, handler signature, registration — and when a tool is warranted at all. +--- + +# Adding a tool + +## First: do you actually need a tool? + +Anton's philosophy is that **most work goes through the scratchpad**. The agent +doesn't need a `read_csv` tool or a `http_get` tool — it writes Python in a +scratchpad cell and gets exactly the capability it needs, with credentials +already injected as `DS_*` env vars and an LLM bridge available via `get_llm()`. + +Tools are reserved for **primitives the model must call directly** — operations +that: + +- need to run in the **main process**, not the scratchpad subprocess + (e.g. `memorize` writes through the Cortex; `recall_skill` reads the + session's skill store); +- must return content the model consumes **as part of the conversation**, not + as program output (e.g. `read_image` returns vision blocks); +- are part of the **turn protocol itself** (e.g. `create_artifact` claims a + folder the renderer watches). + +If your idea is "let Anton do X with an API", the answer is almost always a +[data source definition](/developer/adding-a-datasource) plus scratchpad code — +not a tool. A senior reviewer will push back on tool proposals that the +scratchpad already covers. + +## The pieces + +A tool is three things (see [Tool system](/developer/tool-system) for the full +machinery): + +1. A `ToolDef` — name, LLM-facing description, JSON `input_schema`, and a + handler (defined in `anton/core/tools/tool_defs.py`, or in a dedicated file + like `recall_skill.py`). +2. An async **handler** with the signature + `async def handle_mytool(session: "ChatSession", tc_input: dict) -> str` — + it receives the live session and the tool-call input dict, and returns the + result string sent back to the LLM (vision tools may return a list of + content blocks instead). +3. **Registration** in `ChatSession._build_core_tools()` + (`anton/core/session.py`), optionally guarded by a condition (workspace + bound, episodic enabled, ...). Embedding hosts can instead pass extra + ToolDefs via the session's `_extra_tools`. + +## Walkthrough: `recall_skill.py`, the cleanest template + +`anton/core/tools/recall_skill.py` is the best file to copy because the +definition, schema, handler, and docs live together in ~130 lines. + +**1. The description — written for the LLM, not for humans:** + +```python +_DESCRIPTION = ( + "Retrieve a procedural skill from long-term memory into your working " + "context. Call this when you recognize that one of the skills listed in " + "the '## Procedural memory' section of your system prompt applies to the " + "user's current request. ..." +) +``` + +Say *when* to call the tool and what comes back. Bad descriptions are the +number-one cause of tools being over- or under-used. + +**2. The input schema — plain JSON Schema:** + +```python +_INPUT_SCHEMA = { + "type": "object", + "properties": { + "label": { + "type": "string", + "description": "The skill label to recall, e.g. 'csv_summary'. ...", + }, + }, + "required": ["label"], +} +``` + +**3. The handler — validate defensively, return strings, never raise:** + +```python +async def handle_recall_skill(session: "ChatSession", tc_input: dict) -> str: + label_in = (tc_input.get("label") or "").strip() + if not label_in: + return "ERROR: recall_skill requires a non-empty 'label' parameter. ..." + + store = getattr(session, "_skill_store", None) + if store is None: + return "ERROR: no skill store is wired into this session. ..." + + skill = store.load(label_in) + # ... typo recovery via store.closest_match(label_in) ... + store.increment_recommended(skill.label, stage=1) + return _format_skill_response(skill, warning=warning) +``` + +Handler conventions visible here: + +- **Read session state via `getattr` with a fallback** — sessions can be built + without every subsystem, and a tool must degrade to a clear error string, + not an `AttributeError`. +- **Return errors as strings the LLM can act on** ("Available skills: ...") — + exceptions from handlers turn into opaque failures; informative strings let + the model self-correct on the next round. +- **Capture mechanical signals in the handler** (the `recommended` counter), + not via LLM conventions. + +**4. The ToolDef:** + +```python +RECALL_SKILL_TOOL = ToolDef( + name="recall_skill", + description=_DESCRIPTION, + input_schema=_INPUT_SCHEMA, + handler=handle_recall_skill, +) +``` + +**5. Registration** — in `ChatSession._build_core_tools()`: + +```python +# Procedural memory retrieval — always available, no-op if no skills. +self.tool_registry.register_tool(RECALL_SKILL_TOOL) +``` + +If your tool only makes sense under a condition, guard it the way `recall` +(episodic enabled) and the artifact tools (workspace bound) are guarded — +hiding a tool entirely beats registering one that returns errors. + +## Checklist + +1. Define `_DESCRIPTION`, `_INPUT_SCHEMA`, the handler, and the `ToolDef` — + ideally in one new file under `anton/core/tools/`. +2. Register it in `_build_core_tools()` with the right guard. +3. Make the handler total: every input shape returns a string; no path raises. +4. Keep results within reason — large outputs bloat history (episodic logging + truncates tool results at 2000 chars; your tool result itself goes into the + LLM history uncut, so truncate big payloads yourself). +5. Optional: set `ToolDef.prompt` if the tool needs a system-prompt fragment + (it is appended by the prompt builder; most tools don't need this). +6. Add tests under `tests/` — handler unit tests can call + `await handle_mytool(fake_session, {...})` directly; see + `tests/test_acc.py` and friends for session-faking patterns, and + [Contributing](/developer/contributing) for how to run the suite. diff --git a/docs/docs/developer/architecture.md b/docs/docs/developer/architecture.md new file mode 100644 index 00000000..f8611b5c --- /dev/null +++ b/docs/docs/developer/architecture.md @@ -0,0 +1,143 @@ +--- +title: Architecture overview +description: The contributor's map of Anton — executive, scratchpads, long-term stores, how a turn flows, and where each subsystem lives in the code. +--- + +# Architecture overview + +Anton has a brain-inspired architecture. The user docs deliberately use plain names +(memory, lessons, skills); in the contributor docs we use the real module names — +hippocampus, cortex, cerebellum, ACC, striatum — because the code does. If you want +the full rosetta stone, start with [Brain mapping](/developer/brain-mapping). + +![Anton architecture](/img/anton-diagram.png) + +At the highest level there are three layers: + +- **The executive (orchestrator)** — the chat loop plus `ChatSession`. It plans, + picks tools, monitors progress, and decides when to stop. +- **Scratchpads (working memory)** — isolated Python reasoning environments where + most actual work happens. See [Scratchpad runtime](/developer/scratchpad-runtime). +- **Long-term stores** — the experience store (episodic memory), the declarative + memory files (engrams), the cerebellum/ACC error-learning loop, and the skill + library (procedural memory). + +``` + ┌────────────────────────────────────────────────────┐ + │ EXECUTIVE (the orchestrator) │ + │ │ + │ On new problem: │ + │ 1. Check SKILL LIBRARY → match? │ + │ YES → recall_skill(label) → load procedure │ + │ NO → open fresh scratchpad │ + │ 2. Monitor scratchpad progress │ + │ 3. Detect stuck/failure → pivot strategy │ + │ 4. On success → record to experience store │ + └────────────┬────────────────────↑──────────────────┘ + │ spawns & monitors │ + ▼ │ + ┌──────────────────────────────────────────────────────┐ + │ SCRATCHPADS (working memory) │ + │ │ + │ Each scratchpad is: │ + │ - An isolated reasoning environment (its own venv) │ + │ - A chain-of-thought trace (code + observations) │ + │ - Has a goal, constraints, and a budget │ + │ - Can request sub-scratchpads (decomposition) │ + │ │ + │ Every cell execution fires pre/post hooks observed │ + │ by the CEREBELLUM (post-mortem error learning). │ + └──────┬──────────────┬───────────────────┬────────────┘ + │ │ │ + │ on success │ on cell errors │ on success + ▼ ▼ ▼ + ┌────────────┐ ┌──────────────┐ ┌─────────────────────┐ + │ EXPERIENCE │ │ CEREBELLUM │ │ SKILL LIBRARY │ + │ STORE │ │ │ │ │ + │ (hipp.) │ │ Buffers bad │ │ /skill save → LLM │ + │ │ │ cells, runs │ │ drafts a procedure │ + │ Episodes — │ │ post-mortem │ │ with label + name + │ + │ JSONL log │ │ via LLM, │ │ when_to_use + │ + │ of every │ │ encodes new │ │ declarative_md. │ + │ turn. │ │ lessons via │ │ │ + │ │ │ Cortex. │ │ Future turns recall │ + │ Recall via │ │ │ │ the procedure via │ + │ `recall` │ │ Lessons feed │ │ recall_skill tool. │ + │ tool. │ │ next code │ │ │ + │ │ │ generation │ │ Stored at │ + │ │ │ (procedural │ │ ~/.anton/skills/ │ + │ │ │ priming). │ │