JARVIS

Just A Rather Very Intelligent System

An always-on autonomous AI daemon with desktop awareness, multi-agent hierarchy, visual workflows, and goal pursuit.

JARVIS is not a chatbot with tools. It is a persistent daemon that sees your screen, understands what you're doing, and acts — within the authority limits you define. Run it on a server for 24/7 availability, then connect sidecars on your laptop, desktop, or any other machine to give it eyes and hands everywhere.

🔍 What Makes JARVIS Different

Feature	Typical AI Assistant	JARVIS
Always-on	No — request/response only	Yes — persistent daemon, runs 24/7 on a server or locally
Reach across machines	No — single machine only	Yes — one daemon, unlimited sidecars on any machine
Desktop awareness	No	Yes — screen capture every 5-10s via sidecar
Native app control	No	Yes — Go sidecar with Win32/X11/macOS automation
Multi-agent delegation	No	Yes — 9 specialist roles
Visual workflow builder	No	Yes — 50+ nodes, n8n-style
Voice with wake word	No	Yes — streaming TTS + openwakeword
Goal pursuit (OKRs)	No	Yes — drill sergeant accountability
Authority gating	No	Yes — runtime enforcement + audit trail
LLM provider choice	Usually locked to one	5 providers: Anthropic, OpenAI, Gemini, Ollama, Groq

⚡ Quick Start

bun install -g @usejarvis/brain   # Install the daemon
jarvis onboard                    # Interactive setup wizard
jarvis start -d                   # Start as background daemon

Open http://localhost:3142 — your dashboard is ready.

☁️ Managed Hosting

Don't want to deal with servers, DNS, or TLS certificates? We've partnered with opencove.host — a managed hosting platform built specifically for JARVIS.

No self-hosting hassle — no server to provision, no dependencies to install
Dedicated domain included — no need to buy a domain or configure DNS and TLS
Up and running in under 5 minutes — spin up your JARVIS instance and start using it immediately

Visit opencove.host to get started.

💡 Use Cases

Research while you work — Ask JARVIS to deep-dive a topic. It runs browser searches, reads pages, and compiles a summary in the background while you focus on other things.

Automate across machines — Run the daemon on your home server. Connect sidecars on your work laptop and your desktop. JARVIS can move files between them, run scripts on your server, and open apps on your laptop — all from one conversation.

Inbox triage — Set up a workflow that monitors your Gmail, categorizes incoming messages, drafts replies for your review, and schedules follow-ups on your calendar.

Desktop co-pilot — JARVIS watches your screen via the sidecar. If it sees you struggling with an error message or a complex form, it proactively offers help or fills in fields for you.

Goal accountability — Define OKRs in the Goals dashboard. JARVIS plans your day each morning, checks in during the evening, and escalates if you're falling behind — like a personal drill sergeant.

Multi-step workflows — Build visual automations with 50+ node types: "when a file appears in this folder, OCR it, extract key data, update the spreadsheet, and notify me on Telegram."

📋 Requirements

Bun >= 1.0 (installed automatically if missing)
OS (native daemon install): macOS, Linux, or WSL
Windows: use WSL2 for the Bun install, or Docker for the daemon
LLM API key — at least one of: Anthropic, OpenAI, Google Gemini, or a local Ollama instance
Google OAuth credentials (optional — Calendar and Gmail integration)
Telegram bot token (optional — notification channel)
Discord bot token (optional — notification channel)
ElevenLabs API key (optional — premium TTS)

📦 Installation

bun (recommended)

bun install -g @usejarvis/brain
jarvis onboard

Note: Native Windows installs are blocked for the JARVIS daemon. On Windows, use WSL2 for the Bun install above, or use the Docker install instead.

Docker

Run JARVIS on any OS with a single command — no Bun or dependencies required. Install Docker Desktop (Windows, macOS, Linux) if you don't have Docker yet.

docker run -d --name jarvis \
  -p 3142:3142 \
  -v jarvis-data:/data \
  -e JARVIS_API_KEY=sk-ant-your-key \
  ghcr.io/vierisid/jarvis:latest

The image is available on GHCR. Configuration can be provided via environment variables or by mounting a config.yaml into the /data volume.

Note: Docker runs in an isolated container, so the daemon inside it cannot access your host desktop, browser, or clipboard directly. You must still install the sidecar on each machine where you want JARVIS to have desktop awareness and automation capabilities.

One-liner

curl -fsSL https://raw.githubusercontent.com/vierisid/jarvis/main/install.sh | bash
jarvis onboard

The install script sets up Bun, clones the repo, and links the jarvis CLI. Then run jarvis onboard to configure your assistant interactively.

Note: The one-liner only supports macOS, Linux, and WSL. Native Windows shells such as PowerShell, Git Bash, and CMD should use WSL2 or the Docker install instead.

Manual

git clone https://github.com/vierisid/jarvis.git ~/.jarvis/daemon
cd ~/.jarvis/daemon
bun install
bun run build:ui
jarvis onboard

🚀 Usage

jarvis start            # Start in foreground
jarvis start -d         # Start as background daemon
jarvis start --port 3142 # Start on a specific port
jarvis stop             # Stop the daemon
jarvis status           # Check if running
jarvis doctor           # Verify environment & connectivity
jarvis logs -f          # Follow live logs

The dashboard is available at http://localhost:3142 once the daemon is running.

Updating

jarvis update detects how you installed JARVIS and runs the right update command. Equivalent manual commands per install method:

Install method	`jarvis update` dispatches to
Bun global (`bun install -g @usejarvis/brain`)	`bun update -g @usejarvis/brain`
`install.sh` (git clone under `~/.jarvis/daemon`)	`git pull --ff-only` + `bun install`
Docker	Refused — run `docker pull <image> && docker rm -f jarvis && docker run ...` on the host
Developer checkout	Refused — run `git pull` yourself

Run jarvis doctor to see what was detected and the exact commands for your install.

Removing JARVIS

jarvis uninstall stops the daemon, removes autostart hooks, deletes ~/.jarvis, and — where applicable — runs the correct package-manager uninstall. It does not touch sidecars.

Install method	`jarvis uninstall` dispatches to
Bun global	`bun uninstall -g @usejarvis/brain` + side-effect cleanup
`install.sh`	`rm -rf ~/.jarvis/daemon` + CLI wrapper + side-effect cleanup
Docker	Refused — run `docker rm -f jarvis` (and optionally `docker volume rm jarvis-data`) on the host
Developer checkout	Side-effect cleanup only — your checkout is left in place

Tip: If you already ran bun uninstall -g @usejarvis/brain without going through jarvis uninstall, your daemon may still be running and ~/.jarvis is still on disk. Run jarvis doctor before uninstalling to see what will be cleaned up, or stop the daemon and remove ~/.jarvis manually.

🖥️ Sidecar Setup

The sidecar is what gives JARVIS physical reach beyond the machine it runs on. It is a lightweight agent that you install on any machine — your laptop, a dev server, a home PC — and it connects back to the central daemon over an authenticated WebSocket. Each sidecar gives JARVIS access to that machine's desktop, browser, terminal, filesystem, clipboard, and screenshots.

This means you can run the daemon on an always-on server and still interact with your desktop machines as if JARVIS were running locally. Enroll as many sidecars as you want.

1. Install the sidecar

Via bun:

bun install -g @usejarvis/sidecar

Or download the binary from GitHub Releases for your platform (macOS, Linux, Windows).

2. Enroll in the dashboard

Open the JARVIS dashboard at http://localhost:3142
Go to Settings → Sidecar
Enter a friendly name for this machine (e.g. "work laptop") and click Enroll
Click Copy to copy the token command

3. Run the sidecar

Paste and run the copied command on the machine where you installed the sidecar:

jarvis-sidecar --token <your-token>

The sidecar saves the token locally, so on subsequent runs you just need:

jarvis-sidecar

Once connected, the sidecar appears as online in the Settings page where you can configure its capabilities (terminal, filesystem, desktop, browser, clipboard, screenshot, awareness).

🧠 Core Capabilities

Conversations — Multi-provider LLM routing (Anthropic Claude, OpenAI GPT, Google Gemini, Ollama). Streaming responses, personality engine, vault-injected memory context on every message.

Tool Execution — 14+ builtin tools with up to 200 iterations per turn. The agent loop runs until the task is complete, not until the response looks done.

Memory & Knowledge — Vault knowledge graph (entities, facts, relationships) stored in SQLite. Extracted automatically after each response. Injected into the system prompt so JARVIS always remembers what matters.

Browser Control — Auto-launches Chromium via CDP. 7 browser tools handle navigation, interaction, extraction, and form filling.

Desktop Automation — Go sidecar with JWT-authenticated WebSocket, RPC protocol, and binary streaming. Win32 API automation (EnumWindows, UIAutomation, SendKeys) on Windows, X11 tools on Linux.

Multi-Agent Hierarchy — delegate_task and manage_agents tools. An AgentTaskManager coordinates 9 specialist roles. Sub-agents are denied governed actions — authority stays with the top-level agent.

Voice Interface — Edge TTS or ElevenLabs with streaming sentence-by-sentence playback. Binary WebSocket protocol carries mic audio (WebM) and TTS audio (MP3) on the same connection. Wake word via openwakeword (ONNX, runs in-browser).

Continuous Awareness — Full desktop capture at 5-10 second intervals. Hybrid OCR (Tesseract.js) + Cloud Vision. Struggle detection, activity session inference, entity-linked context graph. Proactive suggestions and an overlay widget.

Workflow Automation — Visual builder powered by @xyflow/react. 50+ nodes across 5 categories. Triggers: cron, webhook, file watch, screen events, polling, clipboard, process, git, email, calendar. NL chat creation, YAML export/import, retry + fallback + AI-powered self-heal.

Goal Pursuit — OKR hierarchy (objective → key result → daily action). Google-style 0.0-1.0 scoring. Morning planning, evening review, drill sergeant escalation. Awareness pipeline auto-advances progress. Three dashboard views: kanban, timeline, metrics.

Authority & Autonomy — Runtime enforcement with soft-gate approvals. Multi-channel approval delivery (chat, Telegram, Discord). Full audit trail. Emergency pause/kill controls. Consecutive-approval learning suggests auto-approve rules.

🎛️ Dashboard

Built with React 19 and Tailwind CSS 4. Served by the daemon at http://localhost:3142.

Page	Purpose
Chat	Primary conversation interface with streaming
Tasks	Active commitments and background work queue
Content Pipeline	Multi-step content generation and review
Knowledge Graph	Visual vault explorer — entities, facts, relationships
Memory	Raw vault search and inspection
Calendar	Google Calendar integration with scheduling tools
Agent Office	Multi-agent delegation status and role management
Command Center	Tool history, execution logs, proactive notifications
Authority	Approval queue, permission rules, audit trail
Awareness	Live desktop feed, activity timeline, suggestions
Workflows	Visual builder, execution monitor, version history
Goals	OKR dashboard — kanban, timeline, and metrics views
Settings	LLM providers, TTS/STT, channels, behavior config

⚙️ Configuration

JARVIS stores its configuration at ~/.jarvis/config.yaml. Run jarvis onboard for interactive setup — it walks through LLM provider, voice, channels, personality, and authority settings.

daemon:
  port: 3142
  data_dir: "~/.jarvis"
  db_path: "~/.jarvis/jarvis.db"

llm:
  primary: "anthropic"
  fallback: ["openai", "gemini", "ollama"]
  anthropic:
    api_key: "sk-ant-..."
    model: "claude-sonnet-4-6"

personality:
  core_traits: ["loyal", "efficient", "proactive"]
  assistant_name: "Jarvis"

authority:
  default_level: 3

active_role: "personal-assistant"

See config.example.yaml for the full reference including Google OAuth, Telegram, Discord, ElevenLabs, and voice settings.

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                     JARVIS Daemon                           │
│                  (server or local machine)                  │
│                                                             │
│  ┌──────────┐  ┌──────────┐  ┌───────────┐  ┌───────────┐   │
│  │ LLM      │  │ Vault    │  │ Agent     │  │ Workflow  │   │
│  │ Router   │  │ Memory   │  │ Manager   │  │ Engine    │   │
│  └──────────┘  └──────────┘  └───────────┘  └───────────┘   │
│  ┌──────────┐  ┌──────────┐  ┌───────────┐  ┌───────────┐   │ 
│  │ Tool     │  │ Authority│  │ Goal      │  │ Awareness │   │
│  │ Executor │  │ Engine   │  │ Tracker   │  │ Pipeline  │   │
│  └──────────┘  └──────────┘  └───────────┘  └───────────┘   │
│                                                             │
│  ┌──────────────────────────────────────────────────────┐   │
│  │ Bun.serve() — HTTP + WebSocket + Dashboard (React)   │   │
│  └──────────────────────────────────────────────────────┘   │
└──────────────┬──────────────────────┬───────────────────────┘
               │ JWT-auth WebSocket   │
       ┌───────┴───────┐       ┌──────┴────────┐
       │  Sidecar #1   │       │  Sidecar #2   │      ...
       │  (laptop)     │       │  (dev server) │
       │               │       │               │
       │  desktop      │       │  terminal     │
       │  browser      │       │  filesystem   │
       │  terminal     │       │  screenshots  │
       │  clipboard    │       │               │
       └───────────────┘       └───────────────┘

The daemon is the brain — it holds the LLM connections, memory vault, agent hierarchy, and all decision-making. It can run on a home server, a VPS, or your local machine.

Sidecars are the hands. Each sidecar is a lightweight Go binary that connects to the daemon and exposes its host machine's capabilities. The daemon can orchestrate actions across all connected sidecars simultaneously. Sidecars authenticate via JWT and communicate over a binary WebSocket protocol.

This separation means JARVIS stays reachable 24/7 on a server while still being able to see your screen, type in your apps, and manage files on any machine where a sidecar is running.

🛠️ Development

bun test                # Run all tests (379 tests across 22 files)
bun run dev             # Hot-reload daemon
bun run build:ui        # Rebuild dashboard
bun run db:init         # Initialize or reset the database

Stack

Runtime: Bun (not Node.js)
Language: TypeScript (ESM)
Database: SQLite via bun:sqlite
UI: React 19, Tailwind CSS 4, @xyflow/react
LLM: Anthropic Claude, OpenAI GPT, Google Gemini, Ollama
Desktop sidecar: Go (JWT auth, WebSocket RPC, platform-specific automation)
Voice: openwakeword (ONNX), Edge TTS / ElevenLabs
Package: @usejarvis/brain (published to npm registry, installable via bun)

🗺️ Roadmap

16 milestones completed — LLM conversations, tool execution, memory vault, browser control, proactive agent, dashboard UI, multi-agent hierarchy, communication channels, native app control, voice interface, authority & autonomy, distribution & onboarding, continuous awareness, workflow automation, plugin ecosystem, and autonomous goal pursuit.

379 tests passing across 22 test files. ~65,000 lines of TypeScript + Go.

Upcoming

Milestone	Description
Smart Home	Home Assistant integration
Financial Intelligence	Plaid, portfolio tracking
Mobile Companion	React Native dashboard
Self-Improvement	Autonomous prompt evolution
Multi-Modal	DALL-E 3, full video/image processing
Swarm Intelligence	Multi-device coordination

See VISION.md for the full roadmap with detailed specifications.

📖 Documentation

VISION.md — Full roadmap and milestone specifications
docs/LLM_PROVIDERS.md — LLM provider configuration
docs/WORKFLOW_AUTOMATION.md — Workflow engine guide
docs/VAULT_EXTRACTOR.md — Memory and knowledge vault
docs/PERSONALITY_ENGINE.md — Personality and role system
config.example.yaml — Full configuration reference

💬 Community

Discord — Chat with other users, ask questions, share workflows
Website — Project homepage and documentation
GitHub Issues — Bug reports and feature requests

🔒 Security

JARVIS includes a built-in authority engine that gates every action at runtime. All tool executions are logged in an audit trail, and sensitive operations require explicit approval via the dashboard, Telegram, or Discord. Emergency pause and kill controls are always available.

If you discover a security vulnerability, please report it privately by emailing the maintainer rather than opening a public issue.

📄 License

Jarvis Source Available License 2.0 (based on RSALv2)

Name		Name	Last commit message	Last commit date
Latest commit History 166 Commits
.githooks		.githooks
.github		.github
bin		bin
docs		docs
examples		examples
roles		roles
scripts		scripts
sidecar		sidecar
src		src
ui		ui
webapp-templates		webapp-templates
.dockerignore		.dockerignore
.gitattributes		.gitattributes
.gitignore		.gitignore
.npmignore		.npmignore
Dockerfile		Dockerfile
LICENSE		LICENSE
QUICKSTART.md		QUICKSTART.md
README.md		README.md
VISION.md		VISION.md
bun.lock		bun.lock
config.example.yaml		config.example.yaml
install.sh		install.sh
package.json		package.json
tsconfig.json		tsconfig.json

Folders and files

Latest commit

History

Repository files navigation