JARVIS - Digital Life Context

An Electron app that captures screenshots every 30 seconds, analyzes them with AI via OpenRouter, and extracts structured digital life context.

Features

• Screenshots every 30 seconds - Automatic screen capture
• AI Analysis via OpenRouter - Uses GPT-4o-mini for vision analysis
• Accessibility Integration - Gets active app name, window title, browser URLs
• Artifact Extraction - Names, emails, URLs, code snippets, files, tasks
• Activity Prediction - Predicts what you'll do next
• Beautiful B&W UI - Built with React + shadcn/ui + Tailwind

Setup

1. Install dependencies:

   npm install

2. Configure OpenRouter API key:

   cp .env.example .env
   # Edit .env and add your OpenRouter API key
   # Get one at https://openrouter.ai/keys

3. Run in development mode:

   npm run dev

4. Grant permissions on first run:

   • Screen Recording: System Preferences > Privacy & Security > Screen Recording
   • Accessibility: System Preferences > Privacy & Security > Accessibility
   • Restart the app after granting permissions

Build for Production

npm run build:mac

Output Structure

Each captured context includes:

{
  "currentActivity": {
    "application": "VS Code",
    "task": "writing code",
    "category": "development",
    "description": "Editing a JavaScript file..."
  },
  "activeWindow": {
    "app": "Code",
    "bundleId": "com.microsoft.VSCode",
    "title": "main.js - JARVIS",
    "windowCount": 2
  },
  "predictedNextAction": {
    "action": "run tests",
    "confidence": "high",
    "reasoning": "User just finished implementing a function"
  },
  "artifacts": {
    "names": ["John Doe"],
    "emails": ["john@example.com"],
    "urls": ["https://github.com/..."],
    "codeSnippets": ["function example() {...}"],
    "files": ["main.js", "package.json"],
    "dates": ["Feb 8, 2026"],
    "tasks": ["Fix login bug"]
  },
  "context": {
    "mood": "focused",
    "workContext": "coding",
    "projectHints": ["JARVIS"]
  },
  "summary": "User is coding a new feature in VS Code"
}

Data Storage

Data is stored in ~/Library/Application Support/jarvis/data/:

• contexts.json - All captured contexts
• artifacts.json - Aggregated artifacts with counts
• screenshots/ - PNG screenshots

Privacy

• All data is stored locally
• Screenshots are only sent to OpenRouter API for analysis
• No data is shared with third parties beyond OpenRouter
• Sensitive data (passwords) is filtered by AI