Email Sorter

A comprehensive, automated email sorting and classification pipeline designed to handle gigabytes of mail offline using heuristics and local language models via OpenAI-compatible APIs (like LM-Studio, Ollama, or vLLM) with multiple llm machine runners.

Prerequisites

1. uv installed.
2. A local or remote LLM server exposing an OpenAI-compatible API (e.g., LM-Studio running locally).
3. A Google Takeout .mbox export, OR a Fastmail API token.
4. Optional for browser-based unsubscribe fallback: Playwright Chromium installed with npx playwright install chromium.

Installation

You can install the tool globally using uv:

uv tool install .

This will expose the email-sort CLI command.

Alternatively, you can run the tool directly from the source using uv run:

uv run email-sort [command]

Pipeline Steps

1. Configure the Pipeline

Copy the example configuration to set up your tokens and server details.

cp conf.example.toml conf.toml

Edit conf.toml to include your Fastmail token (if using Fastmail) and your LLM server details (URLs, worker counts, optional names, and model name).

2. Initialize the Database

Create the SQLite database to store all email metadata and classification state.

email-sort init-db

3. Ingest Mail

For MBOX (e.g., Google Takeout): Extract your .mbox file and run:

email-sort ingest mbox path/to/your/Takeout.mbox

For Fastmail: Ensure your fastmail_token is set in conf.toml and run:

email-sort ingest fastmail

4. Run Fast Heuristics

This step uses fasttext to detect foreign languages instantly and flags obvious automated/non-personal emails based on recipients. It will download a lightweight language model the first time it runs.

email-sort heuristics

5. Run LLM Classification

Make sure your LLM servers are running. This step uses your configured models to categorize all the emails that couldn't be instantly classified by the heuristics.

email-sort classify

You can monitor the progress of the classification in another terminal:

email-sort watch

The script will automatically load your configuration from conf.toml and distribute tasks across all enabled servers in your cluster.

6. Export Results

Generate CSV files containing actionable insights (e.g., senders to ban, and subscriptions to cancel).

email-sort export

This produces ban_list.csv and unsubscribe_list.csv in your current directory.

7. Review and Execute Unsubscribes

Start the local dashboard after ingesting, running heuristics, classifying, and analyzing senders:

npm install
npm run ts:dashboard

The TypeScript dashboard reuses the same SQLite database and unsubscribe_requests table, keeps the Python classification/batch jobs unchanged, and runs on port 8766 by default. It loads immediately, then scans recent unsubscribe-linked messages in the background with a progress bar. It shows candidates in a compact table with bulk selection controls and button-click email previews. The page continuously checks the local server and shows a visible offline/restarting banner if API calls fail, so silent exits are obvious in the Web UI. English senders that look marketing-only and are getting past spam/junk filters are marked recommended, but are still unchecked by default; use Select recommended when you want to bulk opt in. Mixed senders with transactional/account-like mail, such as receipts, delivery notices, invoices, payment alerts, password resets, or security messages, are shown for review but are not checked by default. Non-English senders and senders already caught by spam/junk folders are shown in separate statuses and left unselected. Executed unsubscribe requests are tracked in SQLite with method, URL, result, and logs; after a dashboard job finishes, rows update immediately and the table refreshes in the background without blanking the current results. Successful senders are hidden from the working table on later refreshes/rescans. The Keep checked senders action records a durable kept decision so those senders are also hidden/not preselected on refresh. Failed attempts are recorded as attempted-failed for later debugging/retry.

Dashboard scans are cached on disk and reused across page reloads while the mail database/schema state is unchanged. Unsubscribe request status is applied as a lightweight SQLite overlay, so recording a success/failure does not force a full candidate rescan. When a grouped sender has multiple distinct RFC8058 one-click POST URLs across its messages, the executor posts all of those one-click URLs before falling back to slower methods. Exact duplicate method/URL actions are deduped within a candidate and reused across a dashboard job, so the browser fallback should not repeatedly open the same unsubscribe URL. Click Refresh scan to force a rebuild. Full-email previews strip unsafe/noisy HTML such as scripts, styles, headers, tracking images, and external embeds, then render simple local paragraphs.

For new mail, keep ingesting and classifying incrementally, then refresh the dashboard:

email-sort ingest imap --watch
email-sort heuristics
email-sort classify
email-sort analyze-senders
npm run ts:dashboard

The TypeScript unsubscribe agent is available for non-dashboard runs and is the path the dashboard executor uses:

npm run ts:unsubscribe -- --dry-run
npm run ts:unsubscribe -- --execute --yes

The legacy Python CLI executor remains in place until it is removed:

email-sort unsubscribe --dry-run
email-sort unsubscribe --execute

Unsubscribe execution has configurable local-only safety guardrails ([unsubscribe] max_per_hour, max_per_day) so a bug or bad selection cannot blast many unsubscribe endpoints at once. These defaults are intentionally high for large cleanup runs and are not provider/external rate limits. When the local safety limit is reached, dashboard execution pauses the batch and leaves remaining candidates untouched for a later run.