[Feature]: Email notifications - admin SMTP settings, user notification preferences, branded emails

[cjimti]

Problem Statement

When a user shares an asset, collection, or prompt with another user, the recipient is never told. The share is recorded in portal_shares.shared_with_email and surfaces in the portal's "shared with me" view, but there is no out-of-band signal, so recipients only discover shared items if they happen to go looking. The same gap applies to thread comments and feedback (epic #600): a teammate can comment on something you own and you will not know.

This ticket builds the notification substrate the platform is missing: an admin SMTP configuration surface, a per-user notification preference store, a delivery worker (immediate or daily digest), and branded HTML emails. It is the prerequisite that #627 (@mention tagging on thread comments) calls out as out of its own scope.

Goals

1. Admin can configure SMTP from the portal, in a settings surface designed to hold future global settings (not SMTP-only).
2. Each user can manage notification preferences: turn notifications off entirely, receive them immediately, or batch them into a daily digest.
3. Recipients get a well-designed, portal-branded HTML email when something is shared with them and when a thread comment/feedback event targets them.
4. Email delivery never blocks the share/comment request path and survives a pod restart (queued + retried, not fire-and-forget).

Verified Current State

• No email/SMTP code exists anywhere in the repo. grep -ri "smtp\|net/smtp\|sendmail" over pkg/ returns nothing but data fields. This is greenfield.
• Share recipient is already persisted. Direct user shares set shared_with_email and run through ShareStore.Insert() at three handlers:
  • asset: pkg/portal/handler.go (createShare, ~line 1253)
  • collection: pkg/portal/collection_handler.go (createCollectionShare, ~line 716)
  • prompt: pkg/portal/prompt_handler.go (createPromptShare, ~line 635)
    These three Insert call sites are the share trigger points.
• Thread events exist. pkg/portal/thread_handler.go (createThread ~line 210, appendThreadEvent ~line 348) write to portal_threads / portal_thread_events. These are the comment/feedback trigger points.
• Settings already have a DB-backed home. config_entries (migration 000026) + pkg/configstore is the admin-editable-config pattern, with an audit changelog and a file-mode vs database-mode gate (writes return 405 in file mode). Admin config routes live in pkg/admin/handler.go / config_handler.go.
• No per-user preferences table exists. users (migration 000067) is an admin-maintained directory (name, confirmed, source) only.
• Encryption helper exists. pkg/platform/fieldcrypt.go FieldEncryptor (AES-256-GCM, NewFieldEncryptor(key []byte), EncryptSensitiveFields/DecryptSensitiveFields) already encrypts connection secrets. The SMTP password reuses this, not a new scheme.
• Background-worker pattern exists. pkg/indexjobs/worker.go polls a DB queue with a time.Ticker fallback plus LISTEN/NOTIFY wakeup and lease-based locking. The digest scheduler and the send worker follow this pattern.
• Branding is configurable per deployment. PortalConfig (pkg/platform/config.go ~196-229) carries Title, light/dark logos, and Implementor (name/logo/url); resolved into SVGs at startup (cmd/mcp-data-platform/main.go ~620-653). Email templates pull from the same source so emails match the portal.

Proposed Architecture

graph TB
    subgraph Triggers
        Share[Share created<br/>handler.go / collection / prompt]
        Comment[Thread event<br/>thread_handler.go]
    end
    subgraph Core
        Enqueue[Notification enqueue]
        Prefs[(user_notification_prefs)]
        Queue[(notifications queue)]
    end
    subgraph Delivery
        Worker[Send worker<br/>indexjobs pattern]
        Digest[Daily digest scheduler]
        Render[Branded HTML render]
        SMTP[SMTP client]
    end
    subgraph Config
        Settings[(platform_settings<br/>SMTP, encrypted pass)]
    end
    Share --> Enqueue
    Comment --> Enqueue
    Enqueue --> Prefs
    Prefs -->|off| Drop[Dropped]
    Prefs -->|immediate| Queue
    Prefs -->|daily| Queue
    Queue --> Worker
    Digest --> Worker
    Worker --> Render --> SMTP
    Settings --> SMTP
    Settings --> Render

Loading

Workstreams

Sized so each can be a separate PR. Suggest splitting into child issues; this ticket is the epic.

A. Admin settings surface + SMTP config

• New admin page /admin/settings (general settings, expandable), with an SMTP section. Register in ui/src/components/layout/Sidebar.tsx adminNavItems and ui/src/components/layout/AppShell.tsx.
• Storage: a dedicated platform_settings table (decided; see Decisions Made), following the pkg/user/store.go interface+postgres pattern. Non-secret SMTP fields (host, port, from, username, TLS mode) stored plainly; password encrypted at rest via FieldEncryptor and write-only in the UI (never returned).
• Admin REST endpoints under /api/v1/admin/... (mutable-only gated, like existing config routes). Include a "send test email" action.
• Honor file-mode vs database-mode read-only gating, consistent with existing admin config behavior.

B. User notification preferences

• New user_notification_prefs table keyed by email (FK to users.email): a global mode (off | immediate | daily) and per-category toggles (shares, comments). Default to immediate-on (per the "important features default ON" convention).
• Store interface + postgres impl (pkg/notification or pkg/user), following pkg/user/store.go.
• A non-admin portal settings page with a Notifications section (toggle off / immediate / daily digest). This is a user-facing page, not under /admin.
• REST endpoints scoped to the authenticated user (self-scope server-side; a user can only read/write their own prefs).

C. Notification core (enqueue + store)

• notifications queue table (recipient email, category, payload/refs, status, attempts, scheduled_for, created_at) with a status index, following migration conventions in pkg/database/migrate/migrations/.
• Enqueue function that consults user_notification_prefs: drop if off, queue for immediate send, or stamp scheduled_for the next digest window if daily.
• Wire enqueue into the three share Insert call sites and the thread-event call site. Non-blocking: enqueue is a cheap DB insert; actual send is the worker's job. A share/comment must still succeed if enqueue fails (log, do not fail the request).

D. Delivery worker + daily digest

• Send worker following pkg/indexjobs/worker.go (poll + LISTEN/NOTIFY, lease locking, retry with backoff, heartbeat). Claims immediate rows and due daily rows.
• Daily digest: groups a user's pending daily notifications into one email per window. Confirm window/timezone behavior (suggest a single configurable UTC send hour for v1).
• Wire worker startup into platform lifecycle alongside the existing job worker.

E. Branded HTML email templates

• Responsive HTML email (table-based for client compatibility) with a plaintext fallback part. Pull brand title, logo, and implementor from the same source the portal uses (PortalConfig resolved SVG/values).
• Templates: "shared with you" (asset/collection/prompt variants) and "new comment/feedback". Digest template lists multiple items.
• Deep links back into the portal using Portal.PublicBaseURL.

F. Config + docs

• If any email settings live in platform.yaml (e.g., encryption key reference, digest send hour), add an EmailConfig/notification section in pkg/platform/config.go with defaults in applyDefaults, mirroring existing sections.
• Update README.md, docs/, docs/llms.txt, docs/llms-full.txt (doc-check is a blocking gate for new config/migrations/pages/endpoints).

Relationship to #627

#627 (@mention tagging + notifications on thread comments) names "a notification-delivery mechanism and a per-user preferred channel setting" as a prerequisite it does not build. This ticket delivers exactly that, plus the generic "new comment/feedback" notification. #627 then becomes a thin consumer: parse @user tokens from the comment body and enqueue a mention notification through the substrate built here. This ticket should land first; #627 builds on workstreams B, C, D, E.

Decisions Made

• Triggers in v1: direct shares (asset/collection/prompt) and thread comments/feedback.
• Daily digest is in scope for v1 (worker + scheduler), alongside immediate and off.
• SMTP password encrypted at rest via the existing FieldEncryptor, in a dedicated settings store; write-only in the UI.
• Settings storage: a dedicated platform_settings table (typed sections, encrypted SMTP password via FieldEncryptor, password write-only in the UI), following the pkg/user/store.go interface + postgres pattern. Not config_entries — the value shape is typed and the secret needs encryption, which the untyped key/value store would special-case.
• Email library: a vetted, pinned third-party SMTP library rather than stdlib net/smtp. Recommended: github.com/wneessen/go-mail (actively maintained, STARTTLS + implicit TLS, multipart HTML + plaintext, and a path to XOAUTH2 for future OAuth-based providers). Pin with a SHA per the project's pinned-dependency rule.

Open Questions

• Digest window: single global UTC send hour, or per-user timezone? Suggest global UTC hour for v1.

Acceptance Criteria

• Admin configures SMTP in the portal, sends a test email, and receives it.
• Sharing an asset/collection/prompt with a user whose prefs are "immediate" produces a branded HTML email with a working deep link, within the worker poll interval; a notifications row transitions to a sent status.
• A user who sets notifications to "daily" receives one digest email per window covering that window's events, and no per-event emails.
• A user who turns notifications off receives nothing, and no notifications rows are queued for them.
• A thread comment/feedback event targeting a user triggers the same path.
• SMTP password is never returned by any API and is stored encrypted (verified by reading the row).
• Enqueue failure does not fail the originating share/comment request.
• Integration test wires the real share handler -> enqueue -> worker -> a captured/mock SMTP sink and asserts an email was rendered and "sent" end to end.

Scope

Portal / Backend / Frontend

[cjimti]

Status: implemented on branch feat/631-email-notifications, full verify suite green, awaiting review/commit/PR.

All six workstreams are complete on the branch (86 files, +8,634 lines):

A. Admin SMTP settings. platform_settings table (migration 000080) with a typed store; password encrypted at rest via the existing FieldEncryptor and write-only through the API (responses carry password_set only). REST at GET/PUT /api/v1/admin/settings/smtp plus POST .../test (409 when SMTP is disabled or unconfigured, so nothing sends around the master switch). Writes are gated to database config mode (405 in file mode). New /admin/settings portal page with a send-test action.

B. User notification preferences. user_notification_prefs table (000081), default immediate-on with per-category toggles (shares, comments/feedback). Self-scoped GET/PUT /api/v1/portal/notification-prefs (only the caller's own row is ever read or written) and a user-facing /settings portal page.

C. Notification queue + enqueue. notifications queue table (000082). Enqueue consults preferences: off drops, immediate schedules now, daily stamps the next digest window. Wired into all three share Insert sites and both thread-event sites; enqueue failure is logged and never fails the originating request. A per-actor rate limit (burst 30, 6/min sustained per replica) bounds outbound volume per account.

D. Send worker + daily digest. Worker follows the indexjobs pattern: LISTEN/NOTIFY wakeup with a 30s poll fallback, 2-minute delivery leases with crash reclaim, 5 retry attempts with exponential backoff (30s doubling to a 32m cap). Digest mode delivers one email per recipient per window (configurable global UTC hour, default 13:00). Retention purge bounds the table: sent/failed rows after 30 days, undelivered rows after 7 days, so enabling SMTP months into a deployment never delivers a stale backlog. When SMTP is unconfigured, rows wait without burning attempts. Started/stopped by the HTTP composition root around the serve loop.

E. Branded emails. Table-based responsive HTML with a plaintext alternative via github.com/wneessen/go-mail v0.8.1 (pinned): share (asset/collection/prompt), comment/feedback, digest, and SMTP-test variants. Brand name, implementor footer, and deep links come from the same PortalConfig the portal uses; asset/collection shares link to the token viewer (works without sign-in), prompts link to the in-app prompt page, and every email links to the recipient's notification preferences.

F. Config + docs. New notifications: YAML section (enabled default-on with a database, digest_hour_utc). New docs page (Email Notifications) plus configuration reference, portal/admin guides, README, llms.txt/llms-full.txt, and example config updates.

Verification. Full make verify green end to end (fmt, race tests, >=80% total and patch coverage, lint, gosec, govulncheck, Semgrep, CodeQL, 332 frontend tests + lint + e2e, doc gates, GoReleaser dry-run). Real-Postgres integration tests prove the claim SQL (FOR UPDATE SKIP LOCKED), lease reclaim, digest batching, and retention purge against the actual schema. An end-to-end test drives the real share handler through the real enqueuer, worker, and renderer to a captured SMTP sink, and a live localhost SMTP conversation validates the wire path. All architectural gates hold: the Platform facade is unchanged (frozen field/method ceilings), package size budgets pass, and pkg/portal gained zero new imports (triggers go through a small Notifier interface implemented by the delivery seam).

Scope decisions for v1 (matching the ticket's Decisions Made plus implementation calls):

• Digest window is a single global UTC hour (the ticket's open question), default 13:00.
• Shares targeted by user ID alone are not notified: no email address exists anywhere in the system for such a recipient. Email-addressed shares (the share dialog's normal path) are always notified.
• The default confidentiality notice is viewer chrome and is not rendered as a personal message; a custom notice passes through to the email.
• The per-actor rate limit is per replica (multi-replica deployments multiply the cap by replica count, still bounded).
• The send worker runs under the HTTP transport, which is where every v1 trigger (portal REST) lives.

Next: review, merge via PR, then #627 (@mention tagging) can build on this substrate as planned.