Skip to content

fix(telemetry): cut PostHog event volume (flag events, person.set carrier, error caps) - #1362

Draft
benceruleanlu wants to merge 1 commit into
mainfrom
ben/telemetry-volume-cuts
Draft

fix(telemetry): cut PostHog event volume (flag events, person.set carrier, error caps)#1362
benceruleanlu wants to merge 1 commit into
mainfrom
ben/telemetry-volume-cuts

Conversation

@benceruleanlu

Copy link
Copy Markdown
Member

Part of the PostHog cost-cut program (GTM-394): desktop is the largest event producer on the ~$18.6K/mo bill (target ≤$10K). The four cuts here remove roughly 21M+ events/28d of pure carrier/loop volume — ~$700-800/mo combined across product-analytics ingestion, the warehouse event export, and the error-tracking line ($340/mo) — without removing any analytical signal. All four changes are in the main process (src/main/lib/telemetry.ts, src/main/lib/updater.ts); the renderer has no PostHog SDK, so this covers every desktop emit path.

1. $feature_flag_called — 3.37M/28d, 91.7% from desktop-cloud-capacity

getOpsFlag already opts out per-call (sendFeatureFlagEvents: false, since #1304), but fleet volume shows released builds still emitting one implicit event per flag evaluation. This PR sets the client-level default sendFeatureFlagEvent: false on the PostHog constructor so no current or future getFeatureFlag / isFeatureEnabled call site can silently reintroduce a per-evaluation event.

Deliberately NOT changed: flag evaluation is untouched — getOpsFlag, getAllFlags (which never emitted this event), the ops-flag cache, and experiment variant assignment all behave exactly as before. Experiment exposure remains explicitly recorded via comfy.desktop.experiment.exposed (experiments.ts), which is what dashboards actually join on.

2. comfy.desktop.person.set — 15.2M/28d from ~750K persons (~20/person/mo)

The event's only job was carrying $set person properties. Authenticated person-property updates now ride $set / $set_once on the next captured event (posthog-node applies person properties from any event's properties), so the dedicated carrier event disappears from steady-state traffic.

Carrier choice and why person properties keep flowing:

  • Updates queue in-process (scrubbed at enqueue) and attach to the next admitted capture() — in practice the very next product event, since updates are registered adjacent to the events that cause them (boot settings snapshot, hardware census, cloud launch, settings changes).
  • comfy.desktop.session.ended at shutdown is the guaranteed last carrier of a clean session, so a queued update is delivered at most one session late even in an otherwise event-quiet session.
  • Identity boundaries flush eagerly as a dedicated person.set (logout, account switch) so an update can never land on the wrong person; anything unflushable at that boundary is discarded rather than leaked to the next identity.
  • A carrier dropped by the SDK or volume guards keeps the queue intact for the next event — no silent loss.

Deliberately NOT changed: distinct_id selection, the single client.identify() at Firebase-UID bind (pre-auth $set/$set_once still applied there), the pending-identity-merge replay, and $process_person_profile enforcement are all untouched — so PostHog person profiles, the warehouse persons export, and ~2.5M desktop MAU metering are unaffected. Property freshness semantics shift from "immediate dedicated event" to "next captured event, worst-case session end"; last-write-wins for $set and first-write-wins for $set_once are preserved.

3. comfy.desktop.app_update.error — 2.6M/28d

A broken install re-fails the 10-minute auto-check forever. Identical error messages are now capped at 5 emits per app session (on top of the existing 1s identical-error repeat guard). Keyed on the message alone so operation/version churn can't defeat the cap; a NEW failure mode mid-session still ships its first occurrences. Since *.error events intentionally bypass the SDK-level sliding-window guard, this call-site cap was the missing bound.

4. $exception rate-limiting — ~1.9M/mo, error tracking $340/mo

93.9% of volume is four Electron crash-loop messages ("Comfy window renderer process exited (launch-failed)", "Renderer process gone: launch-failed", "Child process Utility exited: crashed", "Child process GPU exited: crashed"), with individual machines emitting thousands each. captureException now caps identical scrubbed name+message pairs at 5 per session in the local capture wrapper (every $exception funnels through it: process handlers, IPC handler, window attach). First occurrences always ship — this suppresses crash-loop repeats, not signal — and keying on the scrubbed message means PII variation (usernames in paths) can't split a loop into distinct keys. The Datadog mirror stays in sync because forwarding already gates on the capture being accepted.

Out of scope

comfy.desktop.execution.* and comfy.desktop.comfyui.model_usage are separate workstreams and untouched.

Testing

  • vitest run — 224 files, 3700 passed / 1 skipped (18 new tests: client-level flag-event suppression, person-property carrier lifecycle incl. logout flush + scrub + consent revocation, $exception per-message cap incl. SDK-failure slot handling, app_update.error session cap under fake timers)
  • pnpm typecheck (all four tsconfigs) and eslint . — clean (ran via pre-commit hook)
  • All new tests are deterministic: injected fake timers for the updater's time-based guard, explicit state resets, no wall-clock or ordering races

🤖 Generated with Claude Code

…rier, error caps)

Four independent volume cuts in the main-process telemetry pipe, part of
the PostHog cost-cut program (GTM-394):

- $feature_flag_called (3.37M/28d): set client-level
  sendFeatureFlagEvent: false so no flag read can emit the implicit
  per-evaluation event. Flag evaluation behavior is unchanged.
- comfy.desktop.person.set (15.2M/28d): authenticated person-property
  updates now ride $set/$set_once on the next captured event instead of
  a dedicated carrier event. session.ended at shutdown is the guaranteed
  last carrier; identity boundaries (logout/account switch) still flush
  eagerly as a dedicated event so updates never land on the wrong
  person. identify/distinct_id semantics untouched.
- comfy.desktop.app_update.error (2.6M/28d): cap identical error
  messages at 5 emits per app session on top of the existing 1s repeat
  guard; the 10-min auto-check re-fails forever on a broken install.
- $exception (~1.9M/mo, 93.9% four crash-loop messages): cap identical
  scrubbed exception messages at 5 per session in captureException.
  First occurrences always ship; distinct messages are unaffected.
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 942054c9-4880-4820-a9c8-81f609e498c2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant