Skip to content

Blur bystander faces on the way out, not just on the way to a model (Plan CP) - #303

Merged
straff2002 merged 1 commit into
mainfrom
feat/outbound-frame-privacy-pipeline
Aug 8, 2026
Merged

Blur bystander faces on the way out, not just on the way to a model (Plan CP)#303
straff2002 merged 1 commit into
mainfrom
feat/outbound-frame-privacy-pipeline

Conversation

@straff2002

Copy link
Copy Markdown
Owner

Completes CO Item 0, which wired the bystander blur that had never been called and deliberately stopped at the ~1 fps model-facing paths.

What was still unprotected

With "Blur Bystander Faces" on, five consumers still received unblurred frames — every one an egress:

Consumer Where the frame goes
WebRTCStreamingService a shareable browser URL — the widest exposure in the app
BroadcastService RTMP to YouTube/Twitch/Kick — public, and recorded by the platform
WebRTCPeerTransport a remote expert, peer-to-peer
MeetingLinkTransport a remote expert, via a third-party meeting tool
VideoRecordingService disk, and from there wherever the user shares it

The gap was the wrong way round. Someone enabling this while walking through a public space is most likely thinking about the recording and the broadcast — the artefacts that outlive the moment and travel.

Why it's affordable now

One shared pass. OutboundFrameRelay sits between CameraService.framePublisher and every outbound consumer. Filtering per consumer would run Vision up to five times on one frame. It also makes "which consumers get filtered pixels" a single wiring decision rather than five independent ones — the property whose absence let the blur ship uncalled in the first place.

Detection cadence ≠ blur cadence. Detection is tens of milliseconds; compositing over known rectangles is cheap. The relay detects on an interval and reuses expanded rectangles between passes, with a grace period so one missed detection can't flash a face unblurred.

FrameCoalescer exists because at camera rate you must never queue: one frame in flight, one pending, newest wins. An unbounded backlog would drift the stream further behind real time with every frame and never report anything, so drops are counted and surfaced.

Filter off is a true passthrough — forwarded on the spot, no queue hop, no copy. The enabled check is per frame so a mid-session toggle lands, which is the bug the deleted filteredPublisher had (it sampled isEnabled once at construction).

The residual, stated not buried

A face newly entering frame is unblurred until the next detection. Short interval, generous expansion and post-detection grace keep it small; nothing removes it. Settings copy and README say so in plain language rather than implying total coverage.

A defect caught before shipping

Detection returns pixel rectangles and the composite flips them against the CIImage extent, also pixels — but the clamp was against image.size, which is points. On a 2x/3x frame every rectangle would have been truncated or dropped: a silent under-blur on exactly the high-resolution captures carrying the most recognisable detail. Fixed, with a test that passes against the pixel extent and fails against the point size.

The test that was built to fail

CO Item 0 shipped testRecordingAndBroadcastAreNotYetCovered precisely so that building this pipeline would break it and force the user-facing copy to be corrected in the same change. It did exactly that. The assertion is inverted and kept as a record of how the carve-out was retired rather than quietly forgotten.

Verification

  • Full suite: 2852 tests, 0 failures (3 skipped)
  • Release configuration builds clean
  • No Localizable.xcstrings churn

Owed on device — this genuinely gates the defaults

  • Sustained frame rate through the relay while broadcasting, detection at 200 ms
  • Drop counts under real motion (the coalescer's counter is the instrument)
  • Thermal behaviour over a long recording
  • Whether the interval and expansion actually keep a walking bystander covered — the only question that matters, and the only one a simulator cannot answer

Open questions in the plan

  1. Should recordings get a stricter mode than live streams? A recording is re-watchable and shareable; a stream is gone. Needs the device numbers first.
  2. Under thermal pressure, widen the interval or stop and tell the user? Silently widening weakens a privacy promise without saying so — which is the failure mode this whole line of work exists to correct. Leaning toward surfacing it.
  3. The wearer has no way to confirm the blur is working on an outbound stream. The pinned card shows it for stills; streaming has no equivalent.

Plan: docs/plans/CP-outbound-frame-privacy.md

🤖 Generated with Claude Code

Plan CO Item 0 wired the bystander blur that had never been called, and stopped
at the paths that send frames to an AI provider -- roughly 1 fps, where a Vision
pass per frame is affordable. It left five consumers uncovered, every one of
them an egress: WebRTC browser streaming (a shareable URL, the widest exposure
in the app), RTMP broadcast (public, and recorded by the platform), both expert
transports, and video recording (disk, then wherever it is shared).

The gap was the wrong way round. Someone who turns this on while walking
through a public space is most likely thinking about the recording and the
broadcast -- the artefacts that outlive the moment and travel -- and those were
exactly what we did not cover. Saying so in the Settings copy was better than
lying, but it was still a feature that did not do what its name implies.

Two structural changes make it affordable at camera rate.

One shared pass. `OutboundFrameRelay` sits between `CameraService.framePublisher`
and every outbound consumer, blurs once, and republishes; filtering per consumer
would have run Vision up to five times on the same frame. It also makes "which
consumers receive filtered pixels" a single wiring decision instead of five
independent ones -- the property whose absence let the blur ship uncalled in the
first place. Consumers already took a `PassthroughSubject<UIImage, Never>`, so
each call site changes by one argument.

Detection cadence separated from blur cadence. Detection is tens of
milliseconds; compositing over known rectangles is cheap. The relay detects on
an interval and reuses expanded rectangles in between, with a grace period so a
single missed pass cannot flash a face unblurred.

That trade-off has a real cost and it is stated rather than buried: a face newly
entering frame is unblurred until the next detection. A short interval,
generous expansion and post-detection grace keep it small; nothing removes it.
Settings and the README now say so.

`FrameCoalescer` exists because at camera rate you must never queue. One frame
in flight, one pending, newest wins, and the drops are counted so the cost is
measurable on device rather than invisible -- an unbounded backlog would drift
the stream further behind real time with every frame and never report anything.

Filter off is a true passthrough: forwarded on the spot, no queue hop, no copy.
The enabled check is per frame, so a mid-session toggle lands -- the bug the
deleted `filteredPublisher` had, where `isEnabled` was sampled once at
construction.

One defect fixed before it shipped: detection returns pixel rectangles and the
composite flips them against the CIImage extent, also pixels, but the clamp was
against `image.size`, which is points. On a 2x or 3x frame every rectangle would
have been truncated or dropped -- a silent under-blur on exactly the
high-resolution captures carrying the most recognisable detail. Guarded by a
test that fails against the point size.

CO Item 0 shipped `testRecordingAndBroadcastAreNotYetCovered` specifically so
that building this pipeline would break it and force the user-facing copy to be
corrected in the same change. It did. The assertion is now inverted and kept as
a record of how the carve-out was retired rather than quietly forgotten.

Full suite 2852 tests, 0 failures. Release configuration builds clean. Device
measurement gates the defaults and is owed: sustained frame rate through the
relay, drop counts under real motion, thermal behaviour over a long recording,
and whether the interval and expansion actually keep a walking bystander
covered -- the only question that matters and the only one a simulator cannot
answer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@straff2002
straff2002 merged commit 8b060f2 into main Aug 8, 2026
1 check passed
@straff2002
straff2002 deleted the feat/outbound-frame-privacy-pipeline branch August 9, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant