Blur bystander faces on the way out, not just on the way to a model (Plan CP) - #303
Merged
Merged
Conversation
Plan CO Item 0 wired the bystander blur that had never been called, and stopped at the paths that send frames to an AI provider -- roughly 1 fps, where a Vision pass per frame is affordable. It left five consumers uncovered, every one of them an egress: WebRTC browser streaming (a shareable URL, the widest exposure in the app), RTMP broadcast (public, and recorded by the platform), both expert transports, and video recording (disk, then wherever it is shared). The gap was the wrong way round. Someone who turns this on while walking through a public space is most likely thinking about the recording and the broadcast -- the artefacts that outlive the moment and travel -- and those were exactly what we did not cover. Saying so in the Settings copy was better than lying, but it was still a feature that did not do what its name implies. Two structural changes make it affordable at camera rate. One shared pass. `OutboundFrameRelay` sits between `CameraService.framePublisher` and every outbound consumer, blurs once, and republishes; filtering per consumer would have run Vision up to five times on the same frame. It also makes "which consumers receive filtered pixels" a single wiring decision instead of five independent ones -- the property whose absence let the blur ship uncalled in the first place. Consumers already took a `PassthroughSubject<UIImage, Never>`, so each call site changes by one argument. Detection cadence separated from blur cadence. Detection is tens of milliseconds; compositing over known rectangles is cheap. The relay detects on an interval and reuses expanded rectangles in between, with a grace period so a single missed pass cannot flash a face unblurred. That trade-off has a real cost and it is stated rather than buried: a face newly entering frame is unblurred until the next detection. A short interval, generous expansion and post-detection grace keep it small; nothing removes it. Settings and the README now say so. `FrameCoalescer` exists because at camera rate you must never queue. One frame in flight, one pending, newest wins, and the drops are counted so the cost is measurable on device rather than invisible -- an unbounded backlog would drift the stream further behind real time with every frame and never report anything. Filter off is a true passthrough: forwarded on the spot, no queue hop, no copy. The enabled check is per frame, so a mid-session toggle lands -- the bug the deleted `filteredPublisher` had, where `isEnabled` was sampled once at construction. One defect fixed before it shipped: detection returns pixel rectangles and the composite flips them against the CIImage extent, also pixels, but the clamp was against `image.size`, which is points. On a 2x or 3x frame every rectangle would have been truncated or dropped -- a silent under-blur on exactly the high-resolution captures carrying the most recognisable detail. Guarded by a test that fails against the point size. CO Item 0 shipped `testRecordingAndBroadcastAreNotYetCovered` specifically so that building this pipeline would break it and force the user-facing copy to be corrected in the same change. It did. The assertion is now inverted and kept as a record of how the carve-out was retired rather than quietly forgotten. Full suite 2852 tests, 0 failures. Release configuration builds clean. Device measurement gates the defaults and is owed: sustained frame rate through the relay, drop counts under real motion, thermal behaviour over a long recording, and whether the interval and expansion actually keep a walking bystander covered -- the only question that matters and the only one a simulator cannot answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes CO Item 0, which wired the bystander blur that had never been called and deliberately stopped at the ~1 fps model-facing paths.
What was still unprotected
With "Blur Bystander Faces" on, five consumers still received unblurred frames — every one an egress:
WebRTCStreamingServiceBroadcastServiceWebRTCPeerTransportMeetingLinkTransportVideoRecordingServiceThe gap was the wrong way round. Someone enabling this while walking through a public space is most likely thinking about the recording and the broadcast — the artefacts that outlive the moment and travel.
Why it's affordable now
One shared pass.
OutboundFrameRelaysits betweenCameraService.framePublisherand every outbound consumer. Filtering per consumer would run Vision up to five times on one frame. It also makes "which consumers get filtered pixels" a single wiring decision rather than five independent ones — the property whose absence let the blur ship uncalled in the first place.Detection cadence ≠ blur cadence. Detection is tens of milliseconds; compositing over known rectangles is cheap. The relay detects on an interval and reuses expanded rectangles between passes, with a grace period so one missed detection can't flash a face unblurred.
FrameCoalescerexists because at camera rate you must never queue: one frame in flight, one pending, newest wins. An unbounded backlog would drift the stream further behind real time with every frame and never report anything, so drops are counted and surfaced.Filter off is a true passthrough — forwarded on the spot, no queue hop, no copy. The enabled check is per frame so a mid-session toggle lands, which is the bug the deleted
filteredPublisherhad (it sampledisEnabledonce at construction).The residual, stated not buried
A face newly entering frame is unblurred until the next detection. Short interval, generous expansion and post-detection grace keep it small; nothing removes it. Settings copy and README say so in plain language rather than implying total coverage.
A defect caught before shipping
Detection returns pixel rectangles and the composite flips them against the CIImage extent, also pixels — but the clamp was against
image.size, which is points. On a 2x/3x frame every rectangle would have been truncated or dropped: a silent under-blur on exactly the high-resolution captures carrying the most recognisable detail. Fixed, with a test that passes against the pixel extent and fails against the point size.The test that was built to fail
CO Item 0 shipped
testRecordingAndBroadcastAreNotYetCoveredprecisely so that building this pipeline would break it and force the user-facing copy to be corrected in the same change. It did exactly that. The assertion is inverted and kept as a record of how the carve-out was retired rather than quietly forgotten.Verification
Localizable.xcstringschurnOwed on device — this genuinely gates the defaults
Open questions in the plan
Plan:
docs/plans/CP-outbound-frame-privacy.md🤖 Generated with Claude Code