feat: expand native review journeys and performance budgets - #5968
Open
jedwards27 wants to merge 9 commits into
Open
feat: expand native review journeys and performance budgets#5968jedwards27 wants to merge 9 commits into
jedwards27 wants to merge 9 commits into
Conversation
added 5 commits
August 14, 2026 20:54
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
…urneys * origin/main: fix(desktop): hide the offcanvas-collapsed sidebar so it stops painting over the community rail (block#5947) Polish mobile message threads and composer (block#5645) chore(release): release Buzz Desktop version 0.5.14 (block#5917) ci(release): remove desktop smoke gate (block#5914) Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Author
|
Review as a logical stack on #5958. GitHub cannot target the fork-only base branch, so this PR is mechanically based on |
This was referenced Aug 15, 2026
Fail closed on non-standard fixture ports, use fixed loopback database coordinates, scrub repository-controlled subprocess environments, and remove generated review keys when fixture seeding fails. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Apply absolute maximum budgets to the worst candidate sample while retaining median-based relative regression comparisons. Cover the catastrophic-outlier case directly. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Default harness commands to a credential-free environment and isolate cleanup HOME without reconstructing the host environment. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> (cherry picked from commit 3635687)
themiguelamador
suggested changes
Aug 16, 2026
themiguelamador
left a comment
There was a problem hiding this comment.
I found several correctness and evidence-integrity issues that need to be resolved before this can merge:
- The launcher imports PyYAML without declaring or pinning it, so a clean checkout cannot run the documented workflow reproducibly.
- Journey validation accepts boolean/non-finite numeric values and unbounded durations/deltas, and scroll actions do not require or move to their declared target.
- The AVAssetWriter backpressure path does not advance the frame clock, causing a busy-spin, and capture errors do not stop the loop.
- Performance comparisons accept malformed provenance, missing machine identity, non-finite metrics, duplicate samples, and the same receipts in both cohorts; those cases can produce a meaningless passing budget result.
- The semantic snapshot receiver is unauthenticated and accepts unbounded payloads, while the renderer trusts an arbitrary probe destination and malformed fixture identity. That permits local cross-process writes and can exfiltrate the review DOM snapshot if the environment is influenced. Relay parsing also accepts ambiguous URL components.
I prepared a signed fix commit covering all findings: https://github.com/Complear/buzz/commit/69a2a8cb8
Verification on the fix:
- 36 Python native-review tests pass with ResourceWarning promoted to an error
- Swift native driver release build passes
- native-review shell scripts parse cleanly
- desktop Biome/file-size/text/pubkey checks pass (only existing warnings/info)
- desktop TypeScript typecheck passes
- semantic probe URL regression test passes
- full desktop suite previously passed: 4,954 tests
- git diff whitespace check passes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stack
Stacked on #5958 (
feat/native-review-harness) so the journey/performance layer can be reviewed independently.Validation
7c856176d1e1f3282a7739ea2097025fdc373ea6: branch skew, Desktop check/typecheck/test, Rust tests, Tauri checkstest-results/native-review/7c856176d1e1/tooltip_fresh_dwell/tooltip_fresh_dwell-20260815T090411-69d3d0/receipt.jsontest-results/native-review/7c856176d1e1/tooltip_fresh_dwell/tooltip_fresh_dwell-20260815T090449-2619cd/receipt.jsontest-results/native-review/7c856176d1e1/tooltip_fresh_dwell/tooltip_fresh_dwell-20260815T090522-b61840/receipt.jsonPerformance semantics