Various kernel opts - #22
Closed
Human9000-bit wants to merge 1 commit into
Closed
Conversation
Collaborator
Author
|
/bench |
ONNX CPU benchmarkGitHub-hosted runner signal, advisory only. Base and head are compared within the same architecture job.
Head op/kernel summary
Raw JSON artifacts include per-run min/p50/avg/p95/p99, optimized node counts, graph-cost summaries, yscv CPU dispatch report, and per-node runner profile summaries. |
Human9000-bit
force-pushed
the
kernel-opt
branch
from
July 28, 2026 17:21
c037200 to
da1bb51
Compare
Human9000-bit
marked this pull request as ready for review
August 4, 2026 13:33
Owner
Human9000-bit
pushed a commit
to Human9000-bit/YSCV
that referenced
this pull request
Aug 12, 2026
…ssing benches Reviewing PR enthropy7#22 turned up that `prfm pldl1keep` had drifted into three identical copies. Fold them into `ops::prefetch::prefetch_l1_keep`, which also documents the two rules the measured results depend on: hoist the gate out of the K-loop (the per-iteration branch alone costs the A53 more than the hint saves), and only hint operands whose stride actually beats the hardware prefetcher. By that second rule the AVX `binary_same_shape` prefetch does not belong: the access is unit-stride over three streams. Measured on Zen 4 it sits inside run-to-run drift — the A/B flips sign depending on which arm runs first — so it was only adding uops to a bandwidth-bound loop. The transposed-A and `matmul_row_set` prefetch hints from that PR are not here: they measured neutral. On Zen 4 the trans_a kernel at the tracker's real shape is 18.97 vs 19.05 us (p = 0.31) once the A/B is run in both directions, and `FusedTransposeMatMul` — their only caller — is 1.1% of node time on Zen 4 and 1.3% on the A53, so there is nothing there to win. Both kernels were unbenchmarked, which is why that was not visible; they have benchmarks now. Also assert the execution plan carries one action per node. With the sequential fallback gone the runner walks the plan and nothing else, so a short plan would silently skip trailing nodes rather than fail. A53 (Orange Pi, 1T, 60 iters x 3 alternating): 337.6/337.8/337.8 ms vs 338.7/338.9/339.8 on main. Zen 4 tracker unchanged at 8.8 ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
... More to come