Avoid ParseTensorData's double copy in CSE tensor hash/equality for raw_data - #8
Open
take-cheeze wants to merge 1 commit into
Conversation
…aw_data CSETensorHash and CSETensorEqual (used by eliminate_duplicate_initializer and eliminate_common_subexpression) went through ParseTensorData<T> for every tensor comparison, which for raw_data-backed tensors makes two full copies of the tensor's bytes (one to un-const the string, one to convert it into a typed std::vector) plus an element-by-element hash_combine loop. Since raw_data is always little-endian on disk regardless of host byte order, byte-identical raw_data always implies value-identical data on any host. Add a fast path that hashes/compares the raw bytes directly via Tensor::raw() (a zero-copy const std::string&) when both tensors are raw_data-backed, skipping ParseTensorData entirely. A tensor whose duplicate happens to be stored via typed fields instead of raw_data (rare in practice) simply won't be recognized as a duplicate through this path -- a missed optimization, never an incorrect merge. On a 582-node ONNX model with ~300MB of raw_data initializers, eliminate_duplicate_initializer accounted for ~98% of all optimizer pass time (re-hashing every initializer from scratch on each of ~53 fixed-point rounds); this fix cuts total simplify() time roughly in half on that model.
take-cheeze
force-pushed
the
claude/eliminate-duplicate-initializer-perf-633
branch
from
August 18, 2026 07:40
6604bc2 to
229d9bc
Compare
take-cheeze
changed the base branch from
main
to
claude/graph-native-optimize-633
August 18, 2026 07:40
pull Bot
pushed a commit
to naonao-cola/onnx-simplifier
that referenced
this pull request
Aug 18, 2026
Points at onnxsim/optimizer#8, which avoids ParseTensorData's double copy in CSETensorHash/CSETensorEqual for raw_data-backed tensors. eliminate_duplicate_initializer re-hashes every initializer's full byte content from scratch on every fixed-point round; on mixer_l16_224_in21k (~300MB of raw_data initializers, 53 rounds) that pass alone accounted for ~98% of all optimizer pass time. This fix roughly halves total simplify() time on that model, on both the default and opt-in graph-native optimize paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CSETensorHashandCSETensorEqual(shared byeliminate_duplicate_initializerandeliminate_common_subexpression) went throughParseTensorData<T>for every tensor comparison. Forraw_data-backed tensors that makes two full copies of the tensor's bytes per call — one to un-const thestd::string, one to convert it into a typedstd::vector— plus an element-by-elementhash_combineloop over every element.Since ONNX's
raw_datais always little-endian on disk regardless of host byte order, byte-identicalraw_dataalways implies value-identical data, on any host. This adds a fast path that hashes/compares the raw bytes directly viaTensor::raw()(a zero-copyconst std::string&) whenever both tensors areraw_data-backed, skippingParseTensorDataentirely.Base branch note: this targets
claude/graph-native-optimize-633(the commit onnxsim'smasteractually hasthird_party/onnx-optimizerpinned to) rather thanmain, sincemainis missing some already-adopted changes (e.g.InitializersAsConstants) that this fix needs to build against for a clean diff. Retarget tomainonce that catches up.Why this can't be a false positive
CSETensorEqual: byte-identicalraw_data⟹ value-identical (decoding bytes is a pure, deterministic function of those bytes), solhs->raw() == rhs->raw()can never merge two tensors with actually-different values.CSETensorHash: the only requirement on a hash function is that equal tensors hash equally. Since the fast-path equality check above compares the exact sameraw()bytes, hashing those same bytes preserves that invariant.raw_data(rare in practice — real exporters use one storage format consistently) simply won't hash/compare equal through this fast path, so it may go undetected as a duplicate. That's a missed optimization, never an incorrect merge — consistent with this pass's existing doc comment that it's a best-effortNop-type optimization.Motivation
Found while investigating onnxsim#633 (onnxsim vs onnxslim speed gap). Profiling
mixer_l16_224_in21k(582 nodes, ~300MB ofraw_datainitializers, 53 fixed-point rounds) showedeliminate_duplicate_initializeralone accounted for ~98% of all optimizer pass time (up to 302s of 306s), because it re-hashes every initializer from scratch on every round even though most initializer content is unchanged round to round. With this fix, totalonnxsim.simplify()time on that model roughly halves (268s → 111s in one measurement).Testing
eliminate_duplicate_initializerstill merges them to one, and the resulting model produces bit-identical inference output (verified via onnxruntime).test_eliminate_duplicate_initializercoverage (INT32/INT64/FLOAT/DOUBLE, allraw=True) — all pass.test_simple.py,test_backend.py,test_model_checking.py,test_constant_fold_determinism.py,test_moved_optimizer_passes.py): 51 passed, 2 failed on an unrelated missing optional dependency (onnxscript), 0 regressions.mixer_l16_224_in21kend-to-end before/after on both onnxsim's default (ModelProto) and opt-in graph-native optimize paths — both show ~2.4x-4x speedups, with identical output node counts (no change in simplification result).