-
Notifications
You must be signed in to change notification settings - Fork 19
Adds an API to set dimension names, updates SAMv2 to trigger MHA fusion #657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
6207255 to
a14ab49
Compare
a14ab49 to
6a04054
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces a NamedDimension API for naming dynamic dimensions to enhance runtime optimizations and triggers improved MHA fusion in SAMv2. It also adds new performance tests (SDPA and linear_block), updates tracing and compilation to use input_infos, and revises documentation accordingly.
- Added NamedDimension support to express dynamic dimension equality.
- Updated trace and compile flows to replace legacy shape profiles with input_infos.
- Modified performance test parameters and documentation to reflect the new API.
Reviewed Changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tripy/tests/trace/test_trace.py | Added a new test case for generating a trace string that reflects dynamic dimension naming. |
| tripy/tests/performance/cases/sdpa.py | Introduced a new performance test for the SDPA module with updated perf parameters. |
| tripy/tests/performance/cases/linear_block.py | Updated performance test thresholds for the linear block component. |
| tripy/tests/backend/api/test_input_info.py | Expanded tests for InputInfo to validate dynamic dimension naming and JSON serialization. |
| tripy/nvtripy/trace/trace.py | Transitioned from using a separate shapes argument to input_infos for capturing input metadata. |
| tripy/nvtripy/frontend/ops/plugin_qdp.py | Updated documentation for the QDP plugin with reference links. |
| tripy/nvtripy/backend/api/named_dimension.py | Added implementation for the NamedDimension API. |
| tripy/nvtripy/backend/api/input_info.py | Modified InputInfo to support named dimensions and updated encoding/decoding logic. |
| tripy/nvtripy/backend/api/executable.py | Minor update: added a docstring for the stream attribute. |
| tripy/nvtripy/backend/api/compile.py | Removed the redundancy of shapes and now passing input_infos during trace construction. |
| tripy/examples/segment-anything-model-v2/video_demo.py | Adjusted mask processing to use torch operations and ensured proper conversion for display. |
| tripy/examples/segment-anything-model-v2/sam2/modeling/memory_attention.py | Removed an unused variable for clarity. |
| tripy/examples/segment-anything-model-v2/sam2/build_sam.py | Replaced tuple-based shape specifications with NamedDimension constructs. |
| tripy/docs/pre0_user_guides/01-compiler.md | Updated documentation to include guidance on using Named Dynamic Dimensions. |
Comments suppressed due to low confidence (2)
tripy/nvtripy/backend/api/input_info.py:110
- [nitpick] Consider including 'dimension_names' in the equality check if they are intended to be part of the input's identity, so that equality reflects the full input state.
return isinstance(other, InputInfo) and self.shape_bounds == other.shape_bounds and self.dtype == other.dtype
tripy/nvtripy/trace/trace.py:114
- [nitpick] Consider retrieving input_infos by matching input names (e.g., using self.input_infos.get(inp.name)) instead of relying on dictionary iteration order to ensure that the correct metadata is associated with each input.
for inp, inp_shape in zip(self.inputs, ([info for info in self.input_infos.values()] if self.input_infos else [None] * len(self.inputs))):
6a04054 to
50d9758
Compare
Adds a NamedDimension API to express runtime equality of dynamic dimensions
Adds perf tests for SDPA
Updates the compiler guide to mention naming dynamic dimensions
Updates SAMv2 to name dimensions in order to trigger MHA fusions