Read ps3_scales, not s3_scales, from the PS3 vision encoder - #287
Open
appleweiping wants to merge 1 commit into
Open
Read ps3_scales, not s3_scales, from the PS3 vision encoder#287appleweiping wants to merge 1 commit into
appleweiping wants to merge 1 commit into
Conversation
`PS3VisionEncoder` in ps3-torch defines `ps3_scales`; it has no `s3_scales` attribute. Every access of the misspelled name raises AttributeError as soon as a PS3 model is built, so `config.ps3=True` fails before training or inference can start. The name occurs five times: - `multimodal_encoder/builder.py` in `build_vision_tower()`, reached whenever `config.ps3` is set. This is the occurrence reported in NVlabs#282. - `llava_arch.py` in `encode_images_ps3()`, four times, reached when `ps3_dynamic_aspect_ratio` is enabled. Adds two tests that run without torch or ps3-torch installed: one builds a PS3 vision tower against a stubbed encoder and asserts `mm_scale_num` is derived from `ps3_scales`, and one scans the source for the misspelling so the four occurrences on the dynamic-aspect-ratio path stay covered too. The scan matches `.s3_scales` specifically, leaving the unrelated `s2_scales` used by the dynamic-S2 path alone. Both fail against the current code and pass with this change. Fixes NVlabs#282 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Author
|
Validated the latest head b408d23 locally: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #282.
PS3VisionEncoderinps3-torchdefinesps3_scales; there is nos3_scalesattribute on it, so every access of the misspelled name raisesAttributeErroras soon as a PS3 model is built —config.ps3=Truefails before training or inference can start.#282 reports the occurrence in
multimodal_encoder/builder.py. The same misspelling occurs four more times inllava_arch.py'sencode_images_ps3(), on the path reached whenps3_dynamic_aspect_ratiois enabled, so this PR fixes all five.Two tests are included that run without
torchorps3-torchinstalled: one builds a PS3 vision tower against a stubbed encoder and assertsmm_scale_numis derived fromps3_scales; the other scans the source for the misspelling, which keeps the four dynamic-aspect-ratio occurrences covered without needing a GPU. The scan matches.s3_scalesspecifically and leaves the unrelateds2_scalesused by the dynamic-S2 path alone. Both tests fail againstmainand pass with this change.I have not been able to run a full PS3 model end-to-end (no GPU or
ps3-torchin my environment); verification is against theps3-torchsource plus the tests above.🤖 Generated with Claude Code