Removes unnecessary casting around normalization layers

[pranavm-nvidia]

No description provided.

[Copilot]

Pull Request Overview

This PR removes unnecessary casting around normalization layers and updates their initialization to support a configurable dtype. Key changes include:

• Updating LayerNorm2d to accept a dtype parameter and removing explicit cast calls in its forward methods.
• Removing redundant casting calls across transformer, mask decoder, memory encoder, memory attention, and backbone modules.
• Adjusting the skip_dtype_convert configuration in build_sam.py to reflect that explicit dtype conversion is no longer needed.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
sam2/modeling/sam2_utils.py	Updated LayerNorm2d constructor and forward method to remove redundant casts.
sam2/modeling/sam/transformer.py	Removed unnecessary casting in normalization calls for improved clarity.
sam/modeling/sam/mask_decoder.py	Simplified normalization, removing explicit cast operations.
sam2/modeling/memory_encoder.py	Adjusted LayerNorm2d usage to include dtype and removed casting.
sam2/modeling/memory_attention.py	Cleaned up normalization calls by removing explicit casts.
sam2/modeling/backbones/hieradet.py	Replaced custom normalization helper with direct norm calls.
sam2/build_sam.py	Updated skip_dtype_convert configurations to empty lists to match the new normalization behavior.

Comments suppressed due to low confidence (5)

tripy/examples/segment-anything-model-v2/sam2/build_sam.py:87

• Removing keys from skip_dtype_convert may impact dtype conversion during state dict loading; ensure that this configuration change is intended and thoroughly tested.

            "skip_dtype_convert": [],

tripy/examples/segment-anything-model-v2/sam2/modeling/sam2_utils.py:189

• The removal of explicit casting in the forward method improves clarity; please ensure that the input tensor is already in the expected dtype upstream.

        x = super().forward(x)

tripy/examples/segment-anything-model-v2/sam2/modeling/sam/transformer.py:130

• Replacing redundant cast calls with a direct normalization call simplifies the code; please verify that tensor dtypes are handled correctly by the norm layer.

            )

tripy/examples/segment-anything-model-v2/sam2/modeling/sam/mask_decoder.py:292

• Directly using the normalization layer without explicit casting cleans up the code; please confirm that the tensor dtype remains consistent in downstream operations.

            post_ln1 = ln1(dc1(src))

tripy/examples/segment-anything-model-v2/sam2/modeling/backbones/hieradet.py:153

• Replacing the custom call_norm helper with a direct normalization call improves maintainability; please verify that any previously implicit type casting is now handled by the norm layer.

        x = self.norm1(x)