Skip to content

Commit 7b8e39e

Browse files
committed
Revert "[Dev] fix(megatron-fsdp): Resolve hang caused by non-deterministic reduce-scatter (#2252)"
This reverts commit c6e2b29.
1 parent 716bb4a commit 7b8e39e

File tree

1 file changed

+0
-3
lines changed

1 file changed

+0
-3
lines changed

megatron/core/distributed/fsdp/src/megatron_fsdp/param_and_grad_buffer.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2782,9 +2782,6 @@ def reduce_gradients(
27822782
outer_fsdp_group_grad_reduce (bool, optional): Whether to reduce gradients
27832783
across outer-DP groups. Defaults to False.
27842784
"""
2785-
# Sort parameters by their bucket IDs to ensure a deterministic processing order.
2786-
# Performing reduce-scatter operations out of order can lead to hangs.
2787-
params = sorted(list(params), key=lambda x: self.buffer.param_to_param_group[x])
27882785
for param in params:
27892786
bucket_id = self.buffer.param_to_param_group[param]
27902787
param_group = self.buffer.parameter_groups[bucket_id]

0 commit comments

Comments
 (0)