Skip to content

Commit f60f063

Browse files
tushar00jainmeta-codesync[bot]
authored andcommitted
set ranks when initializing pg (meta-pytorch#280)
Summary: Pull Request resolved: meta-pytorch#280 set metadata about ranks on the pg so that it can be used by flight recorder Reviewed By: d4l3k Differential Revision: D84101242 fbshipit-source-id: f5834e2db74f8f65dc3278f71aa6939f1e773784
1 parent 8521e3d commit f60f063

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

torchft/process_group.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -791,6 +791,7 @@ def _create_pg(self, store: Store, rank: int, world_size: int) -> BaseProcessGro
791791
# pyre-fixme[16]: no attribute ProcessGroupNCCL
792792
opts = BaseProcessGroupNCCL.Options()
793793
opts.config.blocking = False
794+
opts.global_ranks_in_group = list(range(world_size))
794795

795796
pg = BaseProcessGroup(store, rank, world_size)
796797
pg._set_default_backend(ProcessGroup.BackendType.NCCL)

0 commit comments

Comments
 (0)