P2P intra-node tput #581
Description
The intra-node p2p tput using the copy engine is still low, compared to NCCL/RCCL's SM copy. To reproduce: torchrun --nproc_per_node=8 benchmark_uccl_collective.py --ring. Is this inherent to the copy engine?
{{ message }}