Skip to content

Add support for MCR-DL to Megatron-LM #15

@Quentin-Anthony

Description

@Quentin-Anthony

Similar to deepspeedai/Megatron-DeepSpeed#50

We want a Megatron fork based on MCR-DL. Need to create a new branch of https://github.com/OSU-Nowlab/Megatron-LM and replace all calls to torch.dist with mcr-dl.

I'll test once ready.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions