Add ep #39501
Add ep #39501
Conversation
Co-authored-by: Nouamane Tazi <[email protected]> Co-authored-by: drbh <[email protected]>
|
Does not work for mixtral, tricky because of the sequentiality of the weights. I'll add something that allows for this, but also a plan that allows for one the fly merging the modulelist to the format for megablocks |
|
Will getting deepseek working too be pretty straightforward? |
|
Yes and no! for both I used nn.Modulelist() (deepseek I had no choice and Mixtral I was junior) and so it's a bit more annoying, but yes because next pr will make sur we have a bit of a better interface! |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: llama4 |
For testing target_parameters, we use a tiny Llama4 model. This model was refactored in huggingface/transformers#39501, resulting in one parameter being accessed an additional time: https://github.com/huggingface/transformers/pull/39501/files#diff-e668ec07f78afdb2cb805d939e47453757f0b9437436cb860fcb7cb2431c9cf5R69 Therefore, a unit test that relied on how often this parameter was accessed started failing. This PR updates the count to the correct number. Additionally debug print statements that were accidentally left over are now removed.
For testing target_parameters, we use a tiny Llama4 model. This model was refactored in huggingface/transformers#39501, resulting in one parameter being accessed an additional time: https://github.com/huggingface/transformers/pull/39501/files#diff-e668ec07f78afdb2cb805d939e47453757f0b9437436cb860fcb7cb2431c9cf5R69 Therefore, a unit test that relied on how often this parameter was accessed started failing. This PR updates the count to the correct number. Additionally debug print statements that were accidentally left over are now removed.
What does this PR do?
Add support for expert parallel!