|
47 | 47 | ## News
|
48 | 48 | 🔥🔥 [2024/01/17] We released MFTCoder v0.3.0, mainly for MFTCoder-accelerate. It now supports new models like Mixtral(MoE), DeepSeek-coder, chatglm3. It supports FSDP as an option. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.
|
49 | 49 |
|
50 |
| -🔥🔥 [2024/01/17] [CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) has been released, achieving a pass@1 (greedy decoding) score of 78.7% on HumanEval. It achieves top1 win-rate on Bigcode Leardboard. |
| 50 | +🔥🔥 [2024/01/17] [CodeFuse-DeepSeek-33B](https://huggingface.co/codefuse-ai/CodeFuse-DeepSeek-33B) has been released, achieving a pass@1 (greedy decoding) score of 78.7% on HumanEval. It lists as top-1 LLM on Bigcode Leardboard in terms of win-rate. |
51 | 51 |
|
52 | 52 | 🔥🔥 [2024/01/17] [CodeFuse-Mixtral-8x7B](https://huggingface.co/codefuse-ai/CodeFuse-Mixtral-8X7B) has been released, achieving a pass@1 (greedy decoding) score of 56.1% on HumanEval.
|
53 | 53 |
|
|
61 | 61 |
|
62 | 62 | 🔥🔥🔥 [2023/09/07]We released **CodeFuse-CodeLlama-34B**, which achieves the **74.4% Python Pass@1** (greedy decoding) and surpasses GPT4 (2023/03/15) and ChatGPT-3.5 on the [HumanEval Benchmarks](https://github.com/openai/human-eval).
|
63 | 63 |
|
64 |
| -🔥🔥 [2023/08/26]We released MFTCoder which supports finetuning Code Llama, Llama, Llama2, StarCoder, ChatGLM2, CodeGeeX2, Qwen, and GPT-NeoX models with LoRA/QLoRA. |
| 64 | +🔥🔥 [2023/08/26]We released MFTCoder-v0.1 which supports finetuning Code Llama, Llama, Llama2, StarCoder, ChatGLM2, CodeGeeX2, Qwen, and GPT-NeoX models with LoRA/QLoRA. |
65 | 65 |
|
66 | 66 | ### HumanEval Performance
|
67 | 67 | | Model | HumanEval(Pass@1) | Date |
|
68 | 68 | |:----------------------------|:-----------------:|:-------:|
|
69 | 69 | | **CodeFuse-DeepSeek-33B** | **78.7%** | 2024/01 |
|
70 |
| -| **CodeFuse-Mixtral-8x7B** | **56.1%** | 2024/01 | |
71 | 70 | | **CodeFuse-CodeLlama-34B** | **74.4%** | 2023/09 |
|
72 | 71 | | **CodeFuse-CodeLlama-34B-4bits** | **73.8%** | 2023/09 |
|
73 | 72 | | WizardCoder-Python-34B-V1.0 | 73.2% | 2023/08 |
|
74 | 73 | | GPT-4(zero-shot) | 67.0% | 2023/03 |
|
75 | 74 | | PanGu-Coder2 15B | 61.6% | 2023/08 |
|
| 75 | +| **CodeFuse-Mixtral-8x7B** | **56.1%** | 2024/01 | |
76 | 76 | | **CodeFuse-StarCoder-15B** | **54.9%** | 2023/08 |
|
77 | 77 | | CodeLlama-34b-Python | 53.7% | 2023/08 |
|
78 | 78 | | **CodeFuse-QWen-14B** | **48.8%** | 2023/10 |
|
|
0 commit comments