【开源实习】bigbird_pegasus模型微调 (#1972)

Author: outbreak-sen

llm/finetune/bigbird_pagesus/README.md

@@ -0,0 +1,25 @@
+# bigbird_pegasus模型微调对比
+## train loss
+
+对比微调训练的loss变化
+
+| epoch | mindnlp+mindspore | transformer+torch（4060） |
+| ----- | ----------------- | ------------------------- |
+| 1     | 2.0958            | 8.7301                    |
+| 2     | 1.969             | 8.1557                    |
+| 3     | 1.8755            | 7.7516                    |
+| 4     | 1.8264            | 7.5017                    |
+| 5     | 1.7349            | 7.2614                    |
+| 6     | 1.678             | 7.0559                    |
+| 7     | 1.6937            | 6.8405                    |
+| 8     | 1.654             | 6.7297                    |
+| 9     | 1.6365            | 6.7136                    |
+| 10    | 1.7003            | 6.6279                    |
+
+## eval loss
+
+对比评估得分
+
+| epoch | mindnlp+mindspore  | transformer+torch（4060） |
+| ----- | ------------------ | ------------------------- |
+| 1     | 2.1257965564727783 | 6.3235931396484375        |