Skip to content

Add new Chinese NLP results #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Add new Chinese NLP results #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
59d4433
Add a paper published in EMNLP-2020
yuanheTian Apr 5, 2022
2a527aa
Add new results on NLPCC DBQA 2016
yuanheTian Apr 5, 2022
1b5201d
Add new results on ChnSentiCorp
yuanheTian Apr 5, 2022
fe26552
Add new results on NER MSRA and Weibo
yuanheTian Apr 5, 2022
3fb1037
update the results on SIGHAN 2005
yuanheTian Apr 5, 2022
77bfb8e
Update other resources
yuanheTian Apr 5, 2022
ae23d81
update other resources
yuanheTian Apr 5, 2022
5667db3
update other resources
yuanheTian Apr 5, 2022
35facb6
Add new results for word embeddings
yuanheTian Apr 6, 2022
4aa964d
Add new results for constituency parsing
yuanheTian Apr 6, 2022
1d0e0ee
Add new results on NER
yuanheTian Apr 6, 2022
f4eba9e
Add new topic classification results
yuanheTian Apr 6, 2022
e3e79a2
Add new sentiment analysis results
yuanheTian Apr 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/constituency_parsing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,10 @@ EM, F1, LP and LR can be calculated using the [Evalb](https://nlp.cs.nyu.edu/eva
| System | EM | F1 | LP | LR | code |
| --- | --- | --- | --- | --- | --- |
| [Liu and Zhang (2017)](https://www.aclweb.org/anthology/Q17-1029/) | 44.94 | 91.81 | - | - | [Github](https://github.com/dpfried/rnng-bert) |
| [Fried et al. (2019)](https://aclanthology.org/P19-1031/) | 44.42 | 92.14 | - | - | [Github](https://github.com/dpfried/rnng-bert) |
| [Zhou and Zhao (2019)](https://www.aclweb.org/anthology/P19-1230/) | - | 92.18 | 92.33 | 92.03 | [Github](https://github.com/DoodleJZ/HPSG-Neural-Parser) |
| [Mrini et al. (2020)](https://arxiv.org/abs/1911.03875) | - | 92.64 | 93.45 | 91.85 | [Github](https://github.com/KhalilMrini/LAL-Parser) |
| [Tian et al. (2020)](https://aclanthology.org/2020.findings-emnlp.153/) | - | 92.66 | 92.50 | 92.83 | [Github](https://github.com/cuhksz-nlp/SAPar) |
| [Yang and Deng (2020)](https://arxiv.org/abs/2010.14568) | 49.72 | 93.59 | 93.80 | 93.40 | [Github](https://github.com/princeton-vl/attach-juxtapose-parser) |


Expand Down
4 changes: 4 additions & 0 deletions docs/entity_tagging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,11 @@ Paper summarizing the bakeoff:

| System | F-score |
| --- | --- |
| [Song et al. (2021)](https://arxiv.org/abs/2105.01279) | 96.2 |
| [Liu et al (2020)](https://arxiv.org/pdf/1909.07606.pdf) | 95.7 |
| [Meng et. al. (2019)](https://arxiv.org/abs/1901.10125) | 95.5 |
| [Ma et al (2020)](https://www.aclweb.org/anthology/2020.acl-main.528.pdf) | 95.4 |
| [Diao et al. (2020)](https://aclanthology.org/2020.findings-emnlp.425/)| 95.3 |
| [Sun et al (2020)](https://arxiv.org/pdf/1907.12412.pdf) | 95.0 |
| [Yan et al (2020)](https://ieeexplore.ieee.org/abstract/document/9141551) | 94.1 |
| [Liu et. al. (2019)](https://www.aclweb.org/anthology/N19-1247) | 93.74 |
Expand Down Expand Up @@ -142,6 +144,8 @@ Using the test split by http://www.aclweb.org/anthology/E17-2113:
| System | F-score (name mentions) | F-score (nominal mentions) | F-score (Overall) |
| --- | --- | --- | --- |
| [Ma et al (2020)](https://www.aclweb.org/anthology/2020.acl-main.528.pdf) | 70.9 | 67.0 | 70.5 |
| [Nie et al. (2020)](https://aclanthology.org/2020.emnlp-main.107/)| | | 69.80 |
| [Nie et al. (2020)](https://aclanthology.org/2020.findings-emnlp.378/)| | | 69.78 |
| [Meng et. al. (2019)](https://arxiv.org/abs/1901.10125) | 67.6 | | |
| [Hu and Zheng (2020)](https://www.jstage.jst.go.jp/article/transinf/E103.D/7/E103.D_2019EDP7253/_pdf/-char/ja) | 56.4 | | |
| [Sui et al. (2019)](https://www.aclweb.org/anthology/D19-1396/) | 56.45 | 68.32 | 63.09 |
Expand Down
1 change: 1 addition & 0 deletions docs/question_answering.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ NLPCC DBQA 2016

| System | MRR | F1 |
| --- | --- | --- |
| [ZEN 2.0](https://arxiv.org/abs/2105.01279) | 96.1 | 86.5 |
| [ERNIE 2.0](https://arxiv.org/pdf/1907.12412.pdf) | 95.8 | 85.8 |
| [Meng et. al. (2019)](https://arxiv.org/pdf/1901.10125.pdf) (Glyce + BERT) | - | 83.4 |
| [ERNIE(baidu)](https://arxiv.org/pdf/1904.09223.pdf) | 95.1 | 82.7 |
Expand Down
2 changes: 2 additions & 0 deletions docs/sentiment_analysis.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ F1-score
| | F1 | Accuracy |
| --- | --- | --- |
| [Chen et al., 2020: 3SiBert](https://www.aclweb.org/anthology/2020.lrec-1.293.pdf) | | 0.967 | https://www.aclweb.org/anthology/2020.lrec-1.293.pdf
| [ZEN 2.0](https://arxiv.org/abs/2105.01279) | | 0.965 |
| [ZEN](https://aclanthology.org/2020.findings-emnlp.425/) | | 0.961 |
| [ERNIE 2.0](https://arxiv.org/pdf/1907.12412.pdf) | | 0.958 |
| [ERNIE](https://arxiv.org/pdf/1904.09223.pdf) | | 0.954 |
| BERT * | | 0.943 |
Expand Down
3 changes: 3 additions & 0 deletions docs/topic_classification.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ Sina News RSS subscription channel data from 2005 to 2011, which contains 74 mil
| | Accuracy |
| --- | --- |
| [J. Chen, C. Cao, X. Jiang](https://www.aclweb.org/anthology/2020.lrec-1.293.pdf) | 98.7% |
| [Song, et al., (2021)](https://arxiv.org/abs/2105.01279) | 97.93% |
| [Cui, et al., (2020)](https://arxiv.org/pdf/2004.13922.pdf) | 97.9% |
| [Diao, et al., (2020)](https://aclanthology.org/2020.findings-emnlp.425/) | 97.64% |
| [Y. Song](https://iopscience.iop.org/article/10.1088/1742-6596/1453/1/012156/pdf)| 97.56% |
| [W. Liu, P. Zhou, et al](https://www.aclweb.org/anthology/2020.acl-main.537.pdf) | 96.71% |
| [S. Xin](https://iopscience.iop.org/article/10.1088/1742-6596/1549/2/022011/pdf) | 96.04% |
Expand Down
3 changes: 3 additions & 0 deletions docs/word_embedding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ See e.g. [Torregrossa et al., 2020](https://www.aclweb.org/anthology/2020.lrec-1
| System | wordsim-240 (⍴) | wordsim-296 (⍴) |
| --- | --- | --- |
| [Sun et. al. (2019)](https://arxiv.org/pdf/1902.08795.pdf) (VCWE) | 57.81 | 61.29 |
| [Song et. al. (2018)](https://www.ijcai.org/Proceedings/2018/0608.pdf) | 54.14 | 57.04 |
| [Yu et. al. (2017)](https://www.aclweb.org/anthology/D17-1027) (JWE) | 51.92 | 59.84 |
| Baseline (CBOW) | 51.01 | 53.65 |



Expand Down Expand Up @@ -146,6 +148,7 @@ Given “France : Paris :: China : ?”, a system should come up with the answer

| Name | Additional features | Training Corpus Size | Source |
| --- | --- | --- | --- |
| DSG | Leverage directional information to improve skip-gram algorithm | | [Song et al., 2018](https://aclanthology.org/N18-2028/) |
| FastText | - | 374M characters | [Grave et al., 2018](https://arxiv.org/pdf/1802.06893.pdf) |
| Mimick | Interpolate between similar characters to improve rare words, multilingual | | [Pinter et al., 2017](https://www.aclweb.org/anthology/D17-1010.pdf) |
| Glyph2vec | Uses character bitmaps, canjie to address OOV problem | 10M chars | [Chen et al., 2020](https://www.aclweb.org/anthology/2020.acl-main.256.pdf) |
Expand Down
2 changes: 1 addition & 1 deletion docs/word_segmentation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ F1 = 0.857
| Model | AS | CITYU | MSRA | PKU |
| --- | --- | --- | --- | --- |
| [Ke et al. (2021)](https://aclanthology.org/2021.naacl-main.436/) | 97.0 | 98.2 | 98.5 | 96.9 |
| [Qiu, Pei, Yan, Huang (2020)](https://aclanthology.org/2020.findings-emnlp.260/) | 96.4 | 96.9 | 98.1 | 96.4 |
| [Tian, Song, Xia, Zhang, Wang (2020)](https://www.aclweb.org/anthology/2020.acl-main.734/) | 96.6 | 97.9 | 98.4 | 96.5 |
| [Qiu, Pei, Yan, Huang (2020)](https://aclanthology.org/2020.findings-emnlp.260/) | 96.4 | 96.9 | 98.1 | 96.4 |
| [Meng et al. (2019)](https://arxiv.org/abs/1901.10125) | 96.7<sup>*</sup> | 97.9<sup>*</sup> | 98.3 | 96.7 |
| [Huang et al. (2019)](https://arxiv.org/abs/1903.04190)| 96.6 | 97.6 | 97.9 | 96.6 |
| [Ma et al. (2018)](http://aclweb.org/anthology/D18-1529) | 96.2 | 97.2 | 97.4 | 96.1 |
Expand Down