Skip to content

Betswish/mRAG-Context-Consistency

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
correlation
correlation
 
 
0_embed_GMMLU.sh
0_embed_GMMLU.sh
 
 
0_embed_MKQA.sh
0_embed_MKQA.sh
 
 
10_LLM_eval.sh
10_LLM_eval.sh
 
 
11_LLM_eval_check.sh
11_LLM_eval_check.sh
 
 
12_count_mirage_multi.sh
12_count_mirage_multi.sh
 
 
1_retrieval_GMMLU.sh
1_retrieval_GMMLU.sh
 
 
1_retrieval_MKQA.sh
1_retrieval_MKQA.sh
 
 
2_filter_subset_GMMLU.sh
2_filter_subset_GMMLU.sh
 
 
2_filter_subset_MKQA.sh
2_filter_subset_MKQA.sh
 
 
3_GMMLU_choice.sh
3_GMMLU_choice.sh
 
 
3_GMMLU_open.sh
3_GMMLU_open.sh
 
 
3_MKQA_open.sh
3_MKQA_open.sh
 
 
3_XQUAD_open.sh
3_XQUAD_open.sh
 
 
4_check_performance.sh
4_check_performance.sh
 
 
5_attribute_MKQA.sh
5_attribute_MKQA.sh
 
 
5_attribute_XQUAD.sh
5_attribute_XQUAD.sh
 
 
6_count_mirage.sh
6_count_mirage.sh
 
 
7_GMMLU_choice_multi.sh
7_GMMLU_choice_multi.sh
 
 
7_MKQA_open_multi.sh
7_MKQA_open_multi.sh
 
 
8_check_performance_multi.sh
8_check_performance_multi.sh
 
 
9_attribute_GMMLU_multi.sh
9_attribute_GMMLU_multi.sh
 
 
9_attribute_MKQA_multi.sh
9_attribute_MKQA_multi.sh
 
 
GMMLU_choice.py
GMMLU_choice.py
 
 
GMMLU_choice_multi.py
GMMLU_choice_multi.py
 
 
GMMLU_open.py
GMMLU_open.py
 
 
MKQA_open.py
MKQA_open.py
 
 
MKQA_open_multi.py
MKQA_open_multi.py
 
 
RAGConsis.yaml
RAGConsis.yaml
 
 
README.md
README.md
 
 
XQUAD_open.py
XQUAD_open.py
 
 
attribute_GMMLU_choice_multi.py
attribute_GMMLU_choice_multi.py
 
 
attribute_MKQA_open.py
attribute_MKQA_open.py
 
 
attribute_MKQA_open_multi.py
attribute_MKQA_open_multi.py
 
 
attribute_XQUAD_open.py
attribute_XQUAD_open.py
 
 
check_GMMLU_choice.py
check_GMMLU_choice.py
 
 
check_GMMLU_choice_multi.py
check_GMMLU_choice_multi.py
 
 
check_GMMLU_choice_multi.txt
check_GMMLU_choice_multi.txt
 
 
check_GMMLU_open.py
check_GMMLU_open.py
 
 
check_LLM_eval.py
check_LLM_eval.py
 
 
check_MKQA.py
check_MKQA.py
 
 
check_MKQA_multi.py
check_MKQA_multi.py
 
 
check_MKQA_multi.txt
check_MKQA_multi.txt
 
 
check_XQUAD.py
check_XQUAD.py
 
 
check_performance.txt
check_performance.txt
 
 
count_mirage.py
count_mirage.py
 
 
count_mirage_GMMLU_choice_multi_1.0_-5.json
count_mirage_GMMLU_choice_multi_1.0_-5.json
 
 
count_mirage_MKQA_open_1.5.txt
count_mirage_MKQA_open_1.5.txt
 
 
count_mirage_MKQA_open_1.txt
count_mirage_MKQA_open_1.txt
 
 
count_mirage_MKQA_open_2.5.txt
count_mirage_MKQA_open_2.5.txt
 
 
count_mirage_MKQA_open_2.txt
count_mirage_MKQA_open_2.txt
 
 
count_mirage_MKQA_open_3.txt
count_mirage_MKQA_open_3.txt
 
 
count_mirage_MKQA_open_multi_1.0_-5.json
count_mirage_MKQA_open_multi_1.0_-5.json
 
 
count_mirage_XQUAD_open_1.5.txt
count_mirage_XQUAD_open_1.5.txt
 
 
count_mirage_XQUAD_open_1.txt
count_mirage_XQUAD_open_1.txt
 
 
count_mirage_XQUAD_open_2.5.txt
count_mirage_XQUAD_open_2.5.txt
 
 
count_mirage_XQUAD_open_2.txt
count_mirage_XQUAD_open_2.txt
 
 
count_mirage_XQUAD_open_3.txt
count_mirage_XQUAD_open_3.txt
 
 
count_mirage_multi.py
count_mirage_multi.py
 
 
embed_GMMLU.py
embed_GMMLU.py
 
 
embed_MKQA.py
embed_MKQA.py
 
 
error_analysis.py
error_analysis.py
 
 
error_analysis_es.json
error_analysis_es.json
 
 
error_analysis_zh.json
error_analysis_zh.json
 
 
filter_golddis_inlang_GMMLU.py
filter_golddis_inlang_GMMLU.py
 
 
filter_golddis_inlang_MKQA.py
filter_golddis_inlang_MKQA.py
 
 
filter_subset_cross_GMMLU.py
filter_subset_cross_GMMLU.py
 
 
filter_subset_inlang_GMMLU.py
filter_subset_inlang_GMMLU.py
 
 
filter_subset_inlang_MKQA.py
filter_subset_inlang_MKQA.py
 
 
fix_data_GMMLU.py
fix_data_GMMLU.py
 
 
fix_data_cross_GMMLU.py
fix_data_cross_GMMLU.py
 
 
has_answer_list_GMMLU.txt
has_answer_list_GMMLU.txt
 
 
has_answer_list_MKQA.txt
has_answer_list_MKQA.txt
 
 
instruction_choice.json
instruction_choice.json
 
 
instruction_open.json
instruction_open.json
 
 
retrieval_cross_GMMLU.py
retrieval_cross_GMMLU.py
 
 
retrieval_cross_MKQA.py
retrieval_cross_MKQA.py
 
 
retrieval_inlang_GMMLU.py
retrieval_inlang_GMMLU.py
 
 
retrieval_inlang_MKQA.py
retrieval_inlang_MKQA.py
 
 
statistics_GMMLU.py
statistics_GMMLU.py
 
 
statistics_GMMLU.txt
statistics_GMMLU.txt
 
 
statistics_MKQA.py
statistics_MKQA.py
 
 
statistics_MKQA.txt
statistics_MKQA.txt
 
 

Repository files navigation

(Work in progress) On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation

Quick Start

You can reproduce the paper’s results using the bash scripts 0_embed_MKQA.sh through 12_count_mirage_multi.sh.

We are refining this repository and expect to complete the updates by the end of November 2025.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages