Regarding Table 2 in your paper:
The CIDEr metric is designed for multi-reference evaluation, but the dataset used appears to have only a single ground truth (GT) reference per question.
This raises two questions:
How were the IDF statistics for CIDEr computed? Was the corpus built from all single GTs in the test set?
What explains the high CIDEr scores (e.g., 1.437)? The scores are significantly higher than other methods, and we are interested in understanding the reason given the single-GT setup.
Thank you.
Regarding Table 2 in your paper:
The CIDEr metric is designed for multi-reference evaluation, but the dataset used appears to have only a single ground truth (GT) reference per question.
This raises two questions:
How were the IDF statistics for CIDEr computed? Was the corpus built from all single GTs in the test set?
What explains the high CIDEr scores (e.g., 1.437)? The scores are significantly higher than other methods, and we are interested in understanding the reason given the single-GT setup.
Thank you.