How to run Explicit, Implicit, Temp Ans., Ordinal's experimental data?

[2399240664]

How to run Explicit, Implicit, Temp Ans., Ordinal's experimental data?

[zhenjia2017]

What do you mean :)

[2399240664]

Now, I can only get the experimental results of all types of questions (Overall) by running Exact. I cannot obtain experimental data of other types of questions, such as Explicit, Implicit, Temp Ans., Ordinal.

[2399240664]

I want to know how I need to operate to obtain these experimental data.

[zhenjia2017]

You can use the type annotation in the benchmark to compute P@1, etc, for each category from the overall results. For example, if you know a question falling into explicit type, and its P@1, etc, you can compute the average P@1 for all questions with explicit type.

[2399240664]

Sorry， In the Exaqt code, I did not find the type annotation are used to calculate P@1.

[zhenjia2017]

I will provide the code soon :）

[zhenjia2017]

The evaluate.py is updated. Now the performance for each category can be computed. Please let me know if there are any problems.

[2399240664]

Thank you.