Handle tracks with heavy custom track functions #754
Description
Msmarco, and the new openai track have a feature where they load queries from a bz2 file - these files are quite bit, and take a lot of time to load. The result is that even in system tests, it can add a lot of time to the run.
msmarco added 50 minutes to run time, openai 11.
We have got around this by skipping, but that could be dangerous that we are not testing all tracks and challenges.
One alternative would be to have a much smaller file (similar to how we do with corpora) that is used instead when running in system tests. The custom runners dont know at the moment if they are in system test mode or not, so an extra parameter would be required.