How did the authors perform hyperparameter selection for the pretrained mdoels?

[MogicianXD]

I found that the pretraining phase does not involve any validation data, and instead, saves checkpoints per epoch. I wonder how the authors decided early stop and hyperparameter tuning. Did the authors run all checkpoints on the evaluation benchmarks?

[gorold]

No early stopping, checkpoints mainly for resuming if the job fails. Also no hyperparam tuning.

[MogicianXD]

Thanks for your reply!