mlr3 presentation

[sebffischer]

It would be great if we had some overview slides for mlr3 that briefly summarize its strengths (e.g. hyperparameter tuning, parallelization, benchmarking) also with respect to other frameworks. The idea is to have something similar as toby does for the data.table project.

[tdhock]

I have some discussion of advantages of mlr3 relative to tidymodels and base R for loops here https://tdhock.github.io/blog/2023/comparing-ml-frameworks/ but it would be nice to also discuss scikit-learn and caret.

Best would be a comparison table. Features on the rows and frameworks on the columns, check mark or x to show support or not for each feature in each framework.

Here are some of the advantages I see

• mlr3 benchmark grid includes different resamplings/algos/tasks, but tidymodels workflow sets only are lists of algos (so you would have to write a for loop over tasks and resamplings)
• in mlr3 benchmark experiments, it is easy to get a table of test error values for each split/algo/task, whereas in tidymodels you need to write a for loop over workflow set results.
• parallel benchmark experiments: big win for mlr3 via batchmark() + batchtools.
• also batchtools vs crew.cluster vs clustermq: batchtools cluster back-ends save intermediate results to the file system, which is nice for fault tolerance (others may not return anything if 1 job out of 100 errors out; in batchtools you still have the other 99 results saved to the file system).
• tidymodels -1 for ontology, for example different backend packages for computing a regularized GLM like Lasso. this is not as flexible as mlr3 which offers some learners like cv_glmnet with auto tuning built in.
• mlr3 has more consistent nomenclature in docs (for example names of split sets are not consistently named in tidymodels docs)
• tidymodels does not expose as many parameters as mlr3 (for example scale parameter in nearest neighbors)
• scikit-learn provides implementations of each learner, where mlr3 interfaces with other packages that provide the implementation