:mag: add benchmarks

[jvdd]

Add benchmarks - that are reproducible - to this repo.

This would also allow contributors to check whether their changes impact (& possibly improve) performance :)

TODOs

• improve Rust benchmarks
• add Python benchmarks

=> try to make the Rust & Python benchmarks comparable - using same arraylength, n_out, dtype configurations

[jvdd]

• add codspeed to the CI-CD, 🐰 add codspeed benchmarking to ci/cd #38

[jayceslesar]

would be super cool to see some comparisons drawn from your paper -- unsure how the following graphs relate but below is a heatmap showing various differences in distance metrics for 250,000 data points and how they stack up against 10% retention (at most 25k points used in the n_out param)

[jayceslesar]

Could also probably do like a png difference? Save the plot file and actually compare the resulting images somehow?

[jvdd]

Hi @jayceslesar are you referring to the comparisons from this paper?

If so, I guess we can cross-reference our ts-datapoint-selection-vis repo!

As MinMaxLTTB is in its essence a novel algorithm (2-step algo using first MinMax & then LTTB), we ended up writing a desperate (short) paper about it (and did not include it in the ts-datapoint-selection-vis repo). The preprint of this paper will appear online tomorrow :) The benchmarks of MinMaxLTTB its visual representativeness are in our MinMaxLTTB repo

---

Could also probably do like a png difference? Save the plot file and actually compare the resulting images somehow?

That is exactly what we do for a matrix difference! We use DSSIM, PEM_20, and MSE to capture this! :)

[jvdd]

@jayceslesar our paper on MinMaxLTTB https://arxiv.org/abs/2305.00332