Hyper AutoOptimize

[kanav99]

What would be the best strategy for finding the right estimate of time to Hyper AutoOptimize? Should we perform a single step and benchmark the time taken and use it as a quantity for what is best?
Also would the greedy approach do? For eg we optimize weather we want to use autodiff or not and weather we want to use GPU or not. We can find this out in 2 ways -

1. Check if it is faster with AD or not with GPU turned off. Then if it is faster with AD, we try GPU with AD. For n parameters, we try 2n configurations
2. We check every configuration i.e. 2^n

[ChrisRackauckas]

What would be the best strategy for finding the right estimate of time to Hyper AutoOptimize? Should we perform a single step and benchmark the time taken and use it as a quantity for what is best?

Hyper auto-optimize should be concerned about algorithm, while auto-optimize can only be concerned about implementation. Things that are essentially algorithmically equivalent (in the mathematical sense) are things like:

• AD or not to AD?
• Static arrays, GPUs, or normal arrays?
• In-place or not?
• Sparsity?

With some quick calls to f those can be determined, so those can be useful even if you're just trying to solve the ODE one time. That's auto_optimize.

hyper_auto_optimize is: I am going to solve this thing 100,000 times, what else should I be doing? At this higher level, this is where we can explore algorithmic details. In order to reduce the computational complexity of this determination, it should be structured at a high level via a decision tree.

• First auto_optimize. Now you should have a faster problem.
• Does the problem seem stiff or not, or in the middle? Solve with an RK method and an implicit method, with limits. See what works better. Try a ROCK method in there if stiffness seems somewhat possible.
• If it's non-stiff, try and see the difference between VCABM vs Vern9 vs Tsit5 vs ?.
• If it's stiff, the gambit of methods depends on size, but Rodas5 vs CVODE_BDF vs ..., and is dependent on GPUs and such
• Try some linear solvers. GMRES or not?

Kick out a _prob,_alg pair that is likely to be the fastest.