add has_init

[SebastianM-C]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Add any other context about the problem here.

[SebastianM-C]

@ChrisRackauckas given that init for DifferentialEquations refers to the integrator interface where you can setp! through the integrator, while init for LinearSolve (and Optimization) is more about the cache interface (as pointed out in https://docs.sciml.ai/SciMLBase/stable/interfaces/Init_Solve/#init-and-the-Iterator-Interface), should we name this something like has_cache_init to avoid confusion?

[ChrisRackauckas]

I don't see the difference. You can setp! any of them, they are both caches.

[SebastianM-C]

Ah, sorry, typo I was referring to stepping with step!(integ)(https://docs.sciml.ai/DiffEqDocs/stable/basics/integrator/nothing/DiffEqDocs/stable/basics/integrator/), which we can't do for caches like the optimization one

[ChrisRackauckas]

Optimization caches should also have step! and an iterator interface.

[SebastianM-C]

That would dramatically change how we use the cache though. I think only the optimizes from OptimizationOptimisers allow stepping, I'm not aware of any others that allow that. Right now the Optimization caches are closer to the LinearSolve caches and they mainly cache AD function generation results as far as I understand.

[SebastianM-C]

So should we get rid of https://github.com/SciML/Optimization.jl/blob/d201417e12a5ad7f47b12c15028c8e2ff5afb09b/lib/OptimizationBase/src/solve.jl#L95 and just call solve!(init(prob, alg, args...; kwargs...)) and use the has_init trait to rewrite https://github.com/SciML/Optimization.jl/blob/d201417e12a5ad7f47b12c15028c8e2ff5afb09b/lib/OptimizationBase/src/solve.jl#L204-L206 to do either a loop with step! or __solve?

[ChrisRackauckas]

We should have a trait for has_step. Anything can solve!, but only some can step! iteratively.

[ChrisRackauckas]

That would dramatically change how we use the cache though.

No not to the user. To the user it's just a cache where step! isn't defined yet, and we need to make new methods that do that.

[SebastianM-C]

so the trait introduced in this PR should be renamed to has_step?

[ChrisRackauckas]

no those are two different things. Any inited object can solve!, but not all can step! or iterate.