Enriched tool-independent experiment annotation

[AHaumer]

There are a few settings stored in an annotation common to all tools: StartTime, StopTime, Interval, Tolerance.
I think it's worth to discuss which settings shouldn't be stored in a tools specific annotation but in a common one,
e.g. Output(Protected variables, Equidistant time grid, Store variables at events, ...)

[HansOlsson]

I can understand that feature, but there's also a need to innovate and replace them with something better:

• Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose that
• To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.
• In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.

I don't know what other tools do for these cases.

[bilderbuchi]

• To me "non-equidistant grid" is not ideal for that, but kind of works.

I'd be interested in what aspect of this you feel is not ideal, could you maybe elaborate?
AFAICT/in my experience, this results in the "grid rate" as dense as the solver needs to observe the required tolerance. Isn't this the "more [actually, fully] automatic way" you are thinking about? It gives you detail where the simulation requires it because "interesting" things happen quickly, and takes larger steps where the transients are slow?

[HansOlsson]

• To me "non-equidistant grid" is not ideal for that, but kind of works.

I'd be interested in what aspect of this you feel is not ideal, could you maybe elaborate? AFAICT/in my experience, this results in the "grid rate" as dense as the solver needs to observe the required tolerance.

There are some reason I find it non-ideal: The solver uses a higher order interpolation - the solution does not (well, depending on tool) so storing once per step is a bit crude and this makes it messy when comparing different solvers. This is especially noticeable for periodic signals, especially after the transient.

[casella]

I can understand that feature, but there's also a need to innovate and replace them with something better:

* Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose that

OpenModelica also allows that, by using regexp filters on variable names. But that's a different feature.

I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables. People have been happily using this feature for years. The only problem is, it's not standardized, so when you switch tool, it doesn't work. Why don't we just start standardizing stuff that is well-established? This doesn not preclude doing something fancier later.

* To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.

For some users (e.g. @luizmac in OpenModelica/OpenModelica#5709) the requirement is to get periodically sampled outputs, to which you can, e.g., apply FFT analysis or discrete-time system identification techniques. In all such cases, the requirement is to have exactly one data point after every Interval of time, and nothing else.

This requirement is absolutely reasonable and I believe it would deserve an experiment annotation.

* In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.

I don't know what other tools do for these cases.

OpenModelica at the moment is providing what solvers like DASSL or IDA already provide, i.e., either they internally advance the simulation with a variable step size solver and return periodically sampled solutions via dense output (what has been informally called "Equidistant time grid" so far, or they just return the solution at the steps taken by the solver, which is what you get if you switch off "Equidistant time grid". I agree with @HansOlsson that "No Equidistant time grid" kind of works when systems have fast transients every now and then, because you get more tight time points where you need them. But in some cases it gives serious aliasing problems, see the discussion in this ticket.

One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.

[bilderbuchi]

One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.

With "natively", am I guessing correctly that you mean "both accessible from one unchanged function call" or somesuch?

If not, AFAICT this is possible e.g. in CVODE, but you need some logic in your solver-calling code to e.g. do internal solver steps with CV_ONE_STEP, and if regular output times have been stepped over, interpolate those results from the solver output at high accuracy.
I suspect the API for IDA will be equivalent.

[HansOlsson]

I can understand that feature, but there's also a need to innovate and replace them with something better:

* Instead of just storing (or not storing) protected variables tools (such as Dymola) may allow you to select (or filter) variables in more advanced ways. I don't know if we could propose that

OpenModelica also allows that, by using regexp filters on variable names. But that's a different feature.

I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables. People have been happily using this feature for years. The only problem is, it's not standardized, so when you switch tool, it doesn't work. Why don't we just start standardizing stuff that is well-established? This doesn not preclude doing something fancier later.

Sure, I was just saying that we should think beyond this as well.

* To me not storing at events is an indication that you have something special causing "events" (such as discretized variables for a clock) where some more specialized solution would be better suited.

For some users (e.g. @luizmac in OpenModelica/OpenModelica#5709) the requirement is to get periodically sampled outputs, to which you can, e.g., apply FFT analysis or discrete-time system identification techniques. In all such cases, the requirement is to have exactly one data point after every Interval of time, and nothing else.

To me the FFT (etc) should ideally pre-process the data to get that. That's at least what you can do in Dymola, so that you don't have to simulate it once for nice plotting and once for FFT-analysis.

This requirement is absolutely reasonable and I believe it would deserve an experiment annotation.

* In the linked issue the problem is how to capture transients and normal simulations in an efficient way. To me "non-equidistant grid" is not ideal for that, but kind of works. One possibility (implemented in Dymola) is to manually modify the grid rate during the simulation, but there might be more automatic ways.

I don't know what other tools do for these cases.

OpenModelica at the moment is providing what solvers like DASSL or IDA already provide, i.e., either they internally advance the simulation with a variable step size solver and return periodically sampled solutions via dense output (what has been informally called "Equidistant time grid" so far, or they just return the solution at the steps taken by the solver, which is what you get if you switch off "Equidistant time grid". I agree with @HansOlsson that "No Equidistant time grid" kind of works when systems have fast transients every now and then, because you get more tight time points where you need them. But in some cases it gives serious aliasing problems, see the discussion in this ticket.

One option that has been put forth is to output the solution both at regular intervals and at variable time steps, though I understand this feature is not available natively in solvers such as IDA.

I was also considering this.

[otronarp]

I only know Dymola and OpenModelica; both have a checkbox (default: unchecked) for storing protected variables.

To add to the list, System Modeler also has that (with the same default).

[HansOlsson]

Language group meeting:

Additional ideas from System Modeler:

• Event hysteresis scaling; not in Dymola
• Non-solver tolerance (separate from simulation tolerance); not in Dymola

Storing protected seems like a good idea.
Improved equidistant needs investigation.

[HansOlsson]

Note: From a Dymola perspective it is deliberate choice to not expose the settings for event hysteresis and non-linear solver tolerance (even if they at some point in time existed), as they are no longer fixed settings.

[HansOlsson]

I thus think we can get agreement on setting for storing protected, and non-equidistant if we reformulate it to indicate that one wants to store transients i.e., more frequently than the equidistant grid - but indicate it is a quality of implementation to also store the equidistant grid.

If people don't want the combination grid it seems obvious that you can enable this and set the interval to be equal to the simulation interval to effectively ignore the grid.