feat: allow selecting last simulation table only - #3939
Open
JasonMarechal25 wants to merge 86 commits into
Open
JasonMarechal25 wants to merge 86 commits into
JasonMarechal25 wants to merge 86 commits into
Conversation
…ctivity checks Introduced the `InactiveComponentsAnalyzer` class to identify and manage structurally inactive components in simulation tables. Integrated the analyzer into `LegacyExtraOutputs` and added comprehensive test cases to validate behavior. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Added methods to detect and exclude components with entirely zero data from simulation outputs. Updated scenarios and steps in tests to validate the absence of such components. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…ver modules Connected the `InactiveComponentsAnalyzer` to simulation workflows for detecting and handling inactive components in various solver modules. Enhanced modularity with the addition of `BuildInactiveComponentsAnalyzer`. Updated relevant outputs to skip inactive components. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…ro inflow inactivity Added support for detecting all-zero hydro inflow series in `InactiveComponentsAnalyzer`. Updated `LegacyExtraOutputs` and simulation workflows to suppress hydro balance_port.flow rows when reservoirs are unmanaged and inflow is entirely zero. Enhanced test coverage for these cases. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…k inactivity Added support in `InactiveComponentsAnalyzer` for detecting all-zero link capacities across both directions. Updated `LegacyExtraOutputs` to suppress link-related rows when flagged as all-zero. Enhanced test coverage and simulation workflows to validate these changes. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…ulation tables Added `has_no_rows_for_component_and_output` to support fine-grained checks for component-output combinations. Updated scenarios and step definitions to validate the absence of specific derived output rows in simulation tables. Enhanced test coverage to align with updated requirements. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
* Introduce unit tests for BuildInactiveComponentsAnalyzer. * Validate that components are marked inactive when only selected chronicles are all-zero. * Ensure correct behavior for load series with multiple chronicles. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…for stage-based outputs * Extract `FillLegacySimulationTable` into a dedicated file and include it in the solver module. * Refactor `OptimisationsSimulationTable` to dynamically manage stage-based simulation tables. * Update `LegacySimulationTablesWriter` to handle stage naming and output file generation per stage. * Add unit tests to validate dynamic table creation, pointer stability, emptying stages, and file generation workflows. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…ines
`LegacySimulationTableSnapshot` was written against
`Antares::Optimisation::LinearProblemApi`, the name the modeler API carries on
the lineage this work was cut from. `feature/ignore-inactive-component` calls it
`Antares::LinearProblem::Api`, so after rebasing onto it the reference dangles
and the header does not compile:
error: no type named 'FillContext' in namespace
'Antares::Optimisation::LinearProblemApi'
Point at the base's names. No behaviour change -- these are the same types.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…dumping * Introduced `DumpLegacySimulationTableAfterPostProcess` to handle result dumping after post-processing. * Added `LegacyWeeklyBlock` utility to derive weekly intervals from hour markers. * Improved memory management by introducing `SolutionRefreshedFromResults` for temporary state preservation during table fills. * Enhanced test coverage for post-processing scenarios, including state preservation and daily/weekly range handling. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…process simulation table dumping * Introduced `DumpSimulationTablePostProcessCmd` to handle simulation table dumping after specific post-process stages. * Updated `OptPostProcessList` and `AdqPatchPostProcessList` to include the new command for `remixHydroStage` and `adequacyPatchStage`. * Extended constructor parameters across post-process lists to support `simulationTables`. * Enhanced memory handling and ensured compatibility with null `simulationTables` scenarios. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
* Renamed `DumpLegacySimulationTableAfterPostProcess` to `DumpSimulationTableAfterPostProcess` for improved clarity and alignment with hybrid workflows. * Introduced `SolvedModelerProblem` struct to retain the modeler state post-solve for enhanced stage-based output handling. * Updated `fillLinearProblem` to use shared pointers for `OptimEntityContainer` to ensure lifecycle management during post-process table dumping. * Extended headers and module dependencies to support new memory and state-handling structures. * Adjusted tests and CMakeLists to align with these changes, ensuring comprehensive test coverage. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
* Updated `LegacySimulationTablesWriter` to omit stages with no rows, preventing the creation of misleading header-only files. * Added unit test `WriteTo_SkipsStagesWithNoRows` to validate this behavior. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…simulation table dumping * Added `DumpSimulationTableStage` to streamline simulation table dumping across post-process and adequacy workflows. * Updated `DumpSimulationTablePostProcessCmd` to use the new utility, reducing redundancy and improving maintainability. * Integrated adequacy-mode table dumping directly into the simulation loop for inline post-treatment. * Extended CMakeLists and dependencies to include the new implementation. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
The stage is written after the whole CSR treatment -- curtailment sharing, DTG netting and the marginal price update -- not just after the patch, so `adequacy-patch` under-describes it. Stage names are file-name suffixes, so fix it here, where the constant is introduced, rather than after tests have started pinning the old name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`lastSolvedModelerProblem` was only ever assigned, never released, so the problem retained by one pass stayed alive through the whole build and solve of the next -- `ProblemesSpx[NumIntervalle]` is overwritten early in that next call, which makes the retained copy the sole owner from that point. Two full problems therefore coexisted across every pass boundary, including pass 1 to pass 2 inside a single week, and the last one of the run survived to the end of the simulation, since PROBLEME_HEBDO lives as long as Economy does. Drop it at the top of OPT_TryToCallSimplex instead, before the next problem is built. Peak is one problem again. No behaviour changes: a post-process dump reads this after the week's last pass, and a week whose solve fails throws out of OPT_OptimisationHebdomadaireLineaire before the post-process list runs, so the value cleared here was never one that could be published. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ines
Same adjustment as the one at the bottom of the stack, for the declarations
this branch adds: `SolvedModelerProblem`'s two members and the fill-context
parameter of `DumpSimulationTableAfterPostProcess`, plus the forward
declarations above them.
Those forward declarations were the reason this did not fail as loudly as it
should have: declaring `Antares::Optimisation::{OptimEntityContainer,
LinearProblemApi::ILinearProblem}` invents two incomplete types rather than
naming the base's, so the members were pointers to types nothing ever defines.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
- Introduced `--simulation-table-stages` command-line option to specify which stages to include in simulation tables. - Updated documentation to reflect the new parameter and its usage. - Implemented validation for stage names, ensuring only recognized stages are accepted. - Enhanced existing functionality to skip writing tables for unselected stages, optimizing memory and performance. Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…are selected `OPT_TryToCallSimplex` published `lastSolvedModelerProblem` from inside its `if (simulationTable)` block. That was exact before stage selection existed -- the table pointer was null only when the run wrote no simulation tables at all, in which case no post-process dump ran either. With a selection it is no longer exact. `--simulation-table-stages=remix-hydro` makes `tableForStage` return null for both optimisation passes, so nothing was retained, and `DumpSimulationTableAfterPostProcess` then fell through its `solved && modelerData` guard and emitted only the legacy half of a hybrid study -- silently, and contradicting the documented promise that the selection is purely subtractive. Decide the retention from the stage selection instead, one level up where it is visible: `PROBLEME_HEBDO::retainSolvedModelerProblem`, set per week by `OPT_OptimisationLineaire`. It asks for everything the dump needs in order to read the value back -- tables on, a modeler side to re-emit, a weekly optimisation range (the dump no-ops on a daily one) and a post-process stage surviving the selection -- so the retention is now paid for exactly when it buys something. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t apply Two loose ends in the stage selection. `parseStageSelection` returned as soon as it saw `all`, so anything after it was never validated: `all,optim-nb-3` was accepted in silence while `optim-nb-3,all` was rejected. Whether a list widens the selection and whether it is well formed are separate questions -- note the `all` and keep reading, then apply it at the end. Selecting stages in a run that writes no simulation tables did nothing, quietly. The two neighbouring checks already warn about tables being disabled; this adds the third, so the flag is never taken for something that had an effect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Three corrections to the release notes. The migration guide said `remix-hydro` was produced "always". It is not: both post-process stages need a weekly `simplex-range`, which the architecture note records but the guide never mentioned. Say so in the table and give it a short section of its own, since a reader on a daily range would otherwise wait for files that never arrive. `### Unreleased` had no `##` above it, so with `toc_depth: 2` it appeared in no table of contents at all. Every other release sits under a `## Branch <x>.<y>.x`; this one now does too. And `## 10.2.0` in the migration guide is `## v10.2.0`, like every other entry in that file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
added 19 commits
September 9, 2026 11:05
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
… doc/mguide_st_per_process # Conflicts: # docs/architecture/legacy-extra-outputs.md # docs/user-guide/04-migration-guides.md # docs/user-guide/solver/10-command-line.md # src/io/outputs/OptimisationsSimulationTable.cpp # src/io/outputs/include/antares/io/outputs/OptimisationsSimulationTable.h # src/libs/antares/study/include/antares/study/parameters.h # src/libs/antares/writer/LegacySimulationTablesWriter.cpp # src/libs/antares/writer/include/antares/writer/LegacySimulationTablesWriter.h # src/solver/application/application.cpp # src/solver/optimisation/LegacySimulationTableSnapshot.cpp # src/solver/optimisation/LegacySimulationTableStage.cpp # src/solver/optimisation/adequacy_patch_csr/adq_patch_post_process_list.cpp # src/solver/optimisation/include/antares/solver/optimisation/LegacySimulationTableSnapshot.h # src/solver/optimisation/include/antares/solver/optimisation/post_process_commands.h # src/solver/optimisation/opt_appel_solveur_lineaire.cpp # src/solver/optimisation/opt_optimisation_lineaire.cpp # src/solver/optimisation/post_process_commands.cpp # src/solver/simulation/adequacy.cpp # src/solver/simulation/economy.cpp # src/solver/simulation/include/antares/solver/simulation/sim_structure_probleme_economique.h # src/tests/cucumber/features/solver-features/legacy_simulation_table.feature # src/tests/cucumber/features/steps/common_steps/solver_steps.py # src/tests/src/io/outputs/testSimulationTable.cpp
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…process # Conflicts: # src/libs/antares/study/parameters.cpp # src/tests/src/solver/optimisation/test_legacy_extra_outputs.cpp
…fix/inactive_component_some_st # Conflicts: # src/solver/optimisation/optim_post_process_list.cpp
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
…fix/inactive_component_some_st
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
JasonMarechal25
marked this pull request as ready for review
September 15, 2026 07:25
# Conflicts: # docs/architecture/legacy-extra-outputs.md # docs/developer-guide/CHANGELOG.md # docs/user-guide/04-migration-guides.md # docs/user-guide/solver/04-parameters.md # docs/user-guide/solver/10-command-line.md # src/io/outputs/OptimisationsSimulationTable.cpp # src/solver/misc/options.cpp # src/tests/cucumber/features/solver-features/legacy_simulation_table.feature # src/tests/src/io/outputs/testSimulationTable.cpp
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
Signed-off-by: Jason Marechal <jason.marechal@pm.me>
behave's default parse type for {stages} requires at least one
character, so the step never matched
`the study asks for the simulation table stages ""`, leaving the
"empty value stops the run" scenario unmatched and reported by CI as
an opaque error. Register a custom MaybeEmpty parse type that also
accepts zero-length values.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--simulation-table-stages-last