Support plain OpenModelica Co-Sim FMUs (relax BOPTEST-only assumptions) - #580
Open
nllong wants to merge 20 commits into
Open
Support plain OpenModelica Co-Sim FMUs (relax BOPTEST-only assumptions)#580nllong wants to merge 20 commits into
nllong wants to merge 20 commits into
Conversation
…load The modelica job path was written around BOPTEST-style wrapped FMUs and made two assumptions that plain FMUs exported straight from OpenModelica do not satisfy, so a run crashed at start: - Data_Manager.load_data_and_kpisjson() read 'resources/kpis.json' from inside the FMU zip and raised KeyError when it was absent. Non-BOPTEST FMUs do not include one. It now tolerates a missing kpis.json (proceeds with an empty KPI definition and warns) and records whether one was present. - TestCase._get_var_metadata() called get_variable_unit/description/min/max, which raise FMUException for variables lacking those attributes (e.g. a dimensionless control input). These are now wrapped in try/except -> None, and _check_value_min_max() skips the range clamp when a bound is undefined. Together these let plain OpenModelica FMI 2.0 Co-Simulation FMUs (no kpis.json, unitless control I/O) load and run without re-wrapping. Regression-tested against the BOPTEST wrapped.fmu, which is unaffected. Also documents FMU compatibility/wrapping requirements. Rebased onto develop (dropping the PACER rename base); paths restored to alfalfa_worker and doc updated to Alfalfa naming accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A missing kpis.json is now tolerated (KPIs disabled) rather than fatal, but a
web user had no way to learn why KPIs were absent or how to enable them -- the
warning only reached worker logs. The web UI only surfaces messages via
error_log on ERROR status, so add a non-fatal notice channel:
- Run.notices (ListField) stores user-facing notices.
- Job.add_run_notice() appends a de-duplicated notice and saves it.
- The modelica StepRun emits an actionable notice when kpis.json is absent,
explaining why KPIs are disabled and how to enable them (add a
resources/kpis.json, minimally {}, inside the FMU and re-upload).
- The web API exposes notices, and the Sites table shows an info icon in the
Status cell (for any status) that opens a NoticeDialog listing the messages.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Covers the tolerance added when relaxing BOPTEST-only assumptions: when an FMU variable declares no min/max (as with plain OpenModelica control inputs), _check_value_min_max must return the value unchanged instead of comparing against None. Also asserts normal clamping is preserved when bounds are present. Pure unit test -- bypasses TestCase.__init__ so it needs no FMU runtime or external services. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The /api/v2/models/upload endpoint returns responses wrapped in a
{ payload } envelope (the convention used everywhere else in the web
UI), and createUploadPost returns modelId (lowercase). Upload.js read
fields/url/modelID off the top-level response instead, so uploadData.fields
was undefined and Object.entries(fields) threw:
TypeError: Object.entries requires that input parameter not be null or undefined
This broke web-UI model uploads for every file. Unwrap the payload before
passing it to uploadFile()/createRun(), and match the server's modelId key.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Self-contained doc describing the FMU output contract Alfalfa expects (FMI 2.0 Co-Simulation, linux64 binaries, input/output causality, optional resources/kpis.json, min/max clamping) plus autogeneration guidance and a per-FMU compatibility checklist. Intended to be copied into the FMU Compilation Workbench repo to drive generation of Alfalfa-compatible FMUs. Renamed from PACER-FMU-Compatibility.md and reworded (PACER -> Alfalfa) as part of rebasing this PR onto develop and reverting the PACER rename. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
useEffect(async () => ...) returns a Promise, which React rejects with "useEffect must not return anything besides a function." Move the fetch into an inner async function called immediately inside the effect. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nllong
force-pushed
the
pacer-custom-fmu-upload-test
branch
from
July 23, 2026 00:29
1b442bf to
439888d
Compare
- tests/worker/test_testcase_metadata.py imported alfalfa_worker.lib.testcase at module scope, which requires pyfmi (and transitively matplotlib via data_manager.py). Neither is a poetry dependency -- they only exist in the worker Docker image -- so this broke test collection in the unit-tests, integration-tests, and integration-tests-historian CI jobs with ModuleNotFoundError. Guard the import with pytest.importorskip so it's skipped in those bare-Python CI jobs instead of erroring, while still running for real wherever the full worker dependency set is installed. - Run prettier on the two new markdown docs (Alfalfa-FMU-Compatibility.md, Modelica FMU Requirements.md) to fix the pre-commit failure; pre-commit had not been run before this branch was pushed. - Add AGENTS.md documenting the required pre-push validation flow (pre-commit, unit tests, then fast integration/simulation checks before the full suite) so AI agents don't claim code is push-ready without actually running these checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The advance() step in the Modelica StepRun job was assigning the entire simulation output dict to each output point's value instead of the point's own scalar value (point.value = y_output instead of point.value = value). This corrupted every output reading (serialized as NaN -> null), so uploaded FMUs like Boiler_EC.fmu never showed any simulation results even though the run itself completed successfully. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Point model gains optional minimum/maximum fields, populated from the FMU's variable metadata (unit/min/max) when available, so writable input ranges can be surfaced and enforced. - formatPoint now returns min/max; validatePointWrite (used by both the single-point and bulk point write routes) rejects values outside the declared range. - Run model gains an external_clock flag, persisted when a run is started, so the API/UI can tell whether a run needs to be stepped manually (external clock) vs. advances on its own. - Documented the new fields in api.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds Boiler_EC.fmu, BuriedPipePair_FMU.fmu, and Chiller_Carnot_y_FMU.fmu -- plain Modelica component models (not BOPTEST-style wrapped test cases) generated using the compilation workflow in https://github.com/NatLabRockies/FMU-compilation-workbench/tree/develop. test_component_fmus.py uploads each FMU, runs it for a couple of simulated minutes on Alfalfa's internal clock and confirms real output values are produced, checks that declared input ranges are surfaced via the API, and separately verifies that values can be written into input points and advanced via an external clock. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Upload page:
- Show a spinner and status text while uploading/processing, with a
clearer two-step Select Model -> Upload Model flow.
- Add an intro description of what the page does.
- Replace the silent success/failure with an error toast and an
'Upload Complete' dialog offering to upload another model or go to
the Sites page (uploading only adds a run; it does not start it).
- Support selecting an unzipped model folder directly ('Select Folder')
which is zipped client-side (jszip) to match how alfalfa-client zips
directories before upload.
- Move the 'Select Model' button to the right of the filename display.
Points dialog (Sites page):
- Add write controls for INPUT/BIDIRECTIONAL points, enforcing any
declared min/max range from the API.
- Add a 'Step Simulation' control for runs started with an external
clock, showing sim time and refreshing points after each step.
Sites page:
- Add a sortable 'Uploaded' column and make all data columns sortable.
- Add a header 'select all' checkbox.
- Rename 'Remove Test Case' to 'Remove Test Case(s)'.
Start dialog:
- Default start time to the current hour (minutes/seconds truncated)
and default the simulation window to 8 hours.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR relaxes BOPTEST-specific assumptions in the Modelica/FMUs execution path so “plain” OpenModelica-exported FMI 2.0 Co-Simulation FMUs (without resources/kpis.json and with partial/absent variable metadata) can upload, start, and run in Alfalfa, while preserving BOPTEST compatibility. It also adds run notices and point min/max metadata to the API/UI so missing KPIs and writable ranges are visible and enforceable.
Changes:
- Worker: tolerate missing
resources/kpis.json, tolerate missing unit/description/min/max metadata, and skip min/max clamping when bounds are undefined. - Data model + API/UI: persist run
noticesandexternal_clock, expose pointmin/max, validate writes against ranges, and show notices + external-clock stepping in the UI. - Tests/docs: add regression/unit/integration coverage for the new behaviors and document FMU compatibility requirements.
Reviewed changes
Copilot reviewed 21 out of 25 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/worker/test_testcase_metadata.py | Unit tests ensuring _check_value_min_max skips clamping when bounds are None. |
| tests/integration/test_component_fmus.py | Integration coverage for running “plain” component FMUs and verifying I/O + optional ranges. |
| docs/Modelica FMU Requirements.md | New doc describing FMU expectations and the relaxed behavior for missing KPIs/metadata. |
| docs/Alfalfa-FMU-Compatibility.md | Detailed compatibility guide/checklist and guidance for workbench-generated FMUs. |
| alfalfa_worker/lib/testcase.py | Make variable metadata lookups tolerant to missing unit/description/min/max; skip clamp when bounds missing. |
| alfalfa_worker/lib/models.py | Add Point.minimum/maximum, Run.external_clock, and Run.notices fields. |
| alfalfa_worker/lib/job.py | Add Job.add_run_notice() helper to persist non-fatal run notices (deduped). |
| alfalfa_worker/lib/data/data_manager.py | Treat missing resources/kpis.json as non-fatal, set kpi_json_missing flag. |
| alfalfa_worker/jobs/modelica/step_run.py | Emit run notice when KPIs are missing; attach point units/min/max from FMU metadata; fix output point value assignment. |
| alfalfa_web/server/api.js | Expose run notices + externalClock, expose point min/max, and validate writes against range. |
| alfalfa_web/server/api-v2.js | Reuse shared validatePointWrite for single-point writes. |
| alfalfa_web/package.json | Add jszip dependency (used for client-side folder zipping). |
| alfalfa_web/package-lock.json | Lockfile updates for jszip (and related dependency tree). |
| alfalfa_web/components/Upload/Upload.scss | Styling for the new upload UI layout/status row. |
| alfalfa_web/components/Upload/Upload.js | Upload UX improvements: status states, completion dialog, error snackbar, disable inputs while busy. |
| alfalfa_web/components/Upload/FileInput.js | Add “Select Folder” and zip folder contents client-side into a .zip upload via JSZip. |
| alfalfa_web/components/Sites/StartDialog.js | Improve defaults for start/end times (rounded start; default 8-hour duration). |
| alfalfa_web/components/Sites/Sites.js | Add sorting, select-all, upload timestamp display, and notice indicator + dialog trigger. |
| alfalfa_web/components/Sites/PointDialog.js | Add writable point control UI, min/max display, and external-clock stepping controls. |
| alfalfa_web/components/Sites/NoticeDialog.js | New dialog to display run notices in the UI. |
| alfalfa_web/api.yml | Document externalClock on run metadata (but currently missing notices). |
| AGENTS.md | Adds/records repository agent instructions (CI/testing expectations). |
Files not reviewed (1)
- alfalfa_web/package-lock.json: Generated file
Comments suppressed due to low confidence (1)
alfalfa_worker/jobs/modelica/step_run.py:71
name_to_idcurrently callsname.replace(" ", "_")but ignores the returned string, so spaces are not actually sanitized. This can leak spaces intoref_id/API point IDs and break lookups if an FMU variable name contains spaces.
inputs_metadata = getattr(self.tc, 'inputs_metadata', {})
def name_to_id(name: str) -> str:
name.replace(" ", "_")
return name
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Merge unique content from "Modelica FMU Requirements.md" (implementation file references, the BoilerPolynomial_FMU worked example, and the historical note on relaxed BOPTEST-only assumptions) into the newer, more complete Alfalfa-FMU-Compatibility.md, then remove the redundant file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Custom FMUs exported straight from OpenModelica (FMI 2.0 Co-Simulation, plain physical I/O, no
resources/kpis.json) failed to upload/run in Alfalfa because the modelica job path assumed two BOPTEST-only conventions. This PR relaxes those assumptions so such FMUs load and run as-is — while remaining fully compatible with BOPTEST-style wrapped FMUs.Root cause
Data_Managerraised on a missingresources/kpis.json(a BOPTEST artifact).TestCaseassumed every variable has a unit/description and finite min/max bounds, then clamped inputs against those bounds — plain control I/O is often unitless/unbounded, so metadata lookups and the min/max clamp failed.Changes
alfalfa_worker/lib/data/data_manager.py— missingresources/kpis.jsonnow degrades to{}(with a clearer warning) and sets akpi_json_missingflag instead of erroring.alfalfa_worker/lib/testcase.py— wrap unit/description/min/max metadata lookups in try/except (→None);_check_value_min_maxskips the clamp when a bound isNone.kpis.json, surface a non-fatal, web-visible notice explaining why and how to fix it (instead of a raw traceback / silent behavior):models.py(Run.notices),job.py(Job.add_run_notice),jobs/modelica/step_run.py(emit notice),server/api.js(expose notices), newcomponents/Sites/NoticeDialog.js+Sites.jsinfo icon.tests/worker/test_testcase_metadata.py— regression test for the None-bounds range-check behavior.docs/Modelica FMU Requirements.md/docs/Alfalfa-FMU-Compatibility.md— FMU compatibility / wrapping guidance.Testing
Rebased onto
developand re-verified end-to-end (Docker Compose: worker + web + mongo/redis/minio):tests/worker/test_testcase_metadata.py: 3 passed — the None-bounds range-check regression test.tests/jobs/test_workflow.py -k model_path1(wrapped.fmu,-m docker): 2 passed.tests/integration/test_modelica.py(-m integration): 1 passed — full HTTP path (AlfalfaClient → web → worker → mongo/redis/minio) submitting the known-goodwrapped.fmu, asserting all BOPTEST_u/_yinputs & outputs, advancing ×5, stopping. Confirms these changes do not regress the BOPTEST path.Notes
develop; the PACER rename is no longer part of this PR (base package names are back toalfalfa_worker/alfalfa_web, and docs/branding reference Alfalfa).Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com