[ENH] Dynamically pull BIDS schema for auto entities and extractions. - #380
Conversation
|
I was afraid fetching v1.9.0 on-the-fly over the web in the GHA would cause issue 🫠. might have to using entities from bundled schema instead. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #380 +/- ##
==========================================
- Coverage 76.61% 75.85% -0.76%
==========================================
Files 15 17 +2
Lines 1180 1557 +377
Branches 202 264 +62
==========================================
+ Hits 904 1181 +277
- Misses 223 299 +76
- Partials 53 77 +24
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
sys.args was more fragile, now if a user explicitly asks for a version, it will be used even if it is the same as the default "bundled" one
arnaudbore
left a comment
There was a problem hiding this comment.
Great work, a couple of comments but looks great.
|
Thanks for the review @arnaudbore. I also pushed the documented function in addition to a modification to the beginner's tutorial |
arnaudbore
left a comment
There was a problem hiding this comment.
Small comment, please do a quick pep8 and we will see what copilot has to say.
There was a problem hiding this comment.
Pull request overview
This PR introduces BIDS-schema–driven defaults into dcm2bids by adding a schema loader/cache layer and updating defaults/docs/tests so entity ordering and “auto-entities” can be derived from the official BIDS schema (with support for aliases like stable/latest and a bundled fallback).
Changes:
- Added
dcm2bids.utils.schemato download/cache/load BIDS schema JSON (including bundled default schema support). - Replaced hard-coded
DEFAULT.entityTableKeys/DEFAULT.auto_entitieswith schema-derived values, and added a new--bids_versionCLI flag. - Added extensive tests for schema loading/caching and updated docs/config examples to reflect the new schema-driven behavior.
Reviewed changes
Copilot reviewed 16 out of 20 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_schema.py | Adds unit tests for schema loading, caching behavior, and schema-derived defaults. |
| tests/data/config_test_auto_extract.json | Updates test config to include custom_entities: "dir" for fmap/epi. |
| pyproject.toml | Excludes bundled schema JSON from codespell scanning. |
| docs/tutorial/first-steps.md | Updates tutorial text and examples for --auto_extract_entities and custom_entities. |
| docs/how-to/use-advanced-commands.md | Documents new --bids_version flag and caching/reproducibility guidance. |
| docs/how-to/create-config-file.md | Fixes a minor doc typo/formatting issue. |
| dcm2bids/utils/utils.py | Switches DEFAULT entity ordering/auto-entities to schema-derived defaults. |
| dcm2bids/utils/tools.py | Adds cached version-check mechanism and version comparison helpers. |
| dcm2bids/utils/schema.py | New module implementing schema URL building, download, caching, derived defaults, and CLI schema resolution. |
| dcm2bids/utils/schema_data/init.py | Adds packaged data access helper for bundled schema JSON. |
| dcm2bids/utils/logger.py | Minor formatting/whitespace fix. |
| dcm2bids/sidecar.py | Minor whitespace/formatting fix in config validation area. |
| dcm2bids/participant.py | Fixes grammar in validation error messages (“contain” vs “contains”). |
| dcm2bids/dcm2bids_gen.py | Minor formatting/whitespace changes (constructor args formatting). |
| dcm2bids/cli/dcm2bids.py | Adds --bids_version argument and schema pre-load call; updates --version output. |
| dcm2bids/cli/dcm2bids_scaffold.py | Routes version check through the new cached check_latest(log_dir=...). |
| dcm2bids/cli/dcm2bids_helper.py | Routes version checks through the new cached check_latest(log_dir=...). |
| dcm2bids/acquisition.py | Updates docstring to reference schema-derived entity ordering; minor whitespace. |
| .github/workflows/tests.yml | Minor whitespace change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This PR makes dcm2bids even more linked to the official BIDS specification by deriving
entity_table_keysandauto_entitiesfrom the schema instead of hard‑coding them. This means dcm2bids can work with any of the BIDS version available so far. I manually tested 1.9.0 up to 1.11.1.It adds configurable BIDS schema version handling (including aliases and a bundled fallback), with caching and better offline behavior reusing version_check.json. Tests cover schema loading (online, cached, and bundled), as well as the derived defaults. Full disclosure: I wasn’t sure how to robustly simulate offline behavior in tests, so those tests were auto‑generated, but they match the scenarios I tested manually.
I also improved how update version check is done through a new caching mechanism in
tmp_dcm2bids/version_check.jsonas well.