Skip to content

Fix: Multiple file summing incorrectly concatenates instead of merging cross-sections#253

Merged
darshdinger merged 8 commits intonextfrom
ewm14351_adding_spectra_not_working
Feb 12, 2026
Merged

Fix: Multiple file summing incorrectly concatenates instead of merging cross-sections#253
darshdinger merged 8 commits intonextfrom
ewm14351_adding_spectra_not_working

Conversation

@darshdinger
Copy link
Contributor

PR Description

Description of work:

Fixed critical bug in File -> Open & Sum Multiple Files where multiple data files were being concatenated (producing 8 cross-sections from 2 files) instead of properly merged (producing 4 cross-sections). This also fixes the "Unknown property search object 'run_numbers'" error that was crashing the application when loading summed files.

Root Causes Fixed:

  1. Dead code in _get_xs_list() that never executed due to local variable shadowing
  2. List concatenation (xs_list +=) instead of workspace merging in load_data()
  3. All files loaded with same workspace name, causing Mantid to overwrite data
  4. Missing run_numbers log property on workspaces

Solution Implemented:

  • Removed non-functional merging logic from _get_xs_list() (lines 243-253)
  • Refactored load_data() to use unique workspace names for each file
  • Implemented proper workspace merging by matching cross_section_id using api.Plus()
  • Added run_numbers log with correct format (e.g., "42112+42113")
  • Added 3 comprehensive unit tests covering file summing, deadtime correction, and cross-section matching
  • Included manual test script for validation

Check all that apply:

  • Source added/refactored
  • Added unit tests
  • updated documentation and checked that it looks correct in the pull request preview - No doc changes needed; internal fix only
  • Added integration tests - Unit tests cover the functionality; existing integration tests validate no regressions

References:

  • Links to IBM EWM items: EWM # 14351
  • Links to related issues or pull requests:

⚠️ Manual test for the reviewer

(instructions to set up the environment)

After setting up the environment:

  1. Run automated tests:

    pixi run pytest test/unit/quicknxs/interfaces/data_handling/test_instrument.py::test_load_data_sum_two_files -v

    This should pass, verifying that summing produces 4 cross-sections (not 8) and the run_numbers property exists.

  2. Run manual test script:

    pixi run python test_file_summing.py

    This validates the fix with real data files (REF_M_42112 and REF_M_42113).

  3. Verify no regressions:

    pixi run pytest test/unit/quicknxs/interfaces/data_handling/test_instrument.py -v

    All 10 tests should pass.


Check list for the reviewer

  • best software practices
    • clearly named variables (better to be verbose in variable names)
    • code comments explaining the intent of code blocks
  • All the tests are passing
  • The documentation is up to date and looks correct in the pull request preview
  • code comments added when explaining intent

@codecov-commenter
Copy link

codecov-commenter commented Feb 9, 2026

Codecov Report

❌ Patch coverage is 97.72727% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 65.64%. Comparing base (de26510) to head (c5b024b).

Files with missing lines Patch % Lines
...c/quicknxs/interfaces/data_handling/quicknxs_io.py 96.55% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             next     #253      +/-   ##
==========================================
+ Coverage   65.43%   65.64%   +0.20%     
==========================================
  Files          37       37              
  Lines        6982     7010      +28     
==========================================
+ Hits         4569     4602      +33     
+ Misses       2413     2408       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@backmari backmari self-requested a review February 9, 2026 15:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes incorrect behavior when summing multiple REF_M data files by ensuring workspaces are uniquely named per input file, cross-sections are merged (rather than concatenated), and summed workspaces include a run_numbers log so loading reduced/summed runs no longer crashes.

Changes:

  • Refactors Instrument.load_data() to load each file into distinct workspaces, then merge results and add a run_numbers sample log.
  • Updates reduced-file parsing to handle run numbers like 42112+42113 and to reconstruct summed file paths when reading saved reductions.
  • Adds/extends unit tests covering summed-run parsing and multi-file summing behavior; bumps a few dependency minimums in pixi/pyproject.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/quicknxs/interfaces/data_handling/instrument.py Refactors multi-file loading to avoid workspace overwrites and sum results; adds run_numbers log.
src/quicknxs/interfaces/data_handling/quicknxs_io.py Improves reduced-file parsing for summed run-number formats and updates file-summing path reconstruction.
src/quicknxs/interfaces/data_manager.py Adjusts additional-peaks initialization to match runs by file path (including summed runs).
test/unit/quicknxs/interfaces/data_handling/test_instrument.py Adds datarepo tests validating multi-file summing and run_numbers presence.
test/unit/quicknxs/interfaces/data_handling/test_quicknxs_io.py Adds tests for summed run parsing and determine_which_files_to_sum() behaviors.
pyproject.toml Updates dependency minimum versions (security-related notes included).
pixi.lock Updates the resolved environment/locks to match new dependency minimums.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Collaborator

@backmari backmari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested in the GUI that the load and sum files functionality now works as expected. Saving and loading of summed files also works as expected. The code changes look good to me.

@darshdinger darshdinger merged commit 1fdb1ac into next Feb 12, 2026
13 checks passed
@darshdinger darshdinger deleted the ewm14351_adding_spectra_not_working branch February 12, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants