Skip to content

Refactor LdgIterator into wrapper function and ADL access#2343

Open
sethrj wants to merge 14 commits intoceleritas-project:developfrom
sethrj:ldg-iterator
Open

Refactor LdgIterator into wrapper function and ADL access#2343
sethrj wants to merge 14 commits intoceleritas-project:developfrom
sethrj:ldg-iterator

Conversation

@sethrj
Copy link
Copy Markdown
Member

@sethrj sethrj commented Mar 30, 2026

This rethinks the LDG wrapping as part of a core Span/OpaqueId refactor before separating it out in #2338. Before, ldg was called as part of LdgIterator. Now, LdgIterator is simply a proxy iterator: it returns a LdgWrapper<T>, modeled after std::reference_wrapper, that implicitly casts to the type being wrapped, and loads on that access.

The refactored ADL lookup allows a simpler override for loading data via ldg, and a two-argument override even allows you to ldg data members from inside a struct:

BIHNodeId parent = ldg(node, &BIHLeafNode::parent);

This also reinstitutes cont/LdgSpan.hh, which is once again the correct file to include when returning LdgSpan data via collections as part of a View.

NOTE: I moved the friend operator changes to #2347 . That PR is required for this one to compile.

@sethrj sethrj requested a review from esseivaju March 30, 2026 23:19
@sethrj sethrj requested a review from elliottbiondo as a code owner March 30, 2026 23:19
@sethrj sethrj added core Software engineering infrastructure (corecel) minor Refactoring or minor internal changes/fixes labels Mar 30, 2026
@sethrj sethrj removed the request for review from elliottbiondo March 30, 2026 23:19
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 30, 2026

Test summary

 5 975 files   9 692 suites   19m 47s ⏱️
 2 254 tests  2 212 ✅  42 💤 0 ❌
33 626 runs  33 499 ✅ 127 💤 0 ❌

Results for commit 963a36e.

♻️ This comment has been updated with latest results.

@sethrj sethrj requested a review from pcanal March 31, 2026 22:58
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 31, 2026

Codecov Report

❌ Patch coverage is 97.27273% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.25%. Comparing base (9a7f6a5) to head (963a36e).

Files with missing lines Patch % Lines
src/corecel/OpaqueId.hh 50.00% 0 Missing and 1 partial ⚠️
src/corecel/cont/detail/LdgSpanImpl.hh 98.64% 0 Missing and 1 partial ⚠️
src/corecel/math/Quantity.hh 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           develop    #2343    +/-   ##
=========================================
  Coverage    87.24%   87.25%            
=========================================
  Files         1377     1377            
  Lines        43848    43867    +19     
  Branches     13432    13880   +448     
=========================================
+ Hits         38257    38276    +19     
- Misses        4359     4527   +168     
+ Partials      1232     1064   -168     
Files with missing lines Coverage Δ
src/celeritas/grid/GridIdFinder.hh 100.00% <ø> (ø)
...rc/celeritas/optical/gen/ScintillationGenerator.hh 98.79% <100.00%> (+0.01%) ⬆️
src/corecel/cont/Span.hh 100.00% <ø> (ø)
src/corecel/cont/detail/RangeImpl.hh 98.98% <100.00%> (ø)
src/corecel/cont/detail/SpanImpl.hh 100.00% <ø> (ø)
src/corecel/data/Collection.hh 100.00% <ø> (ø)
src/corecel/data/Ldg.hh 100.00% <100.00%> (ø)
src/corecel/data/detail/CollectionImpl.hh 71.42% <ø> (ø)
src/corecel/math/ArrayOperators.hh 100.00% <ø> (ø)
src/corecel/random/distribution/Selector.hh 95.65% <ø> (ø)
... and 10 more

... and 111 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@esseivaju
Copy link
Copy Markdown
Member

It turns out that many of the operators for Quantity and OpaqueId have a subtle but unfortunate oversight that, because they're declared as template functions, they don't participate in overload resolution when implicitly casting: this caused *iter + 2 to fail to compile

Do you have a reference to to rule causing an issue with looking up these operators? I've tried to compile

EXPECT_EQ(*ldg_start + 2, TestId{3});

in LdgIterator.test.cc and it is finding operator+

@sethrj
Copy link
Copy Markdown
Member Author

sethrj commented Apr 1, 2026

@esseivaju The issue arose when adjusting the Span constructor to implicitly construct from types that could implicitly convert from U[] -> T[] (which is what drove all this mess in the first place). The solution I had was to change LdgValue to be not a tag but a wrapper that would implicitly convert to the target type exactly as std::reference_wrapper did, freeing LdgIterator to simply be a proxy iterator.

The problem with that is that if LdgIterator::operator* returns RefWrap<T>, the values returned from its implicit operator T will not look up the operators for T. To demonstrate this problem:

{
    using DozenDbl = Quantity<DozenUnit, double>;
    auto two_dozen = native_value_to<DozenDbl>(24);

    auto td_ref = std::ref(two_dozen);
    EXPECT_EQ(two_dozen * 2, td_ref * 2);
}

will fail on develop with the error:

../test/corecel/math/Quantity.test.cc:129:37: error: invalid operands to binary expression ('reference_wrapper<Quantity<DozenUnit, double>>' and 'int')
  129 |     EXPECT_EQ(two_dozen * 2, td_ref * 2);
      |                              ~~~~~~ ^ ~
...
../src/corecel/math/Quantity.hh:223:31: note: candidate template ignored: could not match 'Quantity' against 'std::reference_wrapper'
  223 | CELER_CONSTEXPR_FUNCTION auto operator*(Quantity<U, T> lhs, T2 rhs) noexcept
      |                               ^
../src/corecel/math/Quantity.hh:229:31: note: candidate template ignored: could not match 'Quantity<U, T2>' against 'int'
  229 | CELER_CONSTEXPR_FUNCTION auto operator*(T rhs, Quantity<U, T2> lhs) noexcept
      |                               ^
...

Maybe i should just move all the operator shenanigans to a separate PR, and then have a separate discussion for LDG iterator.

@sethrj sethrj marked this pull request as draft April 1, 2026 15:42
@sethrj sethrj changed the title Define LdgRefWrapper and improve template matching Refactor LdgIterator as LdgRefWrapper Apr 1, 2026
sethrj added 6 commits April 2, 2026 08:59
Replace the LdgTraits specialization-based customization point with an
ADL-based free function ldg_data(T const*) -> U const*. The built-in
overloads for arithmetic and enum types live in LdgTraits.hh; the
OpaqueId and Quantity overloads now live alongside their respective types
in OpaqueId.hh and Quantity.hh. The is_ldg_supported_v trait is
re-expressed using void_t SFINAE detection over the ADL call.

Update ldg() in LdgRefWrapper.hh to call ldg_data via ADL and derive the
arithmetic check from the returned pointer type rather than a traits
member. Add a two-argument ldg(obj, mp) convenience overload and a
storable LdgMember<Class,T> functor with a CTAD deduction guide for
loading a member-pointer field from a struct via __ldg.

Add test coverage for both the two-argument ldg and LdgMember callable.

Prompt: "Please implement those changes in the code, replacing all use of the old LdgTraits."
Assisted-by: GitHub Copilot
Replace the one-line \page stub with a complete reference page that covers:
- What __ldg does and when to use it (scalar, struct member, span)
- Code examples for each usage pattern including LdgMember and LdgSpan
- How to extend ldg support to a custom type via ldg_data ADL overload
- Built-in overloads for arithmetic, enum, OpaqueId, and Quantity
- Brief descriptions of LdgWrapper and LdgIterator implementation detail

Also cleans up the broken placeholder text in the ldg() function doc comment.

Prompt: "Combine the main documentation of the ldg-related functions into a doxygen `\page ldg Cached device loading`. Provide examples of usage in user code (data, struct, span), and of overloading the ldg_data. Briefly mention and summarize the under-the-hood LdgWrapper and LdgIterator."
Assisted-by: GitHub Copilot
@sethrj sethrj marked this pull request as ready for review April 2, 2026 15:18
@sethrj sethrj changed the title Refactor LdgIterator as LdgRefWrapper Refactor LdgIterator into wrapper function and ADL access Apr 2, 2026
@sethrj sethrj requested a review from esseivaju April 2, 2026 15:23
@sethrj sethrj added enhancement New feature or request and removed minor Refactoring or minor internal changes/fixes labels Apr 2, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors LDG (“cached device loading”) support by replacing the old LdgTraits/LdgIterator approach with an ADL-based ldg_data customization point and a proxy LdgWrapper used by LdgSpan, improving extensibility (including member loads) and simplifying includes as part of the broader Span/OpaqueId refactor.

Changes:

  • Introduces corecel/data/Ldg.hh with ldg + ADL-dispatched ldg_data, plus LdgMember and 2-arg ldg(obj, member_ptr) support.
  • Reintroduces corecel/cont/LdgSpan.hh and implements LDG span behavior via detail::LdgWrapper/detail::LdgIterator.
  • Updates call sites, docs, and tests; removes the legacy LdgTraits.hh and LdgIterator.hh.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/corecel/OpaqueIdUtils.hh Switches tests/utilities to include and use LdgSpan/ldg headers.
test/corecel/data/Ldg.test.cc Adds coverage for 2-argument ldg and LdgMember callable behavior.
test/corecel/cont/Span.test.cc Removes LDG span tests from the generic Span test file (migrated).
test/corecel/cont/LdgSpan.test.cc Updates/expands LDG tests for wrapper + iterator + span behavior.
test/corecel/CMakeLists.txt Registers new/renamed LDG-related tests.
src/orange/univ/detail/LogicEvaluator.hh Updates includes to use LdgSpan.hh rather than the removed iterator header.
src/orange/univ/detail/InfixEvaluator.hh Updates includes to use LdgSpan.hh rather than the removed iterator header.
src/geocel/VolumeView.hh Updates include for LDG access (currently mismatched with actual LdgSpan usage).
src/corecel/OpaqueId.hh Replaces LdgTraits specialization with ldg_data overload for ADL.
src/corecel/math/Quantity.hh Replaces LdgTraits specialization with ldg_data overload for ADL.
src/corecel/data/LdgTraits.hh Deletes the old LDG traits mechanism.
src/corecel/data/LdgIterator.hh Deletes the old LDG iterator/value-marker implementation.
src/corecel/data/Ldg.hh Adds new LDG API + documentation page (\\page ldg).
src/corecel/data/detail/CollectionImpl.hh Uses is_ldg_supported_v to return LdgSpan for device const_reference collections.
src/corecel/data/Collection.hh Minor doc fix; removes unused include.
src/corecel/cont/Span.hh Updates LDG-related wording (but still mentions legacy LdgValue elsewhere).
src/corecel/cont/LdgSpan.hh Reintroduces LdgSpan API, to_array overload, and make_span helper.
src/corecel/cont/detail/SpanImpl.hh Updates span traits to use LdgWrapper rather than the removed LdgValue.
src/corecel/cont/detail/LdgSpanImpl.hh Adds LdgWrapper/LdgIterator implementation and LDG support detection.
src/celeritas/grid/GridIdFinder.hh Simplifies includes to rely on LdgSpan types.
doc/implementation/data-model.rst Adds documentation references for the new LDG doxygen page and LdgSpan.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sethrj
Copy link
Copy Markdown
Member Author

sethrj commented Apr 2, 2026

@amandalund I've got this working the CI and a separate CUDA 12.9 build, but nvcc initially gave some weird compiler warnings that I didn't see on any of the normal compilers. Since you saw something on your own CUDA build that I couldn't reproduce, could you check out this branch to see if it builds?

@amandalund
Copy link
Copy Markdown
Contributor

@sethrj checked and it builds with no issues

Add operator==(LdgWrapper, LdgWrapper) and operator!=(LdgWrapper, LdgWrapper)
overloads to LdgWrapper to resolve ambiguity when std::equal iterates over two
LdgSpan ranges. Previously, comparing two LdgWrapper instances matched both
operator==(LdgWrapper, type) and operator==(type, LdgWrapper) after implicit
conversion, causing a hard compile error via libc++ __equal_to.

Add a LdgWrapperTest.equality test that exercises LdgWrapper == LdgWrapper
comparisons directly and via std::equal over two LdgSpans.

Prompt: "Fix the build errors due to LdgWrapper's ambiguous types. Add new tests for LdgWrapper equality that demonstrates the build failure and its fix."
Assisted-by: GitHub Copilot
@sethrj
Copy link
Copy Markdown
Member Author

sethrj commented Apr 4, 2026

@esseivaju Could you review this update? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Software engineering infrastructure (corecel) enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants