MPIAdvance Stream-triggering support#230
Draft
EvanDrakeSuggs wants to merge 15 commits into
Draft
Conversation
* chore: update version before 0.1.0 release * build(nccl): require NCCL 2.20+ and ensure CUDA is enabled * build(mpi): also require MPI-3+ standard revision This requirement was already made clear in the docs, but not enforced in the CMake. Signed-off-by: Gabriel Dos Santos <gabriel.dossantos@cea.fr>
Contributor
Author
|
Figured out that my code is not the issue as it has the same problems with earlier commits. However, I did redo everything and it works with 13/18 tests. Thinking there might be conflicts between what stream-triggering and Kokkos require for HIP on Tuolumne. Non-working tests are: test.core.p2p, test.core.alltoall, test.mpi.view-access, test.mpi.sendrecv, test.mpi.isendrecv. |
…e stream-triggering.
…i sendrecv test, but links stream-triggering.h
…ring specific backend functions.
…st 1000+, but not fully tested) hang. Working on Derek to fix. Plan is to split stream triggering stages to three on context object: init context with stream, queue, etc; send/recv set up their own requests and tasks on queue; finally, enqueue on context object.
…m info from fine to coarse (bug), but still did not change larger size bug.
…es Standard mode, not ready mode, but that is available in MPI Advance Stream-Triggering. Syncronous mode is currently not, nor are there plans to add. Also, cleaned up code base.
… to packing error with packing view lifetime or that packing needs to only happen after MPIS_Recv has completed and its request is over.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add backend support for the MPIAdvance stream-triggering library. Currently, there is only an include statement no major code changes, but unit testsand perf tests do not run correctly. Everything compiles on our test system, LLNL's Tuolumne, but some of the tests timeout or don't work correctly.
We got the library working with Cabana in a similar way and I have included that projects build script, which should mean you only have to supply proper paths to kokkos-comm (cmake -DCMAKE_PREFIX_PATH=/apps/stream-triggering/ -DKokkos_DIR=/apps/kokkos/lib64/cmake/Kokkos/ -DCMAKE_CXX_COMPILER=CC -DCMAKE_INSTALL_PREFIX=/apps/kokkos-comm/ ..). Hopefully, this is just a small bug in the CMake, but it is good to get it out there.
Related issue(s):
KokkosCommBuildTuolumne.sh
Depends on:
Uses Derek Schafer's stream-triggering library (https://github.com/mpi-advance/stream-triggering) at branch feat-common-process-engine on Tuolumne
Additional context
Changes
Checklist