Skip to content

openPMD plugin: Restart from checkpoint with non-matching domain decomposition #5508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: dev
Choose a base branch
from
+415 −100
Open

openPMD plugin: Restart from checkpoint with non-matching domain decomposition #5508

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
369f4ba
Restart with different domain decomposition
franzpoeschel Oct 16, 2025
ececd23
Revert "Remap also on GPU"
franzpoeschel Oct 16, 2025
4ab9853
Revert "wip: redistribute on gpu"
franzpoeschel Oct 16, 2025
05d2127
Factor out loafFull/PartialMatches
franzpoeschel Oct 16, 2025
71a14d2
Continue factoring out common code
franzpoeschel Oct 16, 2025
03b33cd
Do not double-load boundary particles
franzpoeschel Oct 16, 2025
4fdf55b
Cleanup
franzpoeschel Oct 16, 2025
6cdc175
Fix -Werror=sign-compare problem
franzpoeschel Oct 16, 2025
0488d99
Erase unused variables
franzpoeschel Oct 17, 2025
dc6f09b
Replace stdout with logging
franzpoeschel Oct 17, 2025
1d02d93
Auto-detect if PML loads should be skipped
franzpoeschel Oct 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion include/picongpu/plugins/openPMD/NDScalars.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,11 +174,17 @@ namespace picongpu
DataSpace<simDim> gridPos = Environment<simDim>::get().GridController().getPosition();
::openPMD::Offset start;
::openPMD::Extent count;
::openPMD::Extent extent = mrc.getExtent();
start.reserve(ndim);
count.reserve(ndim);
for(int d = 0; d < ndim; ++d)
{
start.push_back(gridPos.revert()[d]);
/*
* When restarting with more parallel processes than the checkpoint had originally been written
* with, we must take care not to index past the dataset boundaries. Just loop around to the start
* in that case. Not the finest way, but it does the job for now..
*/
start.push_back(gridPos.revert()[d] % extent[d]);
Copy link
Contributor Author

@franzpoeschel franzpoeschel Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really the best solution? This loads nextId and startId. If the GPU grid does not match the old one, this logic currently just uses modulus to loop around and hand out some random ID. No idea if that has downsides.

count.push_back(1);
}

Expand Down
6 changes: 6 additions & 0 deletions include/picongpu/plugins/openPMD/openPMDWriter.x.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1024,6 +1024,12 @@ make sure that environment variable OPENPMD_BP_BACKEND is not set to ADIOS1.
{
loadRngStatesImpl(&mThreadParams, restartStep);
}
catch(std::exception const& e)
{
log<picLog::INPUT_OUTPUT>("openPMD: loading RNG states failed, they will be re-initialized "
"instead. Original error:\n\t%1%")
% e.what();
}
catch(...)
{
log<picLog::INPUT_OUTPUT>(
Expand Down
Loading