Skip to content

README.md

Latest commit

 

History

History
193 lines (152 loc) · 8.68 KB

README.md

File metadata and controls

193 lines (152 loc) · 8.68 KB
Raw

PyHermes logo

PyHermes

CI Documentation PyPI Python License

PyHermes is the high-performance Python implementation of Hermes (HypER-speed MultirEsolution cosmic Statistics), an in situ framework that turns particle catalogues into reusable multiresolution fields for cosmic statistics and physical-field analysis.

A catalogue is projected once into a scaling-function-coefficient field (SFCField). Smoothing, geometric binning, multipole decomposition, differentiation, and inverse-Laplacian operations are then expressed through WindowFunc objects. The same field can therefore feed Counting, isotropic and anisotropic 2PCFs, standard and multipole 3PCFs, marked statistics, and derived physical fields without returning to particle-level tuple counting for every configuration.

Hermes field-window workflow

What It Covers

  • catalogue-to-field projection with configurable compactly supported scaling functions and resolution J;
  • built-in and user-defined smoothing, binning, multipole, and operator windows;
  • one-point counting and field sampling;
  • isotropic and anisotropic 2PCF measurements;
  • Monte Carlo and spherical-harmonic 3PCF estimators;
  • MPI/thread parallelism and CPU or CUDA contraction for 3PCF multipoles;
  • weighted fields, velocity derivatives, Poisson potential, acceleration, and density-dependent marks.

Installation

For notebooks, development, and other single-process work, install PyHermes from PyPI:

python -m pip install pyhermes-cosmo

This installs PyHermes and its regular Python dependencies automatically. MPI and CUDA are optional, so neither is required for the default installation.

The Python import name remains unchanged:

import pyhermes

For a ready-to-use MPICH environment on Linux or macOS:

conda create -n pyhermes -c conda-forge python=3.12 mpi4py mpich pip
conda activate pyhermes
python -m pip install pyhermes-cosmo
mpiexec -n 2 python -c "from mpi4py import MPI; print(MPI.COMM_WORLD.rank)"

Here conda provides a mutually compatible MPI runtime and Python binding, while pip installs PyHermes and the remaining Python dependencies. PyHermes does not yet require a separate conda package for this workflow.

Without mpi4py, PyHermes automatically uses its single-process MPI fallback. Users of an existing cluster MPI should follow the installation guide for the matching mpi4py and GPU setup.

Smallest Workflow

The tracked Quick Start configurations use paths relative to examples/:

cd examples
python scripts/run_sfc_projection.py configs/param_sfc_projection.yaml
python scripts/run_2pcf.py configs/param_2pcf.yaml

The first command downloads and caches the original single-file Quijote group_tab catalogue from the URL in the YAML, verifies its SHA256 digest, and writes the base SFCField. The second command consumes that exact field and writes an isotropic Corr2PCFData result. The matching quick_start.ipynb executes the same configs and plotting code rather than maintaining a parallel example.

The dedicated particle_io.ipynb reuses that catalogue and shows how the native FoF reader, an NPZ conversion, and a raw BIN layout all feed the same projection API. The converted NPZ and BIN files are local teaching products, not additional public downloads. sfc_projection.ipynb then builds the J=9, weighted, redshift-space, and explicit sampled-random fields used by later examples; scripts/prepare_sfc_fields.py provides the equivalent non-interactive batch entry point. The optional dark-matter snapshot builder accepts the local Gadget HDF5 snapshot prefix explicitly; no cluster-specific path is embedded in the code:

python scripts/build_quijote_dm_sfc_field.py /path/to/snapdir_004/snap_004
from pyhermes.base.sfc_projection import SFCProjection
from pyhermes.io import WindowFunc
from pyhermes.param.parambase import read_param

params = read_param("./configs/param_sfc_projection.yaml")
field = SFCProjection(params).run()

gaussian = WindowFunc(
    {"type": "gaussian", "len_args": {"R": 10.0}},
    field.sfc_info,
    threads=8,
)
smoothed_field = field @ gaussian

This is the core language of PyHermes: field @ window. Statistical tasks build the required window families and normalizations around the same objects.

Start With The Notebooks

The recommended route through examples/notebooks/ is:

  1. quick_start.ipynb
  2. particle_io.ipynb
  3. sfc_projection.ipynb
  4. window.ipynb
  5. physical_fields.ipynb
  6. counting.ipynb
  7. corr2pcf.ipynb
  8. corr3pcf.ipynb

The first four notebooks establish the common catalogue-to-field-to-window workflow. Continue with physical fields, one-point Counting, or the 2PCF-to-3PCF statistics path according to the calculation you need.

Generated catalogues and estimator products are intentionally not committed. The notebooks state which lightweight cells run locally and which script/YAML pairs are intended for a workstation or cluster.

Documentation

The full guide at pyhermes.astroslacker.com follows the terminology and estimator definitions of the Hermes paper. It covers the mathematical construction, current APIs, window catalogue, parameter mappings, numerical validation, and performance interpretation.

To build the documentation locally:

python -m pip install ".[docs]"
sphinx-build -W -b html docs docs/_build/html

Citing PyHermes

If PyHermes contributes to your work, please cite the software version and the four papers that describe its scientific foundation and implementation:

  1. Hermes/PyHermes: Long-long Feng, Tengpeng Xu, Tian-Cheng Luan et al. (2026), Hermes - Towards an Optimal High-Performance Algorithm for Cosmic Statistics of Large Data Sets, arXiv:2607.23494 [astro-ph.CO].
  2. MRACS foundation: Feng (2007), The Beylkin-Cramer Summation Rule and a New Fast Algorithm of Cosmic Statistics for Large Data Sets.
  3. In-situ correlation functions: Yue et al. (2024), Pair counting without binning -- a new approach to correlation functions in clustering statistics.
  4. Isotropic 3PCF multipoles: Ju et al. (2026), An optimal in situ multipole algorithm for the isotropic three-point correlation function.

Machine-readable metadata is provided in CITATION.cff; complete BibTeX entries are available in the citation guide.