ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (cineca chromosome 22)'

[JafetArauz16]

I’m getting an error when running the test process. I’m working on a cluster and I have already cloned the repository. Both Nextflow and Singularity are installed, but when I run the test command I get the following error:

ERROR ~ Error executing process > 'PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (cineca chromosome 22)'

Caused by:
Process PGSCATALOG_PGSCCALC:PGSCCALC:MATCH:MATCH_VARIANTS (cineca chromosome 22) terminated with an error exit status (1)

Command executed:

export POLARS_MAX_THREADS=2

pgscatalog-match --dataset cineca --scorefile normalised_PGS001229_22.txt normalised_PGS001229_22_duplicate.txt --target GRCh37_cineca_22.pvar.zst --only_match --chrom 22 --outdir $PWD -v

cat <<-END_VERSIONS > versions.yml
MATCH_VARIANTS:
pgscatalog.match: $(echo $(python -c 'import pgscatalog.match; print(pgscatalog.match.version)'))
END_VERSIONS

Command exit status:
1

Command output:
(empty)

Command error:
pgscatalog.match.cli.match_cli: 2025-07-18 01:52:14 WARNING No output format specified, writing to combined scoring file
pgscatalog.match.cli.match_cli: 2025-07-18 01:52:14 DEBUG Verbose logging enabled
pgscatalog.match.cli.match_cli: 2025-07-18 01:52:14 INFO --cleanup set (default), temporary files will be deleted
pgscatalog.match.lib.scoringfileframe: 2025-07-18 01:52:14 DEBUG Converting ScoringFileFrame([NormalisedScoringFile('normalised_PGS001229_22.txt'), NormalisedScoringFile('normalised_PGS001229_22_duplicate.txt')]) to feather format
pgscatalog.match.lib.scoringfileframe: 2025-07-18 01:52:14 DEBUG ScoringFileFrame([NormalisedScoringFile('normalised_PGS001229_22.txt'), NormalisedScoringFile('normalised_PGS001229_22_duplicate.txt')]) feather conversion complete
pgscatalog.match.lib._match.preprocess: 2025-07-18 01:52:14 DEBUG Complementing column effect_allele
pgscatalog.match.lib._match.preprocess: 2025-07-18 01:52:14 DEBUG Complementing column other_allele
pgscatalog.match.lib.scoringfileframe: 2025-07-18 01:52:14 DEBUG Filtering scoring file to chromosome 22
pgscatalog.match.lib.variantframe: 2025-07-18 01:52:14 DEBUG Converting VariantFrame(path='GRCh37_cineca_22.pvar.zst', dataset='cineca', chrom='22', cleanup=True, tmpdir=PosixPath('tmp')) to feather format
pgscatalog.match.lib.variantframe: 2025-07-18 01:52:14 DEBUG VariantFrame(path='GRCh37_cineca_22.pvar.zst', dataset='cineca', chrom='22', cleanup=True, tmpdir=PosixPath('tmp')) feather conversion complete
pgscatalog.match.lib._match.preprocess: 2025-07-18 01:52:14 DEBUG Filtering target to include chromosomes 1 - 22, X, Y
pgscatalog.match.lib._match.preprocess: 2025-07-18 01:52:14 DEBUG No multiallelic variants detected
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Getting matches for scores with effect allele and other allele
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: refalt
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: altref
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: refalt_flip
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: altref_flip
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Getting matches for scores with effect allele only
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: no_oa_ref
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: no_oa_alt
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: no_oa_ref_flip
pgscatalog.match.lib._match.match: 2025-07-18 01:52:14 DEBUG Matching strategy: no_oa_alt_flip
pgscatalog.match.cli.match_cli: 2025-07-18 01:52:14 INFO Renaming matchtmp/tmpsh0eejoh to 0.ipc.zst
Traceback (most recent call last):
File "/app/.venv/bin/pgscatalog-match", line 10, in
sys.exit(run_match())
^^^^^^^^^^^
File "/app/packages/pgscatalog.match/src/pgscatalog/match/cli/match_cli.py", line 107, in run_match
ipc_path.rename(out_path)
File "/usr/local/lib/python3.12/pathlib.py", line 1363, in rename
os.rename(self, target)
OSError: [Errno 16] Device or resource busy: 'matchtmp/tmpsh0eejoh' -> '0.ipc.zst'

[nebfield]

Some users have reported this problem before but it's not something I can reproduce (it works fine on the HPC clusters i have access to).

I'd suggest trying the conda profile as a workaround or talking with the HPC admins to diagnose the problem with your storage configuration. The singularity containers are basically trying to write temporary files to a mounted directory. It's not something you can fix by yourself: it's to do with the way singularity is set up on your cluster.

[JafetArauz16]

The solution to my problem was found in a Nextflow command for HPC execution environments. The suggested line of code that worked for me was the following:

nextflow run pgscatalog/pgsc_calc -profile test,singularity -process.scratch=true

With this, the pipeline can be executed correctly. The documentation for the solution is found in point 7 of this guide, 'Use the scratch directive,' where the use of local disk is recommended for this type of issue: Best practices for deploying pipelines with HPC workload managers