Skip to content

Fix setting of component directories when using buildininstalldir#4065

Open
Flamefire wants to merge 2 commits intoeasybuilders:developfrom
Flamefire:20260216162918_new_pr_bundle
Open

Fix setting of component directories when using buildininstalldir#4065
Flamefire wants to merge 2 commits intoeasybuilders:developfrom
Flamefire:20260216162918_new_pr_bundle

Conversation

@Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Feb 16, 2026

(created using eb --new-pr)

Fixes #4037 see easybuilders/easybuild-easyconfigs#23852 (comment)

@pavelToman
Copy link
Contributor

pavelToman commented Feb 17, 2026

Even with this fix I am still hitting the error:

== FAILED: Installation ended unsuccessfully: Failed to copy directory 
/scratch/gent/vo/001/gvo00117/easybuild/RHEL9/cascadelake-ampere-ib/software/ModPipe/2.3.1-foss-2025b to 
/scratch/gent/vo/001/gvo00117/easybuild/RHEL9/cascadelake-ampere-ib/software/ModPipe/2.3.1-foss-2025b: [Errno 2] No such file or 
directory: '/scratch/gent/vo/001/gvo00117/easybuild/RHEL9/cascadelake-ampere-ib/software/ModPipe/2.3.1-foss-2025b'

The /scratch/gent/vo/001/gvo00117/easybuild/RHEL9/cascadelake-ampere-ib/software/ModPipe is empty

@Flamefire
Copy link
Contributor Author

That easyconfig didn't work in earlier EB versions did it?
Because it uses buildininstalldir and a tarball component with default install_type that deletes the installdir first:

elif self.cfg['install_type'] is None:
# Wipe and copy root of installation directory (default)
install_path = self.installdir
dirs_exist_ok = False
install_logmsg = "Copying tarball contents of %s into %s after wiping it..."

Then of course the directory won't exist anymore

@pavelToman
Copy link
Contributor

pavelToman commented Feb 17, 2026

I've just add the BLAST to the easyconfig, since the version I need was archived last month. So in original (5m old) all works ok.
But it makes sense, because I am hitting same error with v5.1.2. So this new error was introduced by the BLAST component not by the broken bundle.py

@pavelToman
Copy link
Contributor

pavelToman commented Feb 17, 2026

ModPipe v2.3.1 from this PR could be built with this update to the bundle.py (on develop) now

@Thyre
Copy link
Collaborator

Thyre commented Feb 17, 2026

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb gfbf-2024a.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb X11-20240607-GCCcore-13.3.0.eb --sanity-check-only"

@boegelbot
Copy link

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=4065 EB_ARGS="buildenv-default-foss-2023b.eb OpenSSL-3.eb Circuitscape-5.12.3-Julia-1.9.2.eb Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb gfbf-2024a.eb crypt4gh-1.7-GCC-12.3.0.eb jiter-0.4.1-GCCcore-12.3.0.eb X11-20240607-GCCcore-13.3.0.eb --sanity-check-only" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_4065 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 9725

Test results coming soon (I hope)...

Details

- notification for comment with ID 3915410384 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • FAIL buildenv-default-foss-2023b.eb (unhandled exception: expected str, bytes or os.PathLike object, not NoneType)
Traceback (most recent call last):
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/main.py", line 180, in build_and_install_software
  (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 5120, in build_and_install_one
  result = app.run_all_steps(run_test_cases=run_test_cases)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4932, in run_all_steps
  self.run_step(step_name, step_methods)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4773, in run_step
  current_method()
File "/tmp/eb-wksj913n/included-easyblocks-26fz2qet/easybuild/easyblocks/generic/bundle.py", line 493, in sanity_check_step
  fake_mod_data = self.load_fake_module(purge=True)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 1910, in load_fake_module
  fake_mod_path = self.make_module_step(fake=True)
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/buildenv.py", line 119, in make_module_step
  wrappers_dir = os.path.join(self.rpath_wrappers_dir, RPATH_WRAPPERS_SUBDIR)
File "/usr/lib64/python3.9/posixpath.py", line 76, in join
  a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

  • SUCCESS OpenSSL-3.eb

  • FAIL Circuitscape-5.12.3-Julia-1.9.2.eb (unhandled exception: 'NoneType' object has no attribute 'split')

Traceback (most recent call last):
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/main.py", line 180, in build_and_install_software
  (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 5120, in build_and_install_one
  result = app.run_all_steps(run_test_cases=run_test_cases)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4932, in run_all_steps
  self.run_step(step_name, step_methods)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4773, in run_step
  current_method()
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/juliabundle.py", line 94, in sanity_check_step
  super().sanity_check_step(custom_paths=custom_paths)
File "/tmp/eb-wksj913n/included-easyblocks-26fz2qet/easybuild/easyblocks/generic/bundle.py", line 490, in sanity_check_step
  super().sanity_check_step(*args, **kwargs)
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/juliapackage.py", line 308, in sanity_check_step
  return ExtensionEasyBlock.sanity_check_step(self, EXTS_FILTER_JULIA_PACKAGES, *args, **kwargs)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/extensioneasyblock.py", line 184, in sanity_check_step
  with self.fake_module_environment(extra_modules=extra_modules):
File "/usr/lib64/python3.9/contextlib.py", line 119, in __enter__
  return next(self.gen)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 2358, in fake_module_environment
  fake_mod_data = self.load_fake_module(purge=True, extra_modules=extra_modules)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 1910, in load_fake_module
  fake_mod_path = self.make_module_step(fake=True)
File "/tmp/eb-wksj913n/included-easyblocks-26fz2qet/easybuild/easyblocks/generic/bundle.py", line 471, in make_module_step
  return super().make_module_step(*args, **kwargs)
File "/home/boegelbot/easybuild/easybuild-framework/easybuild/framework/easyblock.py", line 4536, in make_module_step
  txt += self.make_module_extra()
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/juliabundle.py", line 98, in make_module_extra
  return super().make_module_extra(*args, **kwargs)
File "/tmp/eb-wksj913n/included-easyblocks-26fz2qet/easybuild/easyblocks/generic/bundle.py", line 482, in make_module_extra
  return super().make_module_extra(*args, **kwargs)
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/juliapackage.py", line 324, in make_module_extra
  mod += self.module_generator.append_paths('JULIA_LOAD_PATH', [self.julia_env_path(absolute=False, base=False)])
File "/home/boegelbot/easybuild/easybuild-easyblocks/easybuild/easyblocks/generic/juliapackage.py", line 125, in julia_env_path
  julia_version = get_software_version('Julia').split('.')
AttributeError: 'NoneType' object has no attribute 'split'

  • SUCCESS Perl-bundle-CPAN-5.38.2-GCCcore-13.3.0.eb

  • SUCCESS gfbf-2024a.eb

  • SUCCESS crypt4gh-1.7-GCC-12.3.0.eb

  • SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb

  • SUCCESS X11-20240607-GCCcore-13.3.0.eb

Build succeeded for 6 out of 8 (total: 2 mins 42 secs) (8 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.7, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.23
See https://gist.github.com/boegelbot/b8e5a3247808777d4e7b9db7c71df615 for a full test report.

@Thyre
Copy link
Collaborator

Thyre commented Feb 17, 2026

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="ROCm-LLVM-19.0.0-GCCcore-14.2.0-ROCm-6.4.1.eb --module-only"

@Flamefire
Copy link
Contributor Author

buildenv doesn't support --sanity-check-only due to make_module_step:

  • It accesses self.rpath_wrappers_dir which is set in prepare_step and overwritten in install_step
  • It needs self.toolchain.vars which is also set in prepare_step (in framework)

Not sure why --sanity-check-only requires a fake module if the real one exists

@Thyre
Copy link
Collaborator

Thyre commented Feb 17, 2026

buildenv doesn't support --sanity-check-only due to make_module_step:

  • It accesses self.rpath_wrappers_dir which is set in prepare_step and overwritten in install_step

  • It needs self.toolchain.vars which is also set in prepare_step (in framework)

Not sure why --sanity-check-only requires a fake module if the real one exists

Yeah... copied that from an older PR, see also #4040

@boegelbot
Copy link

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=4065 EB_ARGS="ROCm-LLVM-19.0.0-GCCcore-14.2.0-ROCm-6.4.1.eb --module-only" EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_4065 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 9730

Test results coming soon (I hope)...

Details

- notification for comment with ID 3915631003 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS ROCm-LLVM-19.0.0-GCCcore-14.2.0-ROCm-6.4.1.eb

Build succeeded for 1 out of 1 (total: 3 mins 19 secs) (1 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.7, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.23
See https://gist.github.com/boegelbot/349f0815849cd1ea1f57c7e7e0c23056 for a full test report.

@Flamefire
Copy link
Contributor Author

Yeah... copied that from an older PR, see also #4040

Fixed in #4068

@Thyre
Copy link
Collaborator

Thyre commented Feb 17, 2026

Changes here look sane to me. Cannot test this further at the moment.

We should probably think about a well-suited test suite for this EasyBlock to test these PRs with.
We have so many corner cases which can break because we forgot something.

Probably something for a regression test suite.

@Thyre Thyre added the bug fix label Feb 17, 2026
@boegel boegel added this to the release after 5.2.1 milestone Feb 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Comments