[Isolated] Update Pypi dependencies and install efs-proxy dependency #3011

hgreebe · 2025-08-16T01:14:00Z

Description of changes

Adding PyPi dependencies for latest Python version 3.12 and using appropriate folder names
Installing Packages with --no-build-isolation as with Python 3.12.x so that we check for existing dependecies.
** packages are installed with isolation from Py 3.12, where they do not check if the dependencies they want already exist and try to install them again.
Installing efs-proxy dependecies from s3 in Isolated region as they cannot connect to cargo and get below error

Stdout: failed to download from `https://index.crates.io/config.json
Stdout: [6] Couldn't resolve host name (Could not resolve host: index.crates.io)

adding GIT_REF for cookbook util script for uploading it in S3

Tests

Build rhel8, AL2, and AL2023 AMIs in ADC
Ran isolated region integ tests

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This reverts commit 976d479.

…me inside the Tar

…side the Tar

….12.8 uses setup.py based installation where it uses a isolated build instead of looking at existing site-packages

…allation

codecov · 2025-08-16T01:14:47Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.50%. Comparing base (98c689e) to head (29ba943).
⚠️ Report is 44 commits behind head on release-3.13.

Additional details and impacted files

@@              Coverage Diff              @@
##           release-3.13    #3011   +/-   ##
=============================================
  Coverage         75.50%   75.50%           
=============================================
  Files                23       23           
  Lines              2356     2356           
=============================================
  Hits               1779     1779           
  Misses              577      577

Flag	Coverage Δ
unittests	`75.50% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

himani2411 · 2025-08-18T17:05:08Z

cookbooks/aws-parallelcluster-platform/spec/unit/recipes/cookbook_virtualenv_spec.rb

          is_expected.to write_node_attributes('dump node attributes')
        end
-
-        it 'installs python packages' do


why is this removed?

himani2411 · 2025-08-18T17:06:53Z

cookbooks/aws-parallelcluster-environment/resources/efs/partial/_install_from_tar.rb


 action :install_utils do
+  package_version = new_resource.efs_utils_version
+  efs_utils_tarball = "#{node['cluster']['sources_dir']}/efs-utils-#{package_version}.tar.gz"


Why is this moved as part of each action?

you can't use new_resource outside of an actionblock, so I can't define the variable globally because I would not be able to use new_resource

himani2411 · 2025-08-18T19:09:13Z

cookbooks/aws-parallelcluster-platform/spec/unit/recipes/cookbook_virtualenv_spec.rb

+      cached(:aws_region) { 'us-iso-test' }
+
+      before do
+        allow_any_instance_of(String).to receive(:start_with?).with("us-iso").and_return(true)


Lets add the test for both partition where start_with is true and false

…ws#3011) * [Isolated] Install cfn-dependencies only for AL2 * Revert "[Isolated] Install cfn-dependencies only for AL2" This reverts commit 976d479. * [Isolated] USe latest cfn-dependencies * [Isolated] Using Git REF for Uploading cookbook * [Isolated] Rename the cfn-dependecies files * [Isolated] Chnage the name of Cookbook Dependencies and the folder name inside the Tar * [Isolated] Chnage the name of CFN Dependencies and the folder name inside the Tar * [Isolated] Installing Cfn-bootstrap using `--no-build-isolation` as 3.12.8 uses setup.py based installation where it uses a isolated build instead of looking at existing site-packages * [Isolated] Install efs-proxy cargo dependecies for isolated environment * [Isolated] Install new node pypi dependencies and move efs-proxy installation * [Isolated] Only install efs-proxy deps when in adc regions * [Isolated] Only install efs-proxy-deps in adc * [Isolated] Fix unit tests * [Isolated] Test python pacakges are installed when in an ADC region --------- Co-authored-by: Himani Anil Deshpande <[email protected]>

…3011) * [Isolated] Install cfn-dependencies only for AL2 * Revert "[Isolated] Install cfn-dependencies only for AL2" This reverts commit 976d479. * [Isolated] USe latest cfn-dependencies * [Isolated] Using Git REF for Uploading cookbook * [Isolated] Rename the cfn-dependecies files * [Isolated] Chnage the name of Cookbook Dependencies and the folder name inside the Tar * [Isolated] Chnage the name of CFN Dependencies and the folder name inside the Tar * [Isolated] Installing Cfn-bootstrap using `--no-build-isolation` as 3.12.8 uses setup.py based installation where it uses a isolated build instead of looking at existing site-packages * [Isolated] Install efs-proxy cargo dependecies for isolated environment * [Isolated] Install new node pypi dependencies and move efs-proxy installation * [Isolated] Only install efs-proxy deps when in adc regions * [Isolated] Only install efs-proxy-deps in adc * [Isolated] Fix unit tests * [Isolated] Test python pacakges are installed when in an ADC region --------- Co-authored-by: Himani Anil Deshpande <[email protected]>

Himani Anil Deshpande and others added 13 commits August 6, 2025 10:41

[Isolated] Install cfn-dependencies only for AL2

7c345ec

Revert "[Isolated] Install cfn-dependencies only for AL2"

a972479

This reverts commit 976d479.

[Isolated] USe latest cfn-dependencies

857638c

[Isolated] Using Git REF for Uploading cookbook

b56bb2c

[Isolated] Rename the cfn-dependecies files

e49d4a1

[Isolated] Chnage the name of Cookbook Dependencies and the folder na…

dd33ac1

…me inside the Tar

[Isolated] Chnage the name of CFN Dependencies and the folder name in…

506bb33

…side the Tar

[Isolated] Installing Cfn-bootstrap using --no-build-isolation as 3…

5a30c1a

….12.8 uses setup.py based installation where it uses a isolated build instead of looking at existing site-packages

[Isolated] Install efs-proxy cargo dependecies for isolated environment

fc056aa

[Isolated] Install new node pypi dependencies and move efs-proxy inst…

38a7577

…allation

[Isolated] Only install efs-proxy deps when in adc regions

4278843

[Isolated] Only install efs-proxy-deps in adc

ab36358

[Isolated] Fix unit tests

29ba943

hgreebe requested review from a team as code owners August 16, 2025 01:14

himani2411 reviewed Aug 18, 2025

View reviewed changes

[Isolated] Test python pacakges are installed when in an ADC region

ddbc975

hgreebe force-pushed the release-3.13.2-isolated branch from de59cb8 to ddbc975 Compare August 18, 2025 20:49

himani2411 approved these changes Aug 18, 2025

View reviewed changes

hgreebe added the skip-changelog-update label Aug 21, 2025

hgreebe enabled auto-merge (squash) August 21, 2025 12:14

hgreebe merged commit 4237326 into aws:release-3.13 Aug 21, 2025
26 of 43 checks passed

hgreebe deleted the release-3.13.2-isolated branch August 21, 2025 12:14

himani2411 mentioned this pull request Oct 6, 2025

[Isolated] Cherry picks from release-3.13 #3034

Merged

himani2411 mentioned this pull request Oct 6, 2025

[Isolated] Cherry picks from release-3.13 #3035

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Isolated] Update Pypi dependencies and install efs-proxy dependency #3011

[Isolated] Update Pypi dependencies and install efs-proxy dependency #3011

Uh oh!

hgreebe commented Aug 16, 2025

Uh oh!

codecov bot commented Aug 16, 2025

Uh oh!

himani2411 Aug 18, 2025

Uh oh!

himani2411 Aug 18, 2025

Uh oh!

hgreebe Aug 18, 2025 •

edited

Loading

Uh oh!

himani2411 Aug 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[Isolated] Update Pypi dependencies and install efs-proxy dependency #3011

[Isolated] Update Pypi dependencies and install efs-proxy dependency #3011

Uh oh!

Conversation

hgreebe commented Aug 16, 2025

Description of changes

Tests

Uh oh!

codecov bot commented Aug 16, 2025

Codecov Report

Uh oh!

himani2411 Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

himani2411 Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

hgreebe Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

himani2411 Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hgreebe Aug 18, 2025 •

edited

Loading