[windows] upgrade to Python 3.10.1 embeddable #83488

charles-zablit · 2025-08-01T16:13:58Z

This patch upgrades the Python version for LLDB on Windows to 3.10.1. It also embeds Python in the toolchain. This fixes 2 things:

Python will drop support for 3.9 in October 2025.
There is no ARM64 installer for Python 3.9.10 on Windows, making lldb challenging to use on ARM64 Windows.

This is a follow-up to this discussion on the forums.

rdar://153842143

charles-zablit · 2025-08-01T16:14:44Z

@swift-ci please test

charles-zablit · 2025-08-01T16:15:29Z

utils/build.ps1

@@ -349,11 +357,6 @@ $PythonModules = @{
    SHA256 = "353815f59a7f64cdaca1c0307ee13558a0512f6db064e92fe833784f08539c7a";
    Dependencies = @();
  };
-  "unittest2" = @{


unittest2 is not compatible with python3.10+ since it's a back port of unittest. There is no need for it anymore.

Lets leave that in place until we have switched to 3.10 by default.

The only references to unittest2 in the whole toolchain will be removed in swiftlang/llvm-project#11084. I think it's safe to remove it, as even the 3.9.10 version of Python comes with unittest.

Ah, nice! In that case, we should split this change out and merge it whenever swiftlang/llvm-project#11084 gets merged.

compnerd · 2025-08-01T16:15:31Z

utils/build.ps1

+    };
+    ARM64_Embedded = @{
+      URL = "https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-arm64.zip";
+      SHA256 = "1f9e215fe4e8f22a8e8fba1859efb1426437044fb3103ce85794630e3b511bc2";
    };
  }


Lets leave the definitions for 3.9, allowing to optionally upgrade to 3.10 while we iterate to get this working. That will allow you to make progress and get the changes merged earlier rather than trying to keep up to date with the changes in this script.

Fixed, thanks

Is there a particular reason that we need both the regular and the embedded URLs? Could we not get away with just the embedded?

Yes, embeddable python does not come with pip. We could download and run get-pip.py, but I don't think we would gain a lot of CI time in doing so. I have not tried it out.

The reasoning for the version selection here is not CI times but rather to match what llvm.org distributes. I think that we don't need to inject pip unless we are building the installer for distribution (i.e. nightlies/releases).

compnerd · 2025-08-01T16:16:13Z

utils/build.ps1

+  Copy-Item -Force `
+    -Path "$(Get-EmbeddedPythonPath $HostPlatform)\*" `
+    -Destination "$($HostPlatform.ToolchainInstallRoot)\usr\bin" `
+    -Recurse


No, we don't want to embed the Python release. This was what we discussed on the forums - this should be an additional separate MSI that you are building and optionally installing.

Embed was not the right choice of words. I want to put the files in a place where wix can find them for the follow up patch here: swiftlang/swift-installer-scripts#447

I don't think that we should merge this into the toolchain. I think that we should be doing a separate installation: S:\Program Files\Swift\Python-3.10.1 for the image (the WiX authoring should be able to pick it up from there).

In the latest revision, the python files are now extracted to S:\Program Files\Swift\Python. Do we really need the version number? Doing so would mean passing a new parameter to Wix to specify the files path right?

I would prefer that we keep the version number. Upgrading python is something that would be rare, so I don't think that it is a big deal. We already have the information in build.ps1 where we can pass that down.

compnerd · 2025-08-04T15:24:54Z

utils/build.ps1

+    };
+    ARM64_Embedded = @{
+      URL = "https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-arm64.zip";
+      SHA256 = "1f9e215fe4e8f22a8e8fba1859efb1426437044fb3103ce85794630e3b511bc2";
    };
  }


Is there a particular reason that we need both the regular and the embedded URLs? Could we not get away with just the embedded?

compnerd · 2025-08-04T15:26:03Z

utils/build.ps1

@@ -349,11 +357,6 @@ $PythonModules = @{
    SHA256 = "353815f59a7f64cdaca1c0307ee13558a0512f6db064e92fe833784f08539c7a";
    Dependencies = @();
  };
-  "unittest2" = @{


Ah, nice! In that case, we should split this change out and merge it whenever swiftlang/llvm-project#11084 gets merged.

compnerd · 2025-08-04T15:26:36Z

utils/build.ps1

@@ -602,6 +615,10 @@ function Get-PythonPath([Hashtable] $Platform) {
  return [IO.Path]::Combine("$BinaryCache\", "Python$($Platform.Architecture.CMakeName)-$PythonVersion")
 }

+function Get-EmbeddedPythonPath([Hashtable] $Platform) {


Again, is there a bigger picture around the embedded vs non-embedded python that the extraction needs special logic?

The only reason is pip missing in embeddedable python. We build with the regular Python, but we bundle the embeddable python in the installer.

There-in lies the trick! We can inject pip into the packaging :)

compnerd · 2025-08-04T15:27:42Z

utils/build.ps1

-    "$($HostPlatform.ToolchainInstallRoot)\usr\lib\swift\windows\_InternalSwiftScan.lib" `
-    "$($HostPlatform.ToolchainInstallRoot)\usr\lib"
+    -Path "$($HostPlatform.ToolchainInstallRoot)\usr\lib\swift\windows\_InternalSwiftScan.lib" `
+    -Destination "$($HostPlatform.ToolchainInstallRoot)\usr\lib"


The named parameters are a nice cleanup; we could split those out and merge them as a cleanup before even getting the python 3.10.1 upgrade completed.

Opened #83517 to split the changes 👍

charles-zablit requested a review from compnerd as a code owner August 1, 2025 16:13

charles-zablit self-assigned this Aug 1, 2025

charles-zablit requested a review from adrian-prantl August 1, 2025 16:14

charles-zablit added Windows Platform: Windows lldb labels Aug 1, 2025

charles-zablit commented Aug 1, 2025

View reviewed changes

compnerd requested changes Aug 1, 2025

View reviewed changes

charles-zablit mentioned this pull request Aug 4, 2025

[windows] add Python 3.10.1 to the installer swiftlang/swift-installer-scripts#447

Open

charles-zablit force-pushed the charles-zablit/windows/upgrade-python-to-3.10 branch from 3a2b03f to bc91500 Compare August 4, 2025 14:36

[windows] upgrade to Python 3.10 embeddable

bd4476a

charles-zablit force-pushed the charles-zablit/windows/upgrade-python-to-3.10 branch from bc91500 to bd4476a Compare August 4, 2025 15:12

compnerd reviewed Aug 4, 2025

View reviewed changes

[windows] upgrade to Python 3.10.1 embeddable #83488

Are you sure you want to change the base?

[windows] upgrade to Python 3.10.1 embeddable #83488

Conversation

charles-zablit commented Aug 1, 2025

Uh oh!

charles-zablit commented Aug 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!