Re-apply "Tests: enable Swift PM tests in Windows toolchain build"" #82274

bkhouri · 2025-06-16T20:16:50Z

Reverts #82265, which re-applies #80405

Enable the SwiftPM tests in the Windows toolchain build to get extra confidence the change did not introduce any regression on the Windows platform.

In addition to the re-application of #80405, execute SwiftPM tesst in parallel

Fixes: swiftlang/swift-package-manager#8895
Depends on: swiftlang/swift-package-manager#8899

bkhouri · 2025-06-16T20:16:58Z

@swift-ci smoke test

bkhouri · 2025-06-17T14:54:36Z

@swift-ci smoke test

bkhouri · 2025-06-18T17:31:02Z

Windows platform build failed with Test Case 'PrepareForIndexTests.testEnaBuild timed out (after 60 minutes). Marking the build as aborted.

Trying again

@swift-ci smoke test windows

bkhouri · 2025-06-19T15:17:59Z

Windows Platform build failed with Test Case 'VersionSetSpecifierTests.testPrereleases' started at 2025-06-18 23Build timed out (after 60 minutes). Marking the build as aborted.

Between this and the previous execution, the test that hangs in non-deterministic.

bkhouri · 2025-06-20T21:19:42Z

@swift-ci smoke test windows

bkhouri · 2025-06-21T21:13:20Z

without code changes, the Windows platform build failed

Test Case 'ManifestEditTests.testAddPackageDependencyExistingNoComma' started at 2025-06-21 02:46:51.493

C:\Users\swift-ci\jenkins\workspace\swift-PR-windoBuild timed out (after 60 minutes). Marking the build as aborted.

bkhouri · 2025-06-30T13:53:55Z

@swift-ci test windows

bkhouri · 2025-07-02T12:34:13Z

@swift-ci test windows

bkhouri · 2025-07-02T18:56:40Z

swiftlang/swift-package-manager#8891

@swift-ci test

compnerd · 2025-07-02T19:56:30Z

In its current state, this is adding ~65m to the overall test time, which might be a bit excessive. I have a set of patches up which need some help to get merged, but bring this time down to ~8m.

bkhouri · 2025-07-02T22:28:04Z

In its current state, this is adding ~65m to the overall test time, which might be a bit excessive. I have a set of patches up which need some help to get merged, but bring this time down to ~8m.

I'm in favour of submitting this, once the test are passing, and then working on the other changes to bring the time down. This will ensure other changes that execute this script will execute the SwiftPM tests as well, helping to ensure regressions are not introduced.

Can you please share the PRs that bring this time down?

compnerd · 2025-07-03T00:17:28Z

Sure, the changes are:

bkhouri · 2025-07-04T13:07:45Z

Adding comment for automatic GitHub traceability.

This PR fixes: swiftlang/swift-package-manager#8895

bkhouri · 2025-07-04T14:02:30Z

swiftlang/swift-package-manager#8891

@swift-ci smoke test

bkhouri · 2025-07-07T12:25:11Z

swiftlang/swift-package-manager#8899

@swift-ci test

bkhouri · 2025-07-07T19:17:38Z

swiftlang/swift-package-manager#8899

@swift-ci test

bkhouri · 2025-07-07T19:17:56Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-07-08T13:23:48Z

It looks like the Swift Test Windows Platform has test hangs

[1929/1930] Testing FunctionalTests.ResourcesTests/testResourcesInMixedClangPackage
[1930/1930] Testing FunctionalTests.ResourcesTests/testResourcesOutsideOfTargetCanBeIncluded
Build timed out (after 60 minutes). Marking the build as aborted.
Build was aborted

bkhouri · 2025-07-09T04:35:58Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-07-09T13:53:52Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-07-09T15:55:09Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test macOS

bkhouri · 2025-07-09T17:58:51Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test macOS

bkhouri · 2025-07-09T19:24:08Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test macOS

bkhouri · 2025-07-10T13:37:40Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test macOS

bkhouri · 2025-07-10T13:40:33Z

The SwiftPM PR was rebased. so re-run all the pipelines.

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-07-15T01:40:33Z

he SwiftPM PR was updated. so re-run all the pipelines.

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-08-01T17:05:28Z

The SwiftPM PR was updated, so run all pipelines

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-08-05T18:47:56Z

The SwiftPM PR was updated, so run all pipelines

swiftlang/swift-package-manager#8899

@swift-ci smoke test

bkhouri · 2025-08-06T13:59:37Z

swiftlang/swift-package-manager#8899

@swift-ci smoke test windows

Update test expectation for executing SwiftPM tests in the Windows Platform Test PR build - the script found in https://github.com/swiftlang/swift/blob/main/utils/build.ps1. Relates to: swiftlang/swift#82274 Relates to: #8895 Depends on: #8975

bkhouri · 2025-08-07T02:20:25Z

Dependent PR has been merged. so let's run the testing again

@swift-ci test

dschaefer2 · 2025-08-07T13:59:08Z

This is adding an hour to the Windows CI job (it's now 6 hours). When you rerun to address the Linux and macOS failures, can we rerun the Windows too to double check?

bkhouri · 2025-08-07T14:12:29Z

This is adding an hour to the Windows CI job (it's now 6 hours). When you rerun to address the Linux and macOS failures, can we rerun the Windows too to double check?

The two build failures are not required and they haven't been run in a while. With that said, my plan was to retriever the required build before merging to ensure the builds still pass.

bkhouri · 2025-08-07T14:13:32Z

@swift-ci test

bkhouri · 2025-08-07T14:13:52Z

@swift-ci smoke test

Reverts #82265, which re-applies #80405 Enable the SwiftPM tests in the Windows toolchain build to get extra confidence the change did not introduce any regression on the Windows platform. Fixes: swiftlang/swift-package-manager#8895

bkhouri · 2025-08-07T14:16:16Z

@swift-ci smoke test

bkhouri · 2025-08-07T14:17:54Z

@compnerd, @shahmishal : could you please review this? This change seems to add 1 hours to the Windows platform build.

compnerd · 2025-08-07T20:03:59Z

This is adding ~45m to the build:

Build-Compilers           Windows  x86_64  00:53:36.11  13.2% 
Test-Foundation           Windows  x86_64  00:53:29.96  13.2% 
Test-PackageManager       Windows  x86_64  00:42:35.09  10.5%

This puts it in third place, which is pretty big. I think that @ahoppen had a new trick up his sleeve (using Xcode multi-space workspaces) to reduce the Test-Foundation time. I would like to see how this compares once that is done. I know that with the test only build (that Foundation does) SPM is possible to test in ~5-7m vs 45, which is still a huge delta.

shahmishal · 2025-08-07T20:06:47Z

The foundation regression most likely is coming from #83572

shahmishal · 2025-08-07T20:07:36Z

Before merging this PR, I would like to understand which tests are taking the longest.

dschaefer2 · 2025-08-08T13:35:32Z

Before merging this PR, I would like to understand which tests are taking the longest.

Agreed. I am wondering if it's the integration tests which call "swift build". Those probably could be done in a nightly regression run instead of on every toolchain build. Just a possibility.

We're also looking forward to completing the SwiftBuild build system transition which I believe will give us build performance that is as good or better than CMake/Ninja. At least that's a goal which would allow us to unify the builds of at least the Swift packages in the toolchain. I'd rather not throw in short term changes that would just make maintenance of the builds even harder.

bkhouri marked this pull request as ready for review July 2, 2025 19:54

bkhouri requested a review from compnerd as a code owner July 2, 2025 19:54

compnerd requested a review from shahmishal July 2, 2025 19:55

bkhouri force-pushed the revert-82265-revert-80405-t/main/update_swiftpm_presets branch from 34cd647 to 0bb3d3d Compare July 7, 2025 19:17

bkhouri mentioned this pull request Jul 7, 2025

Tests: Update test expectation - take 2 swiftlang/swift-package-manager#8899

Merged

This was referenced Jul 9, 2025

Tests: swiftRunQuietLogLevel fails on Windows Platform build with native build system swiftlang/swift-package-manager#8911

Open

Tests: swiftRunQuietLogLevel fails on Windows Platform build with SwiftBuild build system swiftlang/swift-package-manager#8912

Open

bkhouri marked this pull request as draft July 15, 2025 01:40

bkhouri self-assigned this Aug 6, 2025

bkhouri marked this pull request as ready for review August 7, 2025 02:20

bkhouri and others added 2 commits August 7, 2025 10:15

Windows: execute SwiftPM tesst in parallel

3ac47b8

bkhouri force-pushed the revert-82265-revert-80405-t/main/update_swiftpm_presets branch from 0bb3d3d to 3ac47b8 Compare August 7, 2025 14:15

Re-apply "Tests: enable Swift PM tests in Windows toolchain build"" #82274

Are you sure you want to change the base?

Re-apply "Tests: enable Swift PM tests in Windows toolchain build"" #82274

Conversation

bkhouri commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bkhouri commented Jun 16, 2025

Uh oh!

bkhouri commented Jun 17, 2025

Uh oh!

bkhouri commented Jun 18, 2025

Uh oh!

bkhouri commented Jun 19, 2025

Uh oh!

bkhouri commented Jun 20, 2025

Uh oh!

bkhouri commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bkhouri commented Jun 30, 2025

Uh oh!

bkhouri commented Jul 2, 2025

Uh oh!

bkhouri commented Jul 2, 2025

Uh oh!

compnerd commented Jul 2, 2025

Uh oh!

bkhouri commented Jul 2, 2025

Uh oh!

compnerd commented Jul 3, 2025

Uh oh!

bkhouri commented Jul 4, 2025

Uh oh!

bkhouri commented Jul 4, 2025

Uh oh!

bkhouri commented Jul 7, 2025

Uh oh!

bkhouri commented Jul 7, 2025

Uh oh!

bkhouri commented Jul 7, 2025

Uh oh!

bkhouri commented Jul 8, 2025

Uh oh!

bkhouri commented Jul 9, 2025

Uh oh!

bkhouri commented Jul 9, 2025

Uh oh!

bkhouri commented Jul 9, 2025

Uh oh!

bkhouri commented Jul 9, 2025

Uh oh!

bkhouri commented Jul 9, 2025

Uh oh!

bkhouri commented Jul 10, 2025

Uh oh!

bkhouri commented Jul 10, 2025

Uh oh!

bkhouri commented Jul 15, 2025

Uh oh!

bkhouri commented Aug 1, 2025

Uh oh!

bkhouri commented Aug 5, 2025

Uh oh!

bkhouri commented Aug 6, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

dschaefer2 commented Aug 7, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

bkhouri commented Aug 7, 2025

Uh oh!

bkhouri commented Jun 16, 2025 •

edited

Loading

bkhouri commented Jun 21, 2025 •

edited

Loading