Skip to content

Conversation

FranciscoTGouveia
Copy link
Contributor

@FranciscoTGouveia FranciscoTGouveia commented Aug 3, 2025

As of now, each component of a toolchain is downloaded sequentially, leaving room for performance improvements.
This is particularly notorious when considering interleaving downloads with installation, as discussed in #731.

Following the work made in #4426, as the DownloadTracker is now indicatif-based, doing the progress reporting for concurrent downloads is much easier.
As such, this PR comes to enable downloading different components concurrently (possibly closing #3018).

Performance-wise, benchmarks using hyperfine showed a small (1.1x) speedup -- which doesn't seem noticeable but lays the foundations for a future PR to interleave the download with the installation (of components).
The benchmarks were ran 50 times (with 5 warmup runs) over a 50 Mbps connection, each downloading an entire toolchain.

Many thanks to @rami3l and his PR for refactoring part of the test suite for the purpose of this PR.

@FranciscoTGouveia
Copy link
Contributor Author

For reference, the current download of components (sequential) looks like this:

before-ezgif com-speed

... and the changes introduced by this PR go like this:

after-ezgif com-speed

Please note that both animations were sped up and they are not meaningful for comparing performances.

@FranciscoTGouveia FranciscoTGouveia force-pushed the concurrent-downloads branch 3 times, most recently from 10d1514 to 7093333 Compare August 5, 2025 13:13
@FranciscoTGouveia FranciscoTGouveia marked this pull request as ready for review August 5, 2025 13:24
@rami3l rami3l self-assigned this Aug 10, 2025
rami3l

This comment was marked as duplicate.

@FranciscoTGouveia FranciscoTGouveia force-pushed the concurrent-downloads branch 2 times, most recently from 837bf58 to 4737f25 Compare August 11, 2025 14:21
@FranciscoTGouveia
Copy link
Contributor Author

FranciscoTGouveia commented Aug 11, 2025

After my investigation stemming from this comment, I have concluded that the changes presented on this PR are not consistent with the previous behavior of displaying a progress bar during the installation of the component.

Would this be necessary, especially considering that the next step will be to interleave the downloads with the installation?

@rami3l
Copy link
Member

rami3l commented Aug 12, 2025

I have concluded that the changes presented on this PR are not consistent with the previous behavior of displaying a progress bar during the installation of the component.

Would this be necessary, especially considering that the next step will be to interleave the downloads with the installation?

@FranciscoTGouveia No, I don't think it is necessary for that part of the behavior to stay unchanged, as you have already mentioned. Of course, if it is easy to add it back, then it might still be interesting to do so.

@FranciscoTGouveia FranciscoTGouveia force-pushed the concurrent-downloads branch 3 times, most recently from d071c86 to f4e41d5 Compare August 15, 2025 20:32
@FranciscoTGouveia
Copy link
Contributor Author

Thank you for all the reviews comments.
Before anything I would like to apologize for not explaining my questionable decisions in the PR description from the start.
I will definitely have this in mind in my future contributions.

Additionally, I would like to highlight one point:

  • The downloads are being executed using .buffered() rather than .buffer_unordered(). A similar situation arose here, however, in this case, the speedup is somewhat despicable (only a 1.1x speedup). Additionally, making this change would change the order that the installations are made in, which I think is unnecessary for this PR. In a future contribution aiming to interleave the downloads with installations, if the installation process is modified, we can revisit this idea.

Copy link
Member

@rami3l rami3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking much better to me! Many thanks @FranciscoTGouveia for the work and @djc for a second round of review :)

I've left just a final nit but otherwise I believe this PR is quite ready to be merged 🎉

@djc
Copy link
Contributor

djc commented Aug 16, 2025

(Also, CI failures...)

Some notifications needed to be updated to include the download URL,
enabling the identification of the component being downloaded. This was
necessary for accurate progress reporting of each component.
@FranciscoTGouveia
Copy link
Contributor Author

I believe it was a spurious failure.
All comments have now been addressed, and the CI is passing :)

@rami3l rami3l added this pull request to the merge queue Aug 16, 2025
Merged via the queue into rust-lang:master with commit 3f6b75b Aug 16, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants