Skip to content

Apply socket timeout to S3 CRT connections#7024

Open
jorgee wants to merge 1 commit intomasterfrom
add-socket-timeout-s3-crt
Open

Apply socket timeout to S3 CRT connections#7024
jorgee wants to merge 1 commit intomasterfrom
add-socket-timeout-s3-crt

Conversation

@jorgee
Copy link
Copy Markdown
Contributor

@jorgee jorgee commented Apr 10, 2026

This pull request updates how the socket_timeout configuration is handled for AWS S3 transfers using the CRT client in the SDK v2 integration. The main improvement is that when socket_timeout is set, it is now translated into a CRT connection health check, allowing the detection and closure of stalled S3 connections. The documentation and tests have also been updated to reflect this new behavior.

AWS S3 CRT Client Timeout Handling:

  • The socket_timeout property for S3 transfers using the CRT client is now mapped to a connection health check that enforces a minimum throughput of 1 byte per second within the specified timeout period, instead of being ignored. This helps detect and close stalled CRT connections.

Documentation Updates:

  • The user guide (aws-java-sdk-v2.md) and configuration reference (config.md) have been updated to clarify the new behavior of the aws.client.socketTimeout option for CRT-based S3 transfers, including the lack of a default and the new health check mechanism. [1] [2]

Testing Enhancements:

  • The test suite for S3 client configuration now verifies that the CRT health configuration is set with the correct minimum throughput and timeout values when socket_timeout is specified.

Dependency and Import Adjustments:

  • Added the necessary import for S3CrtConnectionHealthConfiguration to support the new health check configuration.
  • Cleaned up unused imports in the S3 client configuration test file.

Signed-off-by: jorgee <jorge.ejarque@seqera.io>
@jorgee jorgee requested a review from a team as a code owner April 10, 2026 17:09
@netlify
Copy link
Copy Markdown

netlify bot commented Apr 10, 2026

Deploy Preview for nextflow-docs-staging ready!

Name Link
🔨 Latest commit 552a511
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69d92ec12768710008a00546
😎 Deploy Preview https://deploy-preview-7024--nextflow-docs-staging.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.


The `aws.client.connectionTimeout` option continues to work as before, controlling the timeout for establishing new connections.

The `aws.client.socketTimeout` option behaves differently depending on the HTTP client. For synchronous connections, it applies directly as a socket timeout with a default of `30000` ms. For S3 transfers using the CRT client, no default is applied; when explicitly set, this value is translated into a CRT connection health check that enforces a minimum throughput of 1 byte per second within the specified timeout period. This allows Nextflow to detect and close stalled CRT connections that are no longer making progress.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we know the default for the sync client is 30s, can we just apply that default manually to the CRT client as well? Then we could simply say that the default is 30s across the board

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants