Skip to content

Rework the integration test timeout logic #9322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

srawlins
Copy link
Contributor

@srawlins srawlins commented Jul 17, 2025

The 8-minute timeout was being applied to every test, even successful test results.

In this code:

      await Future.any([
        process.exitCode,
        timeout,
      ]);

While this does return the first future that completes, the process will still wait for any other futures to complete, including an 8-minute delayed Future.

Additionally, each testWidgets call has a timeout, which is either infinite or 10 minutes (according to the docs). Either way, the timeout is longer than the 8-minute timeout, so we need to reduce these too. I chose some timeouts somewhat at random, but I believe conservatively, at 2, 3, or 4 minutes.

This change brings the following GitHub Action times down as follows:

  • "macos-latest devtools_app integration-test integration_dart2js - flutter - shard 1/3" - 11m 20s → 4m 43s
  • "macos-latest devtools_app integration-test integration_dart2js - flutter - shard 2/3" - 14m 05s → 7m 55s
  • "macos-latest devtools_app integration-test integration_dart2js - flutter - shard 3/3" - 16m 21s → 8m 44s
  • "macos-latest devtools_app integration-test integration_dart2js - flutter-web - shard 1/2" - 13m 15s → 5m 58s
  • "macos-latest devtools_app integration-test integration_dart2js - flutter-web - shard 2/2" - 12m 46s → 7m 34s
  • "macos-latest devtools_app integration-test integration_dart2js - dart-cli - shard 1/1" - 13m 20s → 7m 42s
  • "windows-latest devtools_app integration-test integration_dart2js - flutter - shard 1/3" - 15m 28s → 7m 44s
  • "windows-latest devtools_app integration-test integration_dart2js - flutter - shard 2/3" - 13m 11s → 6m 18s
  • "windows-latest devtools_app integration-test integration_dart2js - flutter - shard 3/3" - 17m 16s → 11m 43s
  • "windows-latest devtools_app integration-test integration_dart2js - flutter-web - shard 1/2" - 14m 36s → 8m 30s
  • "windows-latest devtools_app integration-test integration_dart2js - flutter-web - shard 2/2" - 14m 10s → 7m 05s
  • "windows-latest devtools_app integration-test integration_dart2js - dart-cli - shard 1/1" - 14m 57s → 8m 47s
  • "ubuntu-latest devtools_extensions integration-test integration_dart2js" - 10m 41s → 3m 18s
  • "windows-latest devtools_extensions integration-test integration_dart2js" - 12m 03s → 4m 56s

@kenzieschmoll
Copy link
Member

The reduction in test times here is fantastic! Nice work!

});
testWidgets(
'connect to app and switch tabs',
timeout: const Timeout(Duration(minutes: 2)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for consistency, can we add consts shortTimeout, mediumTimeout and longTimeout and put them in integration_test/test_infra/_utils.dart?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants