[Performance] Slow JUnit 5 test discovery in a large test source folder

[wenytang-ms]

Summary

microsoft/vscode-java-test#1915 reports that the complete "Searching tests..." phase takes approximately 100–150+ seconds when running all tests from a large project node.

Thread dumps collected during this phase repeatedly show work under:

ASTParser.createAST
CompilationUnitResolver.resolve
JUnit5TestFinder.internalIsTest
JUnit5TestFinder.findTestsInContainer

This issue focuses on the JUnit 5 discovery path observed while the command is running in JDT LS. It does not assume that the complete 100–150 seconds is spent inside JUnit5TestFinder, because vscode-java-test performs additional test-item expansion after class discovery.

Reproduction

Repository and pinned revision:

https://github.com/magefree/mage/tree/9b886598681fe0c9faf80d4db210b069e0979431

Dev container:

https://github.com/magefree/mage/blob/9b886598681fe0c9faf80d4db210b069e0979431/.devcontainer/xmage-dev-tools/devcontainer.json

At this revision, Mage.Tests/src/test contains 2,024 Java files. The reporter indicates that the project contains approximately 9,000 JUnit 4/JUnit 5 tests.

1. Open the repository root with the supplied Java 25 dev container.

2. Install Language Support for Java and Test Runner for Java in the container.

3. Run:

   mvn install -DskipTests

4. Wait for Java project import to complete.

5. From the Testing view, run all tests under the mage-tests project node.

The dev container does not pin the Java and Test Runner extension versions, so the actual versions used for profiling should be recorded with the measurements.

A thread dump can be collected during discovery with:

jstack $(pgrep -f equinox.launcher) > jdtls-test-discovery.txt

Code observation

The current JUnit5TestFinder.findTestsInContainer() implementation builds a type hierarchy and calls internalIsTest() for each type contained in the requested region.

For an accessible source type that is not handled by an earlier fast path, internalIsTest() creates an ASTParser, enables binding resolution, and calls createAST() for the type's compilation unit. Because the outer loop operates on IType, multiple types from the same compilation unit may cause separate parser invocations.

The supplied thread dumps are consistent with this path, although additional timing or profiling is needed to quantify its share of the total discovery time. The sampled threads are RUNNABLE; the dumps do not show a deadlock.

JUnit5TestFinder is provided by Eclipse JDT's JUnit core bundle and consumed by JDT LS. This issue tracks the behavior observed through JDT LS; any finder-level change may ultimately belong in eclipse-jdt/eclipse.jdt.ui.

Related discussions

• [JUnit] JUnit4TestFinder large project performance eclipse-jdt/eclipse.jdt.ui#302 discussed binding AST cost in the JUnit test finders.
• Make some performance enhancements to test finders eclipse-jdt/eclipse.jdt.ui#309 moved the region check before internalIsTest().
• Run As/Debug As context menu in java source editor very slow eclipse-jdt/eclipse.jdt.ui#1056 discusses a possible Java Model-based fast path and asks for a reproducible project for performance measurement.

Possible direction

Could a positive-only Java Model or index-based fast path reduce the number of binding-resolved AST parses?

Such a fast path should return early only when a test can be identified conclusively, while keeping the current binding-based implementation as the fallback. It must preserve custom @Testable meta-annotations, inherited tests, @Nested, suites, and JUnit 3/4 compatibility.

The existing annotation-resolution logic in TestSearchEngine, including IType.resolveType() and recursive @Testable lookup, may be reusable. This is only a possible direction; measurements with the reproduction project are needed before selecting an implementation.