[tests] Improve speed and reliability of test_transcription_api_correctness #23854
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ctness Improve the performance of this test by only creating the tokenizer once instead of hundreds of times + serialized due to doing it while holding a semaphore. The previous code would also frequently get rate limited by HuggingFace from requesting https://huggingface.co/openai/whisper-large-v3/resolve/main/tokenizer_config.json too many times. This would sometimes cause the test to fail. On my laptop, here is the time difference: Before: - 5m3.389s After: - 2m5.471s This is a piece split out from vllm-project#21088. Signed-off-by: Russell Bryant <[email protected]>
russellb
requested review from
DarkLight1337,
robertgshaw2-redhat,
simon-mo and
aarnphm
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request provides a significant and well-executed optimization for the test_transcription_api_correctness test. By initializing the tokenizer once outside the main processing loop, it correctly addresses a performance bottleneck and a source of test flakiness caused by repeated network requests to HuggingFace. The code changes are clear, logical, and effectively improve both the speed and reliability of the test suite. No issues were found in the implementation.
3 tasks
DarkLight1337
added
the
ready
nopperl
pushed a commit
to pfnet/vllm
that referenced
this pull request
Sep 3, 2025
…ctness (vllm-project#23854) Signed-off-by: Russell Bryant <[email protected]>
MatthewBonanni
pushed a commit
to MatthewBonanni/vllm
that referenced
this pull request
Sep 3, 2025
…ctness (vllm-project#23854) Signed-off-by: Russell Bryant <[email protected]> Signed-off-by: Matthew Bonanni <[email protected]>
MatthewBonanni
pushed a commit
to MatthewBonanni/vllm
that referenced
this pull request
Sep 3, 2025
…ctness (vllm-project#23854) Signed-off-by: Russell Bryant <[email protected]>
842974287
pushed a commit
to 842974287/vllm
that referenced
this pull request
Sep 3, 2025
…ctness (vllm-project#23854) Signed-off-by: Russell Bryant <[email protected]> Signed-off-by: Shiyan Deng <[email protected]>
lengrongfu
pushed a commit
to lengrongfu/vllm
that referenced
this pull request
Sep 4, 2025
…ctness (vllm-project#23854) Signed-off-by: Russell Bryant <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improve the performance of this test by only creating the tokenizer
once instead of hundreds of times + serialized due to doing it while
holding a semaphore.
The previous code would also frequently get rate limited by
HuggingFace from requesting https://huggingface.co/openai/whisper-large-v3/resolve/main/tokenizer_config.json
too many times. This would sometimes cause the test to fail.
On my laptop, here is the time difference:
Before:
After:
This is a piece split out from #21088.
Signed-off-by: Russell Bryant [email protected]