Skip to content

[Deprecation][2/N] Replace --task with --runner and --convert #21470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 56 commits into from
Jul 28, 2025
Merged

[Deprecation][2/N] Replace --task with --runner and --convert #21470

merged 56 commits into from
Jul 28, 2025

Conversation

DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Jul 23, 2025 •
edited by github-actions bot
Loading

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Follow-up to #21227

--task (model_config.task) is now deprecated. Users are expected to pass --runner and/or --convert instead, according to the logic of https://github.com/vllm-project/vllm/pull/21470/files#diff-7eaad0b7dee0626bf29d10081b0f0c5e3ea15a4af97e7b182a4e0d35f8346953R610-R670

This PR also makes it a lot easier to add new converters without having to wrestle with model registry, since registry.is_pooling_model is now based on the un-converted architecture.

FIX #21514

cc @maxdebayser @noooop @22quinn

Test Plan

Added new tests to check the handling of passing custom --pooling.

The existing tests, especially those that are updated in this PR, should pass.

Test Result

(Optional) Documentation Update

Updated all documentation to stop referring to --task.

@DarkLight1337 DarkLight1337 requested a review from Isotr0py July 23, 2025 16:46
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@mergify mergify bot added documentation Improvements or additions to documentation frontend multi-modality Related to multi-modality (#4194) new-model Requests to new models qwen Related to Qwen models structured-output speculative-decoding v1 labels Jul 23, 2025
@mergify mergify bot added the tpu Related to Google TPUs label Jul 23, 2025
gemini-code-assist[bot]
gemini-code-assist bot reviewed Jul 23, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the model configuration, replacing the overloaded --task argument with --runner and --convert. The changes are consistently applied across documentation, examples, and tests. The review focuses on vllm/config.py, identifying leftover debug print statements and a potential regression in configuration validation. A validation check is suggested to address the potential regression.

@DarkLight1337
Update
9d48e9b 
Signed-off-by: DarkLight1337 <[email protected]>
DarkLight1337
DarkLight1337 commented Jul 23, 2025
View reviewed changes
vllm/config.py
Comment on lines -597 to -602
if self.runner_type in ("draft",
"generate") and self.task != "transcription":
self.truncation_side = "left"
else:
self.truncation_side = "right"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been moved to transformers_utils/tokenizer_group.py

@DarkLight1337
Update
e0e1f24 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Remove downstream usages of model_config.task
aff0874 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Copy link
Member Author

DarkLight1337 commented Jul 27, 2025 •
edited
Loading

Also cc @maxdebayser in case it is related to your PRs. Those failing tests are all V1 only

@DarkLight1337
Skip roberta seq cls for V1
1ecd6bd 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337 DarkLight1337 marked this pull request as ready for review July 27, 2025 09:39
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) July 27, 2025 09:39
@DarkLight1337
Fix transformers loading
8925fac 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Cleanup
cc999d3 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Don't load HF in registry test
11377a4 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Fix model impl
af6498a 
Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
Fix remaining test
741be47 
Signed-off-by: DarkLight1337 <[email protected]>
@vllm-bot vllm-bot merged commit 86ae693 into vllm-project:main Jul 28, 2025
8 of 15 checks passed
@DarkLight1337 DarkLight1337 deleted the rename-task branch July 28, 2025 02:43
This was referenced Jul 28, 2025
Merged
Closed
Merged
This was referenced Jul 28, 2025
Merged
Merged
@noooop noooop mentioned this pull request Jul 30, 2025
Closed
4 tasks
liuyumoye pushed a commit to liuyumoye/vllm that referenced this pull request Jul 31, 2025
@DarkLight1337 @hmellor
[Deprecation][2/N] Replace --task with --runner and --convert (v…
d5d9047 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
HsChen-sys pushed a commit to HsChen-sys/vllm that referenced this pull request Aug 1, 2025
@DarkLight1337 @hmellor
[Deprecation][2/N] Replace --task with --runner and --convert (v…
0931f17 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
wenscarl pushed a commit to wenscarl/vllm that referenced this pull request Aug 4, 2025
@DarkLight1337 @hmellor @wenscarl
[Deprecation][2/N] Replace --task with --runner and --convert (v…
c62a808 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Signed-off-by: shuw <[email protected]>
x22x22 pushed a commit to x22x22/vllm that referenced this pull request Aug 5, 2025
@DarkLight1337 @hmellor @x22x22
[Deprecation][2/N] Replace --task with --runner and --convert (v…
1b15af0 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Pradyun92 pushed a commit to Pradyun92/vllm that referenced this pull request Aug 6, 2025
@DarkLight1337 @hmellor
[Deprecation][2/N] Replace --task with --runner and --convert (v…
3b9819b 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025
@DarkLight1337 @hmellor @npanpaliya
[Deprecation][2/N] Replace --task with --runner and --convert (v…
9493587 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025
@DarkLight1337 @hmellor @jinzhen-lin
[Deprecation][2/N] Replace --task with --runner and --convert (v…
82d4b7a 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Signed-off-by: Jinzhen Lin <[email protected]>
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
@DarkLight1337 @hmellor @paulpak58
[Deprecation][2/N] Replace --task with --runner and --convert (v…
f062185 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Signed-off-by: Paul Pak <[email protected]>
taneem-ibrahim pushed a commit to taneem-ibrahim/vllm that referenced this pull request Aug 14, 2025
@DarkLight1337 @hmellor @taneem-ibrahim
[Deprecation][2/N] Replace --task with --runner and --convert (v…
d7a5f78 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
BoyuanFeng pushed a commit to BoyuanFeng/vllm that referenced this pull request Aug 14, 2025
@DarkLight1337 @hmellor @BoyuanFeng
[Deprecation][2/N] Replace --task with --runner and --convert (v…
e7ee597 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Signed-off-by: Boyuan Feng <[email protected]>
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
@DarkLight1337 @hmellor @diegocastanibm
[Deprecation][2/N] Replace --task with --runner and --convert (v…
beb2555 
…llm-project#21470)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Signed-off-by: Diego-Castan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hmellor hmellor hmellor left review comments

@jeejeelee jeejeelee jeejeelee approved these changes

@Isotr0py Isotr0py Isotr0py approved these changes

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

@russellb russellb Awaiting requested review from russellb

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth

@houseroad houseroad Awaiting requested review from houseroad

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@comaniac comaniac Awaiting requested review from comaniac comaniac is a code owner

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat alexm-redhat is a code owner

+4 more reviewers

@noooop noooop noooop left review comments

@22quinn 22quinn 22quinn approved these changes

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

@maxdebayser maxdebayser maxdebayser approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation frontend multi-modality Related to multi-modality (#4194) new-model Requests to new models qwen Related to Qwen models ready ONLY add when PR is ready to merge/full CI is needed speculative-decoding structured-output tpu Related to Google TPUs v1
Projects
Structured Output
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CI Failure]: Multi-model Models Test (Extended 1)
8 participants
@DarkLight1337 @noooop @maxdebayser @jeejeelee @hmellor @22quinn @Isotr0py @vllm-bot