Merging main changes as of 20250410 #15

Alexei-V-Ivanov-AMD · 2025-04-10T15:02:40Z

No description provided.

github-actions · 2025-04-10T15:02:53Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

…29162) Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

Signed-off-by: Mark McLoughlin <[email protected]> Co-authored-by: Nicolò Lucchesi <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

…m_speculative_tokens` (#29102) Signed-off-by: Lucas Wilkinson <[email protected]>

Signed-off-by: angelayi <[email protected]>

Signed-off-by: charlifu <[email protected]>

Signed-off-by: yewentao256 <[email protected]>

Signed-off-by: Lukas Geiger <[email protected]>

Signed-off-by: Ryan Rock <[email protected]>

Signed-off-by: Livinfly <[email protected]>

… platform (#29204) Signed-off-by: qli88 <[email protected]>

Signed-off-by: FlintyLemming <[email protected]>

…without error on ROCm (#29212) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

…#29069) Signed-off-by: ApostaC <[email protected]>

…l tests (#29149) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

Signed-off-by: Yanan Cao <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: Andrew Xia <[email protected]> Co-authored-by: Andrew Xia <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

…rt for non-harmony requests (#29175) Signed-off-by: Mads Kildegård <[email protected]>

Signed-off-by: Jane Xu <[email protected]>

…ion.py (#29228) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: nandan2003 <[email protected]> Signed-off-by: Nandan Vallamdasu <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: yihong0618 <[email protected]>

…28832) Signed-off-by: Bram Wasti <[email protected]> Signed-off-by: Bram Wasti <[email protected]> Co-authored-by: Wentao Ye <[email protected]>

#29084) Signed-off-by: NickLucche <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

Signed-off-by: Yanan Cao <[email protected]> Co-authored-by: Claude <[email protected]>

Signed-off-by: Benjamin Chislett <[email protected]> Signed-off-by: Benjamin Chislett <[email protected]>

#29241) Signed-off-by: Ralf Gommers <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]> Signed-off-by: Cyrus Leung <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: zjy0516 <[email protected]> Co-authored-by: Roger Wang <[email protected]>

#29697) Signed-off-by: Huamin Li <[email protected]>

Signed-off-by: augusto.yjh <[email protected]>

Signed-off-by: zhxchen17 <[email protected]>

Signed-off-by: mertunsall <[email protected]> Co-authored-by: Roger Wang <[email protected]>

Signed-off-by: Andreas Karatzas <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: Tsukasa OI <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

…rable (#29698) Signed-off-by: Lucas Wilkinson <[email protected]>

Signed-off-by: angelayi <[email protected]> Co-authored-by: Luka Govedič <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: Didier Durand <[email protected]>

…29720) Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: dublc <[email protected]>

Signed-off-by: Chukwuma Nwaugha <[email protected]>

Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: Jinzhen Lin <[email protected]> Signed-off-by: Michael Goin <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: Michael Goin <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]>

Signed-off-by: Xin Yang <[email protected]> Signed-off-by: Xin Yang <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

varun-sundar-rabindranath and others added 29 commits November 21, 2025 14:28

[BugFix] EPLB + B200 + DeepGEMM : Handle column-major scales tensor (#…

3137991

…29162) Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>

[KV Connector] Fix async connector prefix cache metrics (#28585)

c6fa389

Signed-off-by: Mark McLoughlin <[email protected]> Co-authored-by: Nicolò Lucchesi <[email protected]>

[Model Runner V2] Optimize Gumbel Sampling Kernel (#29210)

e9af6ba

Signed-off-by: Woosuk Kwon <[email protected]>

[BugFix] Fix Eagle IndexError: list index out of range for even `nu…

30d6466

…m_speculative_tokens` (#29102) Signed-off-by: Lucas Wilkinson <[email protected]>

[docs] Fix cudagraph mode config (#29170)

d5dbdbf

Signed-off-by: angelayi <[email protected]>

[Rocm][CI] Fix DeekSeek V2-Lite Accuracy CI (#29135)

9a3101b

Signed-off-by: charlifu <[email protected]>

[CI] Bug: Fix triton import issue (#29202)

1d34eb1

Signed-off-by: yewentao256 <[email protected]>

[Model][Qwen3VL] Tune Triton w8a8 block fp8 kernel for L40s (#29217)

d045e22

Signed-off-by: Lukas Geiger <[email protected]>

[CI/Build] Add terratorch for AMD (#29205)

ed8e684

Signed-off-by: Ryan Rock <[email protected]>

[Bugfix] Fix block size in block_table with PCP (#29094)

5c8f2ad

Signed-off-by: Livinfly <[email protected]>

[CI/Build] Disable test_gptoss_tp.py in 'LoRA TP Test' group for ROCm…

1ef9c9e

… platform (#29204) Signed-off-by: qli88 <[email protected]>

Add fused MoE config for H200 E160 N192 fp8 (#29182)

052950e

Signed-off-by: FlintyLemming <[email protected]>

[CI/Build][AMD] Enable Entrypoints Integration Test (Pooling) to run …

6f40350

…without error on ROCm (#29212) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

[chore][LMCache connector] Remove useless logs from lmcache connector (…

77e1c03

…#29069) Signed-off-by: ApostaC <[email protected]>

[CI/Build] Only use supported types and features on ROCm in MoE kerne…

fd65015

…l tests (#29149) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

[Bugfix]Fix a conditional to not check zero value (#28754)

933f67e

Signed-off-by: Yanan Cao <[email protected]>

[LoRA] Cleanup FusedMoEWithLoRA (#29187)

1489902

Signed-off-by: Jee Jee Li <[email protected]>

[Model Runner V2] Limit cudagraph size to max decode batch size (#29221)

e905605

Signed-off-by: Woosuk Kwon <[email protected]>

[responsesAPI] parse reasoning item input (#28248)

742e9ff

Signed-off-by: Andrew Xia <[email protected]> Co-authored-by: Andrew Xia <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

[Frontend][Responses API] Multi-turn (with type: "output_text") suppo…

ea38474

…rt for non-harmony requests (#29175) Signed-off-by: Mads Kildegård <[email protected]>

Handle triton kernel import exception (#29062)

988ee66

Simplify from_blob usage in get_cuda_view_from_cpu_tensor (#29027)

e6309ac

Signed-off-by: Jane Xu <[email protected]>

[CI/Build] Skip tests that require libcudart in test_lmcache_integrat…

a4fdf24

…ion.py (#29228) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

[CI/Build Don't add FLASHINFER backend in test_cpu_offloading.py (#29229

8e22da1

) Signed-off-by: Randall Smith <[email protected]> Co-authored-by: Randall Smith <[email protected]>

[Misc] Further clean up chunked prefill and prefix caching init (#29186)

5a48025

Signed-off-by: DarkLight1337 <[email protected]>

fix: clean up function never use in setup.py (#29061)

2d4978a

Signed-off-by: yihong0618 <[email protected]>

[tiny] Remove unsupported TRITON_MLA backend from batch invariance (#…

5f7209a

…28832) Signed-off-by: Bram Wasti <[email protected]> Signed-off-by: Bram Wasti <[email protected]> Co-authored-by: Wentao Ye <[email protected]>

[Attention] Refactor FA block_size limitations to hybrid models only (

066209a

#29084) Signed-off-by: NickLucche <[email protected]>

hmellor and others added 30 commits November 28, 2025 20:26

Remove all_special_tokens_extended from tokenizer code (#29686)

fecae12

Signed-off-by: Harry Mellor <[email protected]>

[Frontend] Remap -O to -cc commandline flag (#29557)

3461e7e

Signed-off-by: Yanan Cao <[email protected]> Co-authored-by: Claude <[email protected]>

[Perf] Optimize EAGLE prepare_inputs_padded with triton kernels (#28597)

1986de1

Signed-off-by: Benjamin Chislett <[email protected]> Signed-off-by: Benjamin Chislett <[email protected]>

[CI/Build]: make it possible to build with a free-threaded interpreter (

7c1ed45

#29241) Signed-off-by: Ralf Gommers <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

[Misc] Remove redundant ClassRegistry (#29681)

7675ba3

Signed-off-by: DarkLight1337 <[email protected]> Signed-off-by: Cyrus Leung <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[Bugfix] fix dots.llm1.inst (#29687)

a51f418

Signed-off-by: zjy0516 <[email protected]> Co-authored-by: Roger Wang <[email protected]>

Revert "[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#28971)" (

3fd1fb0

#29697) Signed-off-by: Huamin Li <[email protected]>

bugfix: correct attn output with base 2 or e (#28840)

9726e64

Signed-off-by: augusto.yjh <[email protected]>

[compile] Include enable_sleep_mode into caching factors. (#29696)

6173682

Signed-off-by: zhxchen17 <[email protected]>

[Bugfix] Fix O(n²) multimodal string prompt processing (#29667)

c625d7b

Signed-off-by: mertunsall <[email protected]> Co-authored-by: Roger Wang <[email protected]>

[ROCm][Bugfix] Patch for the Multi-Modal Processor Test group (#29702)

ea3370b

Signed-off-by: Andreas Karatzas <[email protected]>

[Model Runner V2] Support penalties using bin counts (#29703)

1dcafb3

Signed-off-by: Woosuk Kwon <[email protected]>

[Bugfix] Fix wrong mock attribute (#29704)

b2c50ed

Signed-off-by: DarkLight1337 <[email protected]>

[Frontend] Perform offline path replacement to tokenizer (#29706)

762a4a6

Signed-off-by: Tsukasa OI <[email protected]>

[Model Runner V2] Refactor prefill token preparation (#29712)

ca1b1e7

Signed-off-by: Woosuk Kwon <[email protected]>

[BugFix] Fix DBO failing with TypeError: 'NoneType' object is not ite…

e23f665

…rable (#29698) Signed-off-by: Lucas Wilkinson <[email protected]>

Add gpu memory wait before test_async_tp (#28893)

4b17ce6

Signed-off-by: angelayi <[email protected]> Co-authored-by: Luka Govedič <[email protected]>

[Model Runner V2] Don't use UVA buffer for prefill_len (#29713)

4a80ad0

Signed-off-by: Woosuk Kwon <[email protected]>

[LoRA] Cleanup LoRA unused code (#29611)

39e63de

Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

[Model Runner V2] Add sample/ directory and reorganize files (#29719)

6afc0ff

Signed-off-by: Woosuk Kwon <[email protected]>

[Doc]: fixing typos in various files. (#29717)

04a797c

Signed-off-by: Didier Durand <[email protected]>

[Model Runner V2] Fuse penalties and temperature into single kernel (#…

f223ed4

…29720) Signed-off-by: Woosuk Kwon <[email protected]>

[Misc] Refactor tokenizer interface (#29693)

34a9842

Signed-off-by: DarkLight1337 <[email protected]>

[Doc]: fix code block rendering (#29728)

f4341f4

Signed-off-by: dublc <[email protected]>

hfrunner.classify should return list[list[float]] not list[str] (#29671)

ad7f714

Signed-off-by: Chukwuma Nwaugha <[email protected]>

[Chore] Enable passing tokenizer=None into MM processor (#29724)

fe3398f

Signed-off-by: DarkLight1337 <[email protected]>

[Chore] Move detokenizer_utils to vllm/tokenizers (#29727)

fa59fe4

Signed-off-by: DarkLight1337 <[email protected]>

[Bugfix] Revert test_tokenization.py (#29729)

b9d0504

Signed-off-by: Jee Jee Li <[email protected]>

[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#29708)

a491b09

Signed-off-by: Xin Yang <[email protected]> Signed-off-by: Xin Yang <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merging main changes as of 20250410 #15

Merging main changes as of 20250410 #15

Uh oh!

Alexei-V-Ivanov-AMD commented Apr 10, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Apr 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

118 participants

Merging main changes as of 20250410 #15

Are you sure you want to change the base?

Merging main changes as of 20250410 #15

Uh oh!

Conversation

Alexei-V-Ivanov-AMD commented Apr 10, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

118 participants

Alexei-V-Ivanov-AMD commented Apr 10, 2025 •

edited by github-actions bot

Loading