[Bugfix][V1][P/D]Fix the uneven polling issue in the toy proxy for P2pNcclConnector #21819
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Abatom <[email protected]>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request correctly fixes an uneven polling bug in the example proxy server by changing dict.pop() to dict.get(). This prevents reordering of registered instances and ensures fair round-robin selection. The change is effective and well-targeted.
However, I've identified a critical issue in the surrounding code that could lead to an UnboundLocalError and crash the listener thread if an unexpected message type is received. I've left a detailed comment on this. Addressing this would significantly improve the robustness of the example.
examples/online_serving/disaggregated_serving_p2p_nccl_xpyd/disagg_proxy_p2p_nccl_xpyd.py
Show resolved
Hide resolved
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
Signed-off-by: Abatom <[email protected]>
Signed-off-by: Abatom <[email protected]>
|
@chaunceyjiang Could you help me review this PR? |
|
OK /assign |
chaunceyjiang
approved these changes
Aug 4, 2025
|
@DarkLight1337 PTAL. |
KuntaiDu
approved these changes
Aug 4, 2025
github-actions
bot
added
the
ready
juuice-lee
pushed a commit
to juuice-lee/vllm-moe.code
that referenced
this pull request
Aug 5, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
x22x22
pushed a commit
to x22x22/vllm
that referenced
this pull request
Aug 5, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
x22x22
pushed a commit
to x22x22/vllm
that referenced
this pull request
Aug 5, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: x22x22 <[email protected]>
x22x22
pushed a commit
to x22x22/vllm
that referenced
this pull request
Aug 5, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: x22x22 <[email protected]>
npanpaliya
pushed a commit
to odh-on-pz/vllm-upstream
that referenced
this pull request
Aug 6, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
myselvess
pushed a commit
to myselvess/vllm
that referenced
this pull request
Aug 7, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
jingyu-ml
pushed a commit
to jingyu-ml/vllm
that referenced
this pull request
Aug 8, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: jingyu <[email protected]>
jinzhen-lin
pushed a commit
to jinzhen-lin/vllm
that referenced
this pull request
Aug 9, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>
noamgat
pushed a commit
to noamgat/vllm
that referenced
this pull request
Aug 9, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Noam Gat <[email protected]>
yyihuang
pushed a commit
to yyihuang/vllm
that referenced
this pull request
Aug 11, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Avery Yingyi Huang <[email protected]>
paulpak58
pushed a commit
to paulpak58/vllm
that referenced
this pull request
Aug 13, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Paul Pak <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 13, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
taneem-ibrahim
pushed a commit
to taneem-ibrahim/vllm
that referenced
this pull request
Aug 14, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
BoyuanFeng
pushed a commit
to BoyuanFeng/vllm
that referenced
this pull request
Aug 14, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Boyuan Feng <[email protected]>
diegocastanibm
pushed a commit
to diegocastanibm/vllm
that referenced
this pull request
Aug 15, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Diego-Castan <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 26, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 27, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af.
epwalsh
pushed a commit
to epwalsh/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af. Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af. Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af.
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af.
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af. Signed-off-by: Xiao Yu <[email protected]>
zhewenl
pushed a commit
to zhewenl/vllm
that referenced
this pull request
Aug 28, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
xiao-llm
pushed a commit
to xiao-llm/vllm
that referenced
this pull request
Aug 28, 2025
…y for P2pNcclConnector (vllm-project#21819)" This reverts commit 9f518af. Signed-off-by: Xiao Yu <[email protected]>
googlercolin
pushed a commit
to googlercolin/vllm
that referenced
this pull request
Aug 29, 2025
…pNcclConnector (vllm-project#21819) Signed-off-by: Abatom <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using
popalters the original P/D instance order in the dictionary, and the round-robin selection of 1P1D relies on that order, which leads to uneven polling. After replacingpopwithget, the unevenness issue has been fixed.