[XPU] Enable sequence parallel support for XPU#38608
[XPU] Enable sequence parallel support for XPU#38608chaojun-zhang wants to merge 1 commit intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request enables sequence parallelism tests on XPU platforms by updating pytest markers and generalizing device selection using current_platform.device_type and torch.accelerator. It also moves the SequenceParallelismPass import out of the CUDA-specific guard in the pass manager. Feedback indicates that moving this pass alone is insufficient, as RMSNormQuantFusionPass remains guarded but is required by the newly enabled XPU tests, which will likely result in a NameError.
| RocmAiterTritonAddRMSNormPadFusionPass, | ||
| ) | ||
|
|
||
| from .fusion.sequence_parallelism import SequenceParallelismPass |
There was a problem hiding this comment.
Moving SequenceParallelismPass out of the is_cuda_alike() guard is necessary to support it on XPU. However, RMSNormQuantFusionPass (currently at line 32) remains inside the guard. Since the XPU tests added in this PR (tests/compile/passes/distributed/test_sequence_parallelism.py) explicitly enable fuse_norm_quant, the PostGradPassManager.configure() method will raise a NameError on XPU platforms when it attempts to instantiate RMSNormQuantFusionPass.
Additionally, AsyncTPPass (line 39) is guarded by is_cuda(), which will cause a similar NameError if fuse_gemm_comms is enabled on XPU. You should move RMSNormQuantFusionPass out of the guard as well, and ensure AsyncTPPass is handled safely for XPU.
8e50c2c to
0bcadcd
Compare
0bcadcd to
9b5336e
Compare
9b5336e to
aae4a27
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
|
@chaojun-zhang Any latency difference w/ and w/o this feature enabled? and also the case with asyncTP enabled. |
751f05a to
e31dea0
Compare
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
|
Test Plan
Test Result
UT :
pytest -s -v tests/compile/correctness_e2e/test_sequence_parallel.py
pytest -s -v pytest -s -v tests/compile/correctness_e2e/test_sequence_parallel.py
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.