Skip to content

Conversation

@ForBetterCodeNine
Copy link

@ForBetterCodeNine ForBetterCodeNine commented Oct 28, 2025

What this PR does / why we need it?

The existing tests lack coverage for the DeepSeek-V3-Lite network model under the GE Graph mode, so supplementary tests will be conducted.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds end-to-end tests for the DeepSeek-V3-Lite model in GE Graph mode. The implementation introduces a new test fixture that is largely a duplication of an existing one, which impacts maintainability. More critically, the new test appears to use incorrect golden data copied from another test, which undermines the validity of the test itself. I've provided two comments: one critical issue regarding the test data and a high-severity issue about code duplication.

Comment on lines 263 to 272
# NOTE: vllm-ascend/DeepSeek-V3-Pruning is a random weight of
# DeepSeek-V3 with 2 hidden layers, thus the golden results seems
# inaccurate. This will only change if accuracy improves with the
# official weights of DeepSeek-V3.
golden_results = [
'Hello, my name is下载早点向前很有่อง',
'The president of the United States isSender)## physiological Albany',
'The capital of France is Rocky转角 hospitalizedinterval sparked',
'The future of AI is её asegο BIOS一扫',
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The golden_results and the accompanying comment appear to be copied from the test for vllm-ascend/DeepSeek-V3-Pruning without being updated for the deepseek-ai/DeepSeek-V3-Lite model.

  1. Incorrect Comment: The comment at lines 263-266 refers to vllm-ascend/DeepSeek-V3-Pruning, which is not the model under test here. This is misleading.
  2. Suspicious Golden Results: The golden_results are identical to those for vllm-ascend/DeepSeek-V3-Pruning. It is extremely unlikely that two different models produce the exact same output. This strongly suggests the golden values are incorrect for deepseek-ai/DeepSeek-V3-Lite.

A test with incorrect golden data is critically flawed as it doesn't validate the model's behavior and can mask regressions. Please regenerate the golden_results using the actual output from deepseek-ai/DeepSeek-V3-Lite and update the comment to be relevant to this model, or remove it if the output is accurate.

Comment on lines 231 to 278
def _deepseek_v3_lite_torchair_test_fixure(
additional_config:Dict,
*,
tensor_parallel_size=2,
use_v1_schduler=False,
):
example_prompts = [
"Hello, my name is",
"The president of the United States is",
"The capital of France is",
"The future of AI is",
]

kwargs = {}
if not use_v1_schduler:
kwargs = {
"ascend_scheduler_config":{
"enable":True,
},
"refresh":True,
}
additional_config.update(**kwargs)

with VllmRunner(
"deepseek-ai/DeepSeek-V3-Lite",
dtype="half",
tensor_parallel_size=tensor_parallel_size,
distributed_executor_backend="mp",
additional_config=additional_config,
)as vllm_model:
vllm_output = vllm_model.generate_greedy(example_prompts, 5)

# NOTE: vllm-ascend/DeepSeek-V3-Pruning is a random weight of
# DeepSeek-V3 with 2 hidden layers, thus the golden results seems
# inaccurate. This will only change if accuracy improves with the
# official weights of DeepSeek-V3.
golden_results = [
'Hello, my name is下载早点向前很有่อง',
'The president of the United States isSender)## physiological Albany',
'The capital of France is Rocky转角 hospitalizedinterval sparked',
'The future of AI is её asegο BIOS一扫',
]

assert len(golden_results) == len(vllm_output)
for i in range(len(vllm_output)):
assert golden_results[i] == vllm_output[i][1]
print(f"Generated text:{vllm_output[i][1]!r}")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This new test fixture _deepseek_v3_lite_torchair_test_fixure is almost an exact copy of the existing _deepseek_torchair_test_fixture function (lines 32-79). The main difference is the model name passed to VllmRunner.

To improve maintainability and avoid code duplication, this should be refactored. You could parameterize the existing _deepseek_torchair_test_fixture to accept the model name and golden results as arguments. This would eliminate the need for the new _deepseek_v3_lite_torchair_test_fixure function entirely, making the code cleaner and easier to maintain.

For example:

def _deepseek_torchair_test_fixture(
    model_name: str,
    golden_results: list[str],
    additional_config: dict,
    # ... other params
):
    # ... existing logic ...
    with VllmRunner(
            model_name, # Use the new parameter
            # ...
    ) as vllm_model:
        vllm_output = vllm_model.generate_greedy(example_prompts, 5)

    # ... assertion logic using golden_results ...

# Then update call sites:
def test_e2e_deepseekv3_with_torchair():
    # ...
    _deepseek_torchair_test_fixture(
        "vllm-ascend/DeepSeek-V3-Pruning", 
        DEEPSEEK_V3_PRUNING_GOLDEN_RESULTS, 
        additional_config
    )

def test_e2e_deepseekv3lite_with_torchair():
    # ...
    _deepseek_torchair_test_fixture(
        "deepseek-ai/DeepSeek-V3-Lite", 
        DEEPSEEK_V3_LITE_GOLDEN_RESULTS, 
        additional_config
    )

Addressing the code duplication will also make it easier to manage model-specific details like the golden_results and explanatory comments.

Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
Signed-off-by: CodeNine-CJ <[email protected]>
@ForBetterCodeNine ForBetterCodeNine changed the title [Test] Test ge graph use DeepSeek-V3-Lite model [Test] Test ge graph use DeepSeek-V2-Lite model Oct 30, 2025
@wangxiyuan wangxiyuan added the ready read for review label Oct 31, 2025
Signed-off-by: CodeNine-CJ <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:tests ready read for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants