[feat] Support EAGLE for Qwen2

[Ximingwang-09]

Purpose

new PR for #21363

Test Plan

#21363

Test Result

#21363

(Optional) Documentation Update

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request adds support for EAGLE for Qwen2 models. The implementation in vllm/model_executor/models/qwen2_eagle.py has a couple of critical issues. The __init__ method of EagleQwen2ForCausalLMEagle is incomplete, missing initialization for lm_head and lora_config, which will lead to runtime errors. Additionally, the load_weights method does not return the set of loaded weights, which breaks the contract of the weight loading interface in vLLM. I've provided suggestions to fix these issues.

[gemini-code-assist]

The __init__ method is incomplete. It inherits from Qwen2ForCausalLM but doesn't initialize all necessary attributes. Specifically, self.lm_head is not initialized, which is used by the inherited compute_logits method, and self.lora_config is not set, which is required by the SupportsLoRA interface. This will lead to runtime errors.

    def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
        nn.Module.__init__(self)
        self.config = vllm_config.speculative_config.draft_model_config.hf_config
        self.lora_config = vllm_config.lora_config
        self.quant_config = vllm_config.quant_config

        target_layer_num = vllm_config.model_config.get_num_layers(
            vllm_config.parallel_config)
        self.model = Qwen2Model(vllm_config=vllm_config,
                                prefix="model",
                                start_layer_id=target_layer_num)

        from vllm.distributed import get_pp_group
        from vllm.model_executor.layers.vocab_parallel_embedding import (
            ParallelLMHead)
        from vllm.model_executor.models.utils import (PPMissingLayer,
                                                      maybe_prefix)
        if get_pp_group().is_last_rank:
            self.lm_head = ParallelLMHead(
                self.config.vocab_size,
                self.config.hidden_size,
                quant_config=self.quant_config,
                prefix=maybe_prefix(prefix, "lm_head"))
        else:
            self.lm_head = PPMissingLayer()

        logit_scale = getattr(self.config, "logit_scale", 1.0)
        self.logits_processor = LogitsProcessor(self.config.vocab_size,
                                                scale=logit_scale)

[DarkLight1337]

Thanks for updating the PR, can you make use of the newly-introduced SupportsEagle3 interface?

[Ximingwang-09]

Thanks for updating the PR, can you make use of the newly-introduced SupportsEagle3 interface?

Sure

[DarkLight1337]

LGTM, thanks

[DarkLight1337]

PTAL at the failing model test. By the way, Eagle3 support is going to be added soon (#23337), so not sure whether this PR is still needed