[Bugfix] Validate custom logits processor xargs for online serving

[Isotr0py]

Purpose

• The default model_type for deepseek-ocr is deepseek_vl_v2, so the default deepseek-ocr chat template won't be used by default:

  vllm/vllm/transformers_utils/chat_templates/registry.py

  Line 36 in a806c14

  "deepseek_ocr": CHAT_TEMPLATES_DIR / "template_deepseek_ocr.jinja",

• Also allow list of int as vllm_xargs for ChatCompletionRequest, otherwise "whitelist_token_ids": [128821, 128822], parameters for deepseek-ocr's custom logits processor is not allowed.

openai.BadRequestError: Error code: 400 - {'error': {'message': "[{'type': 'string_type', 'loc': ('body', 'vllm_xargs', 'whitelist_token_ids', 'str'), 'msg': 'Input should be a valid string', 'input': [128821, 128822]}, {'type': 'int_type', 'loc': ('body', 'vllm_xargs', 'whitelist_token_ids', 'int'), 'msg': 'Input should be a valid integer', 'input': [128821, 128822]}, {'type': 'float_type', 'loc': ('body', 'vllm_xargs', 'whitelist_token_ids', 'float'), 'msg': 'Input should be a valid number', 'input': [128821, 128822]}]", 'type': 'Bad Request', 'param': None, 'code': 400}}

• Also validate request's vllm_xargs to avoid unexpected behavior from custom logits processor.

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request addresses a bug in the DeepSeek-OCR chat template fallback and allows a list of integers as vllm_xargs for ChatCompletionRequest to accommodate custom logits processors. The code changes modify vllm/entrypoints/openai/protocol.py to allow lists in vllm_xargs and vllm/transformers_utils/configs/deepseek_vl2.py to correctly set the model_type for DeepSeek OCR models. No style guide was provided, and the changes appear consistent with common Python practices.

[gemini-code-assist]

The condition if "DeepseekOCRForCausalLM" in (self.architectures or kwargs.get("architectures", [])) could potentially be simplified by directly checking if "DeepseekOCRForCausalLM" in self.architectures + kwargs.get("architectures", []). This avoids the need for the or operator and might improve readability.

However, it's crucial to ensure that this change doesn't alter the behavior of the code, especially if self.architectures or kwargs.get("architectures", []) could be None or not a list. Adding a check to ensure that these are lists before concatenation could mitigate this risk.

Suggested change

	# update model_type for OCR model
	if "DeepseekOCRForCausalLM" in (
	self.architectures or kwargs.get("architectures", [])
	):
	self.model_type = "deepseek_ocr"
	# update model_type for OCR model
	architectures = self.architectures if self.architectures else []
	architectures += kwargs.get("architectures", []) if kwargs.get("architectures", []) else []
	if "DeepseekOCRForCausalLM" in architectures:
	self.model_type = "deepseek_ocr"

[DarkLight1337]

We are going to patch a security vulnerability before merging this

[mergify]

Documentation preview: https://vllm--27560.org.readthedocs.build/en/27560/

[njhill]

cc @afeldman-nm I think we discussed this at one point

[njhill]

Thanks @Isotr0py! I would also like to review this

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Isotr0py.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[afeldman-nm]

Thanks for this @Isotr0py ! It will be good to have argument validation

[afeldman-nm]

It is great that you updated the documentation to reflect these changes.

Correct me if wrong, but it appears like this PR right now is only updating the documentation for the special case of adapting a request-level logits processor.

I think we will probably also want to update the documentation for

1. The "Programming Model" section of the logits processors design docs https://docs.vllm.ai/en/latest/design/logits_processors.html#logits-processor-programming-model

2. Other sections of the custom logits processor design docs, specifically "Creating a custom logits processor" (https://docs.vllm.ai/en/latest/features/custom_logitsprocs.html#creating-a-custom-logits-processor), "Passing Custom Argument to a Custom Logits Procesor" (https://docs.vllm.ai/en/latest/features/custom_logitsprocs.html#passing-custom-argument-to-a-custom-logits-processor), "Example custom logits processor implementation" (https://docs.vllm.ai/en/latest/features/custom_logitsprocs.html#example-custom-logits-processor-implementation)

[Isotr0py]

Sorry for the delayed update! I was a bit occupied last week. 😅 This PR should be ready for a further review now!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[DarkLight1337]

Thanks, LGTM

[DarkLight1337]

/gemini review

Just in case

[gemini-code-assist]

Code Review

This pull request introduces a robust validation mechanism for custom logits processor arguments, especially for online serving, by adding a validate_params method to the LogitsProcessor interface. This is a great improvement to prevent invalid parameters from causing issues downstream. The changes to allow list values in vllm_xargs and the fix for the deepseek-ocr model type are also valuable. The documentation and tests have been updated accordingly. I have one suggestion to refactor some duplicated code for better maintainability.