[Frontend] Require flag for loading text and image embeds #27204
Conversation
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
…is not passed Signed-off-by: DarkLight1337 <[email protected]>
russellb
requested review from
DarkLight1337,
NickLucche,
ProExpertProg,
WoosukKwon,
aarnphm,
chaunceyjiang,
hmellor,
houseroad,
mgoin,
robertgshaw2-redhat,
simon-mo,
tlrmchlsmth,
yewentao256,
youkaichao and
ywang96
as code owners
|
Documentation preview: https://vllm--27204.org.readthedocs.build/en/27204/ |
mergify
bot
added
documentation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses security vulnerabilities GHSA-mrw7-hf4f-83pf and GHSA-pmqf-x6x8-p7qw by introducing --enable-prompt-embeds and --enable-mm-embeds flags. These flags gate the functionality of loading user-provided text and multimodal embeddings, which can be a security risk. The changes are well-implemented across the configuration, core logic, and entrypoints, with checks to ensure the flags are respected. The documentation has been updated with clear warnings, and new tests have been added to verify the behavior. The implementation appears correct and robust. I have not found any issues in this pull request.
github-actions
bot
added
the
ready
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
|
cc @christian-pinto |
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Russell Bryant <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
| `prompt_embeds` key. | ||
| WARNING: The vLLM engine may crash if incorrect shape of embeddings is passed. | ||
| Only enable this flag for trusted users!""" |
There was a problem hiding this comment.
Should a shape check be added to the renderer?
There was a problem hiding this comment.
The shape depends on the embedding size of each model so it requires a lot more effort to perform shape validation outside of the model class.
…ct#27204) Signed-off-by: DarkLight1337 <[email protected]> Co-authored-by: DarkLight1337 <[email protected]>
…ct#27204) Signed-off-by: DarkLight1337 <[email protected]> Co-authored-by: DarkLight1337 <[email protected]> Signed-off-by: Alberto Perdomo <[email protected]>
…ct#27204) Signed-off-by: DarkLight1337 <[email protected]> Co-authored-by: DarkLight1337 <[email protected]>
…ct#27204) Signed-off-by: DarkLight1337 <[email protected]> Co-authored-by: DarkLight1337 <[email protected]> Signed-off-by: 0xrushi <[email protected]>
…ct#27204) Signed-off-by: DarkLight1337 <[email protected]> Co-authored-by: DarkLight1337 <[email protected]> Signed-off-by: 0xrushi <[email protected]>
6 participants
https://github.com/vllm-project/vllm/security/advisories/GHSA-mrw7-hf4f-83pf
https://github.com/vllm-project/vllm/security/advisories/GHSA-pmqf-x6x8-p7qw
Issue #26928