[Doc] Visual Token Pruning #2861

peterchen-intel · 2025-10-18T12:16:35Z

Description

Document for Visual Token Pruning
Ticket: CVS-173220, CVS-170139

Implementation is in #2714
Doc build: https://github.com/openvinotoolkit/openvino.genai/actions/runs/18670224384?pr=2861

Signed-off-by: Chen, Peter <[email protected]>

Copilot

Pull Request Overview

Adds a new documentation page describing the Visual Token Pruning (CDPruner) feature for VLMs, including conceptual overview, configuration parameters, and a sample usage snippet.

Introduces pruning concepts and workflow.
Documents new GenerationConfig fields (pruning_ratio, relevance_weight) and their effects.
Provides a benchmark script usage example for measuring performance impact.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

site/docs/concepts/optimization-techniques/visual-token-pruning.md

Co-authored-by: Copilot <[email protected]>

site/docs/concepts/optimization-techniques/visual-token-pruning.md

Signed-off-by: Chen, Peter <[email protected]>

Copilot

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

site/docs/concepts/optimization-techniques/visual-token-pruning.md

peterchen-intel · 2025-10-19T01:14:00Z

@liangali

Co-authored-by: Copilot <[email protected]>

Wovchena · 2025-10-20T06:21:03Z

Build your docs at https://github.com/peterchen-intel/openvino.genai/actions/workflows/deploy_gh_pages.yml to see everything is fine. Add the link of the resulting docs to the PR description

rkazants · 2025-10-20T06:32:08Z

Should be merged only after #2714

Signed-off-by: Chen, Peter <[email protected]>

site/docs/concepts/optimization-techniques/visual-token-pruning.md

l-bat · 2025-10-20T11:20:58Z

site/docs/concepts/optimization-techniques/visual-token-pruning.md

+The visual token sequence extracted from the image encoder can be partitioned into:
+
+* Retained Tokens: Subset judged most relevant by dominance scoring.
+* Pruned Tokens: Dropped from future decoding (no longer participate in cross-attention or self-attention depending on architecture).
+
+Pruning is controlled by a ratio (percentage of tokens to remove) and a relevance weight scaling that influences importance estimation.


CDPruner operates on the sequence of visual token embeddings produced by the vision encoder before they are passed to the language model. Instead of forwarding all tokens, it selects a subset based on conditional diversity, combining token similarity and instruction relevance.

Token Partitioning The visual tokens are conceptually divided into:

Retained Tokens: A selected subset that provides diverse and instruction-relevant visual information.

Pruned Tokens: Tokens excluded from further processing because they contribute redundant or low-relevance information.

site/docs/concepts/optimization-techniques/visual-token-pruning.md

Co-authored-by: Liubov Talamanova <[email protected]>

Signed-off-by: Chen, Peter <[email protected]>

l-bat · 2025-10-21T09:26:49Z

site/docs/concepts/optimization-techniques/visual-token-pruning.md

+1. Encode image producing N visual tokens (embeddings).
+2. Compute pairwise token similarity and per-token relevance scores.
+3. Relevance and similarity are combined into a conditional kernel. A greedy DPP-based MAP algorithm identifies the least important tokens to discard according to `pruning_ratio`, adjusting scores using `relevance_weight` to control the trade-off between diversity and relevance.
+4. Optionally adjust scores using `relevance_weight` before selecting final kept set.


This step is already incorporated in the previous step.

site/docs/concepts/optimization-techniques/visual-token-pruning.md

peterchen-intel · 2025-10-22T03:17:40Z

Build your docs at https://github.com/peterchen-intel/openvino.genai/actions/workflows/deploy_gh_pages.yml to see everything is fine. Add the link of the resulting docs to the PR description

@Wovchena Is following as expectation? I will remove the change in deploy_gh_pages.yml if it is OK.
https://github.com/openvinotoolkit/openvino.genai/actions/runs/18670224384?pr=2861 Download "github-pages" and view in local SimpleHTTPsServer.

Copilot

Pull Request Overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

site/docs/concepts/optimization-techniques/visual-token-pruning.md

Co-authored-by: Copilot <[email protected]>

[Doc] Visual Token Pruning

a3f6d32

Signed-off-by: Chen, Peter <[email protected]>

Copilot AI review requested due to automatic review settings October 18, 2025 12:16

github-actions bot added the category: GH Pages Docs Github Pages documentation label Oct 18, 2025

peterchen-intel requested review from l-bat and yangwang201911 October 18, 2025 12:17

Copilot AI reviewed Oct 18, 2025

View reviewed changes

peterchen-intel requested a review from xipingyan October 18, 2025 12:17

peterchen-intel assigned Wovchena Oct 18, 2025

Wording update per copolit suggestion

674f7ed

Co-authored-by: Copilot <[email protected]>

peterchen-intel mentioned this pull request Oct 18, 2025

Conditional visual token pruning for QWen-VL models. #2714

Open

peterchen-intel added this to the 2025.4 milestone Oct 18, 2025

peterchen-intel commented Oct 18, 2025

View reviewed changes

site/docs/concepts/optimization-techniques/visual-token-pruning.md Outdated Show resolved Hide resolved

Limit the model supported.

03fe7da

peterchen-intel commented Oct 18, 2025

View reviewed changes

site/docs/concepts/optimization-techniques/visual-token-pruning.md Outdated Show resolved Hide resolved

Update wording and conf code

ef02325

Signed-off-by: Chen, Peter <[email protected]>

peterchen-intel requested a review from Copilot October 19, 2025 01:13

Copilot AI reviewed Oct 19, 2025

View reviewed changes

site/docs/concepts/optimization-techniques/visual-token-pruning.md Outdated Show resolved Hide resolved

Update wording

664348e

Co-authored-by: Copilot <[email protected]>

rkazants added the do_not_merge label Oct 20, 2025

peterchen-intel added 2 commits October 20, 2025 14:37

Trigger build for pull_request

655d2a6

Signed-off-by: Chen, Peter <[email protected]>

Add read permission

92c84b8

Signed-off-by: Chen, Peter <[email protected]>

github-actions bot added the category: GHA CI based on Github actions label Oct 20, 2025

peterchen-intel had a problem deploying to github-pages October 20, 2025 07:30 — with GitHub Actions Failure

peterchen-intel had a problem deploying to github-pages October 20, 2025 07:52 — with GitHub Actions Failure

l-bat reviewed Oct 20, 2025

View reviewed changes

Update wording per review comments

fd3cf32

Co-authored-by: Liubov Talamanova <[email protected]>

peterchen-intel had a problem deploying to github-pages October 21, 2025 00:29 — with GitHub Actions Failure

Update per review comments

6393915

Signed-off-by: Chen, Peter <[email protected]>

peterchen-intel had a problem deploying to github-pages October 21, 2025 01:49 — with GitHub Actions Failure

yangwang201911 approved these changes Oct 21, 2025

View reviewed changes

l-bat reviewed Oct 21, 2025

View reviewed changes

peterchen-intel commented Oct 22, 2025

View reviewed changes

site/docs/concepts/optimization-techniques/visual-token-pruning.md Outdated Show resolved Hide resolved

Remove duplicate step

3141506

peterchen-intel had a problem deploying to github-pages October 22, 2025 02:58 — with GitHub Actions Failure

peterchen-intel requested review from Copilot and l-bat October 22, 2025 03:18

Copilot AI reviewed Oct 22, 2025

View reviewed changes

site/docs/concepts/optimization-techniques/visual-token-pruning.md Outdated Show resolved Hide resolved

l-bat approved these changes Oct 22, 2025

View reviewed changes

peterchen-intel had a problem deploying to github-pages October 22, 2025 08:56 — with GitHub Actions Failure

Update per review comment

dbc7a04

Co-authored-by: Copilot <[email protected]>

peterchen-intel had a problem deploying to github-pages October 22, 2025 13:58 — with GitHub Actions Failure

Uh oh!

[Doc] Visual Token Pruning #2861

Are you sure you want to change the base?

[Doc] Visual Token Pruning #2861

Conversation

peterchen-intel commented Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

peterchen-intel commented Oct 19, 2025

Uh oh!

Wovchena commented Oct 20, 2025

Uh oh!

rkazants commented Oct 20, 2025

Uh oh!

Uh oh!

Uh oh!

l-bat Oct 20, 2025

Choose a reason for hiding this comment

Token Partitioning The visual tokens are conceptually divided into:

Uh oh!

peterchen-intel Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

l-bat Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

peterchen-intel Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peterchen-intel commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

peterchen-intel commented Oct 18, 2025 •

edited

Loading

peterchen-intel commented Oct 22, 2025 •

edited

Loading