Skip to content

README.md

Latest commit

 

History

History
39 lines (27 loc) · 1.18 KB

README.md

File metadata and controls

39 lines (27 loc) · 1.18 KB
Raw

FastV + Impromptu-VLA Reproduction

This repository documents my reproduction and analysis of FastV-style attention-based image token pruning, and its applicability to Impromptu-VLA (Qwen2.5-VL) under different inference backends.

What is included

  • ✅ FastV reproduced under HuggingFace / LLaVA path
  • ✅ Impromptu-VLA baseline inference (Qwen2.5-VL, sglang)
  • ✅ DivPrune-style static image token truncation
  • ❌ True FastV-style in-place pruning under sglang (analyzed)

Key findings (TL;DR)

  • FastV requires decoder-level attention access and dynamic sequence length
  • sglang pre-allocates KV cache slots, preventing mid-forward token pruning
  • DivPrune works because it truncates tokens at preprocessing stage
  • True KV-dropping FastV is infeasible in sglang without kernel-level changes

Repository Structure

fastv-impromptu-vla/ ├── README.md ├── REPRODUCTION.md ├── ANALYSIS.md ├── scripts/ ├── src/ └── configs/

How to run

See scripts/ for runnable examples.

References