[Bug]: prefix cache bug happens when use w4a16 for GLM5

[terenceucla]

Your current environment

when using w4a16 for GLM5 Prefix cache hit rate: 0.0%

(APIServer pid=1) INFO 03-09 02:05:27 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 2356.1 tokens/s, Avg generation throughput: 11.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.7%, Prefix cache hit rate: 0.0%

🐛 Describe the bug

(APIServer pid=1) INFO 03-09 02:05:27 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 2356.1 tokens/s, Avg generation throughput: 11.5 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 45.7%, Prefix cache hit rate: 0.0%

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[flutist]

could you share your script or command to reproduce it

[terenceucla]

could you share your script or command to reproduce it
i am using GLM5 awq model
--model
/data/llm/GLM5-awq-exp1
--served-model-name
GLM5-awq
--enable-prefix-caching
--tool-call-parser
glm47
--tensor-parallel-size
8
--swap-space
16
--max-num-seqs
32
--max-model-len
32768
--gpu-memory-utilization
0.8
--enable-auto-tool-choice
--reasoning-parser
glm45
--trust-remote-code
--host
0.0.0.0
--port
8090