Skip to content

Commit 3d71c06

Browse files
committed
flashinfer: head_dim -> head_dim_qk
1 parent e893362 commit 3d71c06

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

server/text_generation_server/layers/attention/flashinfer.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ def use_prefill_with_paged_kv_state(
9090
paged_kv_last_page_len=last_page_len,
9191
num_qo_heads=num_heads,
9292
num_kv_heads=num_kv_heads,
93-
head_dim=head_size,
93+
head_dim_qk=head_size,
9494
kv_data_type=kv_dtype,
9595
q_data_type=q_dtype,
9696
page_size=page_size,

0 commit comments

Comments
 (0)