Skip to content

Fix DeepSeek-V4 Native history across prefill chunks - #30

Open
remer wants to merge 1 commit into
brontoguana:mainfrom
remer:fix/deepseek-v4-native-history
Open

Fix DeepSeek-V4 Native history across prefill chunks#30
remer wants to merge 1 commit into
brontoguana:mainfrom
remer:fix/deepseek-v4-native-history

Conversation

@remer

@remer remer commented Aug 21, 2026

Copy link
Copy Markdown

Closes #29

Summary

  • restore each DeepSeek-V4 Native compressor's packed prefix before appending to the shared BF16 history scratch
  • exercise Native FP4 pack/unpack at a high destination-row offset
  • compare Native learned-index scores bit-for-bit with the BF16 scorer using the same expanded codes

Root cause

The request-scoped Native history allocation is shared across layers and across the main and learned-index compressors. It therefore cannot preserve one compressor's prefix across intervening owners or later prefill chunks. The previous active_prefill_chunk_idx == 0 guard skipped the required restore after chunk 0, allowing learned-index selection to consume mixed history.

The packed Native cache remains the persistent source of truth. This change restores its prefix whenever first_group > 0 before appending new rows.

Validation

  • exact 59,168-token all-Native retrieval test: 4/10 before, 10/10 after
  • 103,349-token three-chunk retrieval test: 2/2
  • high-offset Native FP4 pack/unpack: bit-exact at destination row 14,792
  • Native learned-index scorer versus BF16 scorer over identical expanded codes: bit-exact
  • git diff --check

The source-build compatibility change used on the validation host is intentionally excluded from this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DeepSeek-V4 Native learned-index history is not restored after prefill chunk 0

1 participant