Commit bbe97e5
committed
opencl: stop writing zeros into the padded MoE activation slots
kernel_moe_reorder_b zero-filled every padded token slot. The MoE GEMMs
accumulate per output column and scatter only the real columns, so whatever sits
in a padded slot never reaches dst and the fill is pure bandwidth.
Established by a poison test rather than by reading the closed-source kernel:
filling padded slots with 1e30 leaves MUL_MAT_ID at 383 OK / 0 FAIL. The
positive control is what gives that meaning -- poisoning the real gather as well
fails 188 of 383 with NaN at index 0, and 188 is exactly the moe_reorder_b
dispatch count, so the buffer is read and the test can fail.
The destination is the reused prealloc_act_trans scratch, so padded slots now
retain stale activations from a previous layer rather than zeros. That is safe
for the same reason.
Worth about 5% of the kernel on granite-3b MXFP4. Perplexity is bit-identical on
three models, which covers the cross-layer stale-buffer case test-backend-ops
cannot reproduce.1 parent cda33c2 commit bbe97e5
1 file changed
Lines changed: 9 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
27 | 30 | | |
28 | 31 | | |
29 | | - | |
| 32 | + | |
| 33 | + | |
30 | 34 | | |
0 commit comments