Skip to content

feat: implement Reduced Matrix Multiplication (arXiv:2608.13426) - #690

Open
erkinalp wants to merge 1 commit into
Scottcjn:mainfrom
erkinalp:devin/rmm-arxiv-2608-13426
Open

erkinalp wants to merge 1 commit into
Scottcjn:mainfrom
erkinalp:devin/rmm-arxiv-2608-13426

Conversation

@erkinalp

Copy link
Copy Markdown

Input-adaptive contraction-axis reduction for the products that dominate inference: Y = A B keeps only the ceil(rho*d) slices with the largest activation column norms ||A[:,j]||_2, chosen deterministically from the current activations. Training-free and weight-preserving.

  • ggml-rmm.h: column-norm scoring, quickselect TopK with lowest-index tie breaking, reduced GEMV/GEMM, projections, attention with QK^T feature reduction and optional PV token reduction, reusable workspace, MAC stats. Scalar C11 everywhere, VSX/AltiVec fast path on POWER8, dcbt prefetch of only the retained weight rows. rho = 1 is bit-identical to dense.
  • tests/rmm_test.c: TopK determinism and ties, dense equivalence at rho = 1, Proposition 1 error bound, activation-aware vs random selection, minimax claim, attention accuracy at rho_d = 0.5, MAC accounting, GEMV benchmark.
  • Makefile: 'make' runs the suite on x86-64, 'make power8' builds the VSX path.
  • docs/RMM.md, README and CHANGELOG entries.

Input-adaptive contraction-axis reduction for the products that dominate
inference: Y = A B keeps only the ceil(rho*d) slices with the largest
activation column norms ||A[:,j]||_2, chosen deterministically from the
current activations. Training-free and weight-preserving.

- ggml-rmm.h: column-norm scoring, quickselect TopK with lowest-index tie
  breaking, reduced GEMV/GEMM, projections, attention with QK^T feature
  reduction and optional PV token reduction, reusable workspace, MAC stats.
  Scalar C11 everywhere, VSX/AltiVec fast path on POWER8, dcbt prefetch of
  only the retained weight rows. rho = 1 is bit-identical to dense.
- tests/rmm_test.c: TopK determinism and ties, dense equivalence at rho = 1,
  Proposition 1 error bound, activation-aware vs random selection, minimax
  claim, attention accuracy at rho_d = 0.5, MAC accounting, GEMV benchmark.
- Makefile: 'make' runs the suite on x86-64, 'make power8' builds the VSX path.
- docs/RMM.md, README and CHANGELOG entries.

Co-Authored-By: Erkin Alp Güney <erkinalp9035@gmail.com>
@erkinalp
erkinalp requested a review from Scottcjn as a code owner August 19, 2026 06:26
@Scottcjn

Copy link
Copy Markdown
Owner

Received, and queued for review — apologies for the silence up to now.

This is a maintainer-side backlog rather than any verdict on your contribution. We
had a large number of open PRs with no reply on them at all, which is our failure to
manage, not a signal about your work. Yours is in the queue and will get a real
review with a real decision.

I'm not going to give you a date I might miss. What I can tell you:

  • Reviews run oldest-first.
  • If this PR has gone stale against main, say so and I'll prioritise closing it so
    the queue reflects what you actually still want landed.
  • If you're submitting this for bounty credit, link the bounty issue in a comment —
    several contributions are eligible and we'd rather pay you than have you wonder.

Thanks for your patience, and for the work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants