Skip to content

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented May 12, 2025

Purpose

  • Make actorder option more intuitive for users
  • Enable easier adjustment of actorder default [GPTQ] Change actorder default to "static" #1425
  • This change is conceptually intuitive because activation ordering is a concept that only applies to the GPTQ algorithm (the only algorithm for which quantization group order matters)

Changes

  • Add actorder argument to GPTQModifier
  • Override resolve_quantization_config method to resolve config groups with actorder argument
  • (Misc) rearrange method order to match the typical order in which they are called in the modifier lifecycle

Testing

  • Ran llama w4a16 example to completion

Signed-off-by: Kyle Sayers <[email protected]>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@kylesayrs kylesayrs added the ready When a PR is ready for review label May 12, 2025
@kylesayrs kylesayrs changed the title [GPTQ] Add actorder option to modifier [GPTQ] Add actorder option to modifier May 12, 2025
Copy link
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, pending clarification on one question

@kylesayrs kylesayrs enabled auto-merge (squash) May 16, 2025 16:18
@kylesayrs kylesayrs merged commit 4b969af into main May 19, 2025
11 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/gptq-actorder branch May 19, 2025 16:39
aireilly pushed a commit to aireilly/llm-compressor that referenced this pull request Jul 30, 2025
## Purpose ##
* Make actorder option more intuitive for users
* Enable easier adjustment of actorder default vllm-project#1425
* This change is conceptually intuitive because activation ordering is a
concept that only applies to the GPTQ algorithm (the only algorithm for
which quantization group order matters)

## Changes ##
* Add `actorder` argument to `GPTQModifier`
* Override `resolve_quantization_config` method to resolve config groups
with `actorder` argument
* (Misc) rearrange method order to match the typical order in which they
are called in the modifier lifecycle

## Testing ##
* Ran llama w4a16 example to completion

Signed-off-by: Kyle Sayers <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants