Skip to content

[TEST]Autoround version test #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[TEST]Autoround version test #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bc97d48
update autoround version
yiliu30 Nov 21, 2025
19ab4f2
Merge branch 'main' into autoround-version
yiliu30 Nov 21, 2025
9ba113c
expose bs
yiliu30 Nov 24, 2025
646982a
Merge branch 'autoround-version' of https://github.com/yiliu30/llm-co…
yiliu30 Nov 24, 2025
1050335
use 0.9.1
yiliu30 Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,7 @@ def localversion_func(version: ScmVersion) -> str:
if BUILD_TYPE == "release"
else "compressed-tensors>=0.12.3a2"
),
# TODO: replace it with the release version
("auto_round @ git+https://github.com/intel/auto-round.git@llmc"),
("auto-round==0.9.1"),
],
extras_require={
"dev": [
Expand Down
4 changes: 3 additions & 1 deletion src/llmcompressor/modifiers/autoround/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ class AutoRoundModifier(Modifier, QuantizationMixin):
# AutoRound modifier arguments
iters: int = 200
enable_torch_compile: bool = True
batch_size: int = 8
Copy link

@gemini-code-assist gemini-code-assist bot Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The new batch_size parameter should be documented. Please consider adding it to the class docstring, explaining its purpose, and including it in the sample YAML configuration to guide users on how to configure it. This will improve the usability of the modifier.


# private variables
_module_names: Dict[torch.nn.Module, str] = PrivateAttr(default_factory=dict)
Expand Down Expand Up @@ -223,6 +224,7 @@ def apply_autoround(self, state, subgraph):
scheme=ar_quant_scheme,
iters=self.iters,
enable_torch_compile=self.enable_torch_compile,
batch_size=self.batch_size,
)
# TODO: configure layer-wise config based on self.resolved_config
ar.configure_layer_config(enable_gguf_official_mixed=False)
Expand All @@ -236,7 +238,7 @@ def apply_autoround(self, state, subgraph):
block=decoding_layer,
inputs=cur_inputs,
q_input=self._q_input,
device=device,
device=str(device),
Copy link

@gemini-code-assist gemini-code-assist bot Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This change to cast device to a string is a necessary fix. To improve code clarity for future maintainers, it would be helpful to add an inline comment explaining why this cast is needed, as the underlying auto-round library's expectation of a string is not immediately obvious.

Suggested change
device=str(device),
device=str(device), # auto-round expects device as a string

# Leave offload for LLMC
auto_offload=False,
)
Expand Down
Loading