Add Intel AutoRound algorithm support

[yiliu30]

Resolve #1968

Highlights

• Introduced AutoRoundModifier to enable AutoRound quantization for wNa16.
• Added an end-to-end example and unit tests.
• Verified functionality with local accuracy tests (GSM8K with a limit of 1000, the results may fluctuate due to non-determinism.)

- LLMC-AutoRound
vllm (pretrained=/storage/yiliu7/Meta-Llama-3-8B-Instruct-W4A16-G128-disbale-shuffule,tensor_parallel_size=1,max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False), gen_kwargs: (None), limit: 1000.0, num_fewshot: None, batch_size: 128
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value|   |Stderr|
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.737|±  |0.0139|
|     |       |strict-match    |     5|exact_match|↑  |0.736|±  |0.0139|

- AutoRound result as ref
vllm (pretrained=/storage/yiliu7/meta-llama/Meta-Llama-3-8B-Instruct-ar/Meta-Llama-3-8B-Instruct-w4g128/,tensor_parallel_size=1,max_model_len=8192,max_num_batched_tokens=32768,max_num_seqs=128,add_bos_token=True,gpu_memory_utilization=0.8,dtype=bfloat16,max_gen_toks=2048,enable_prefix_caching=False), gen_kwargs: (None), limit: 1000.0, num_fewshot: None, batch_size: 128
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value|   |Stderr|
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.739|±  |0.0139|
|     |       |strict-match    |     5|exact_match|↑  |0.740|±  |0.0139|

Attached eval cmd FYI.

Next stage (in later PRs)

• Extend support for additional data types.
• Add group-wise quantization recipes mapping between LLMC and AutoRound.
• Add end-to-end tests.

cc @hshen14 @thuang6 @wenhuach21

[kylesayrs]

Looks great. There's a few more things I'd like to point out, will give a full review soon.

[kylesayrs]

The sequential pipeline is not guaranteed to break a model into decoder layers. If a user specifies sequential_targets="Linear", then each SEQUENTIAL_EPOCH_END will trigger on each linear layer of the model.

One way to generalize this would be to have the sequential pipeline return the modules that were in the sequential layer

class Subgraph:
    def get_modules(self, model: torch.nn.Module, recurse: bool = False) -> Set[torch.nn.Module]:
        nodes = self.graph.find_nodes(op="call_module")
        modules = set(model.get_submodule(node.target) for node in nodes)
        if recurse:
            modules = set(module.modules() for module in modules)

        return modules

class SequentialPipeline:
...
    for subgraph in subgraphs:
         LifecycleCallbacks.sequential_epoch_end(subgraph)

def apply_autoround(self, state, subgraph):
    decoding_layer = torch.nn.ModuleList(list(subgraph.modules()))

[kylesayrs]

FYI these changes are implemented here, and this PR can potentially rebase on them.

[yiliu30]

Thanks for the detailed explanation — this approach is definitely more robust. I’ve tested it locally, and it works well.

Will #1998 be merged soon? If so, I’d prefer to rebase on main and update my PR accordingly to avoid introducing too much code here.

[yiliu30]

I have rebased the main branch and updated it accordingly.

[brian-dellabetta]

Thanks for the contribution! This is really cool. I have a few comments/questions in an initial review

[brian-dellabetta]

Thanks for addressing my comments! A few more small things:

[brian-dellabetta]

Hi @yiliu30 , do you have an estimate for when the next version of autoround will release? Does it have the appropriate licensing to avoid issues like this?

[yiliu30]

Hi @yiliu30 , do you have an estimate for when the next version of autoround will release? Does it have the appropriate licensing to avoid issues like vllm-project/compressed-tensors#468?

Hi @brian-dellabetta , We're planning to release the next version within the next 1–2 weeks—hope that works for you!
As for AutoRound, it's licensed under Apache License 2.0, so I guess there shouldn't be any licensing concerns.