Skip to content

static quantization files for llama4-maverick #1968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: habana_main
Choose a base branch
from
Open

static quantization files for llama4-maverick #1968

wants to merge 5 commits into from

Conversation

sureshnam
Copy link

@sureshnam sureshnam commented Sep 22, 2025 •
edited by github-actions bot
Loading

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results

Purpose

Static quantization files for models to share to customer

Test Plan

Test Result

@varistar
Copy link

@sureshnam , are these quant scales identical to https://huggingface.co/sureshnam9/Llama-4-Maverick-17B-128E-Instruct-FP8/tree/main/quant/g3?

PatrykWo
PatrykWo requested changes Sep 23, 2025
View reviewed changes
Copy link

@PatrykWo PatrykWo left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move the 1.22.0 lower in structure, as subfolder of .static_quant.
EDIT: It's UI issue. Ignore my comment.

@PatrykWo
Copy link

PatrykWo commented Sep 23, 2025 •
edited
Loading

@sureshnam please resolve pre-commit error.

@sureshnam
Copy link
Author

@varistar, the files are the same uploaded to HF as well.

PatrykWo
PatrykWo approved these changes Sep 24, 2025
View reviewed changes
Copy link

@PatrykWo PatrykWo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@PatrykWo
Copy link

/skip-gaudi-tests

@varistar
Copy link

We have another version of scales which give +20% perf speed-up enabling expert-parallelism, please hold the merge till we complete comparison and run accuracy testing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PatrykWo PatrykWo PatrykWo approved these changes

@kzawora-intel kzawora-intel Awaiting requested review from kzawora-intel kzawora-intel is a code owner

@madamczyk-intel madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel is a code owner

@michalkuligowski michalkuligowski Awaiting requested review from michalkuligowski michalkuligowski is a code owner

@mgawarkiewicz-intel mgawarkiewicz-intel Awaiting requested review from mgawarkiewicz-intel mgawarkiewicz-intel is a code owner

@vivekgoe vivekgoe Awaiting requested review from vivekgoe vivekgoe is a code owner

@afierka-intel afierka-intel Awaiting requested review from afierka-intel afierka-intel is a code owner

@xuechendi xuechendi Awaiting requested review from xuechendi xuechendi is a code owner

@jikunshang jikunshang Awaiting requested review from jikunshang jikunshang is a code owner

@mswiniarsk mswiniarsk Awaiting requested review from mswiniarsk mswiniarsk is a code owner

@wpyszka wpyszka Awaiting requested review from wpyszka wpyszka is a code owner

@deepvars deepvars Awaiting requested review from deepvars deepvars is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sureshnam @varistar @PatrykWo