Add support for float8 activation for Int4PreshuffledTensor

[jerryzh168]

Stacked PRs:

• Add all fbgemm kernel Tensors into Int4WeightOnlyConfig and Float8DynamicActivationInt4WeightConfig #2474
• Add Float8Tensor #2463
• Rename torchao.float8.Float8Tensor to torchao.float8.Float8TrainingTensor #2479
• ->Add support for float8 activation for Int4PreshuffledTensor #2437

---

Add support for float8 activation for Int4PreshuffledTensor

Summary:
Added basic op support like linear and bmm, we have both float8 and bf16 in the same Tensor
because it's the same dtype, only difference is whether the activation is quantized or not. Although
there is some differneces in implementation:

bf16 activaton:

• group_scale
• group_zero

fp8 activation

• group_scale
• row_scale

Test Plan:
python test/quantization/quantize_/workflows/int4/test_int4_preshuffled_tensor.py

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2437

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 889dca3 with merge base e5ca515 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Run Regression Tests / test-nightly (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch --index-url https://downloa... / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[drisspg]

Can you add serialization tests

[jerryzh168]

Can you add serialization tests

you mean serialization of the models? it's moved to https://github.com/pytorch/ao/pull/2463/files#diff-9f6b6c4b39656e797cfda97536a4cf8a82004c64da518ad524637b471b716739, I don't exactly remember the reason

for config serialization we did a config refactor in the last PR, I can add after we are aligned on what config should look like