Skip to content

Fix recipe logic to propagate quantized graph in the pipeline (fixes #12659) #12661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 23, 2025

Conversation

abhinaykukkadapu
Copy link
Contributor

Summary:
I've found couple of issues with the original export recipes logic has incomplete functionality:

  1. The output of quantize stage is not getting propagated to next stages
  2. When quantize stage is run, we should re-export the model before we lower to edge.

This diff adds support for both. After this change the quantization flow revealed few gaps with xnnpack quantization and after which i've disable few tests due to the accuracy issues and an issue with dynamic per tensor quantization.

Changes:

  1. Adds support for above gaps
  2. This gap could've avoided with few unittests and this ads comprehensive tests for export recipe pipeline and stages
  3. Includes tests in pytest for oss to run

(fixes #12659)

Rollback Plan:

Differential Revision: D78585588

Copy link

pytorch-bot bot commented Jul 19, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12661

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit e35fd86 with merge base bc0ad82 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 19, 2025
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D78585588

Copy link

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@@ -74,7 +74,7 @@ def test_int8_dynamic_quant_recipe(self) -> None:
torch.allclose(
session.run_method("forward", example_inputs[0])[0],
m_eager(*example_inputs[0]),
atol=1e-3,
atol=1e-1,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically I would set rtol along with atol, ∣input−other∣≤atol+rtol×∣other∣

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also do we need it to be this high for two linears?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is consistent with my offline run without the recipes, i had to choose high tolerance. Can share more offline.

@@ -189,6 +189,7 @@ def _test_model_with_factory(self, model_name: str) -> None:
atol=1e-3,
)

@unittest.skip("T187799178: Debugging Numerical Issues with Calibration")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we should make a GH Issue perhaps?

…ytorch#12659) (pytorch#12661)

Summary:

I've found couple of issues with the original export recipes logic has incomplete functionality:
1.  The output of quantize stage is not getting propagated to next stages
2. When quantize stage is run, we should re-export the model before we lower to edge.

This diff adds support for both. After this change the quantization flow revealed few gaps with xnnpack quantization and after which i've disable few tests due to the accuracy issues and an issue with dynamic per tensor quantization.

Changes:
1. Adds support for above gaps
2. This gap could've avoided with few unittests and this adds comprehensive tests for export recipe pipeline and stages
3. Includes tests in pytest for oss to run

(fixes pytorch#12659)

Differential Revision: D78585588
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D78585588

@abhinaykukkadapu abhinaykukkadapu merged commit 0814358 into pytorch:main Jul 23, 2025
95 of 101 checks passed
@github-project-automation github-project-automation bot moved this from In progress to Done in ExecuTorch Core Jul 23, 2025
larryliu0820 added a commit that referenced this pull request Jul 24, 2025
Preventing the test added in #12661 from running since it's broken.
Conarnar pushed a commit to Conarnar/executorch that referenced this pull request Jul 25, 2025
…ytorch#12659) (pytorch#12661)

Summary:
I've found couple of issues with the original export recipes logic has
incomplete functionality:
1. The output of quantize stage is not getting propagated to next stages
2. When quantize stage is run, we should re-export the model before we
lower to edge.

This diff adds support for both. After this change the quantization flow
revealed few gaps with xnnpack quantization and after which i've disable
few tests due to the accuracy issues and an issue with dynamic per
tensor quantization.

Changes:
1. Adds support for above gaps
2. This gap could've avoided with few unittests and this ads
comprehensive tests for export recipe pipeline and stages
3. Includes tests in pytest for oss to run

(fixes pytorch#12659)

Rollback Plan:

Differential Revision: D78585588
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Fix bug in export recipe logic where quantization output is not being forwarded and reexport if quantized.
3 participants