Add TorchAO wrapper config to allow filter_fn for quantize_ #13264
Conversation
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13264
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New Failures, 7 Pending, 1 Unrelated FailureAs of commit 482f3d6 with merge base 0e76a97 ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
| eager_quantized_model = source_transform_output.data["forward"] | ||
| output = session.run_method("forward", example_inputs[0])[0] | ||
| expected = eager_quantized_model(*example_inputs[0]) | ||
| self.assertTrue(torch.allclose(output, expected, atol=atol)) |
There was a problem hiding this comment.
You might want to print more stats if this fails - see https://github.com/pytorch/executorch/blob/main/backends/test/harness/tester.py#L337
| atol=1e-1, | ||
| ) | ||
| self._compare_eager_quantized_model_outputs( | ||
| session, example_inputs, 1e-1 |
There was a problem hiding this comment.
atol? Why is this so high for two linears?
| session, example_inputs, 1e-1 | |
| session, example_inputs, atol=1e-1 |
There was a problem hiding this comment.
Yeah, i think 1e-2 is working on my mac, will check if linux passes on CI. Nevertheless, i'm updating the tolerance tests similar to CoreML (let me know if there is any objection) to use sqnr to compare eager model vs lowered model output.
But use tolerance checks to compare post quantized model and lowered model.
| raise ValueError("Mixed quantizer types are not supported") | ||
| if len(torch_ao_quantizers) > 1: | ||
| raise ValueError( | ||
| "Multiple quantizers of torch.ao.quantization.quantizer not supported" |
There was a problem hiding this comment.
Doesn't torchao already detect this and give an error if mixing? I thought I added that
There was a problem hiding this comment.
May be the torchao version is different?
|
Overall, looks good. Address comments before merging |
Changes:
ComposableQuantizerif there are multiple quantizers and is of type torchao, for legacy quantizers use them directly with prepare_pt2e.