Torchao floatx version guard #12923
Torchao floatx version guard #12923
Conversation
Summary: TorchAO removing floatx support, added version guard in quantization_config.py
Summary: TorchAO removing floatx support, added version guard in quantization_config.py Altered tests in test_torchao.py to version guard floatx Created new test to verify version guard of floatx support
Summary: TorchAO removing floatx support, added version guard in quantization_config.py Altered tests in test_torchao.py to version guard floatx Created new test to verify version guard of floatx support
Summary: TorchAO removing floatx support, added version guard in quantization_config.py Altered tests in test_torchao.py to version guard floatx Created new test to verify version guard of floatx support
|
Also wanted to note that it seems like a lot of the tests in test_torchao (like test_quantization for example) were failing in the main branch |
| ("fp4", np.array([0.4668, 0.5195, 0.5547, 0.4199, 0.4434, 0.6445, 0.4316, 0.4531, 0.5625])), | ||
| ("fp6", np.array([0.4668, 0.5195, 0.5547, 0.4199, 0.4434, 0.6445, 0.4316, 0.4531, 0.5625])), | ||
| ]) | ||
| if version.parse(importlib.metadata.version("torchao")) <= version.Version("0.14.0"): |
There was a problem hiding this comment.
use is_torchao_version here as well?
There was a problem hiding this comment.
The import would be a bit ugly for it lives in a different folder, also it seemed like other torchao version checks in that file seemed to use this syntax. Should I still change it?
Summary: TorchAO removing floatx support, added version guard in quantization_config.py Altered tests in test_torchao.py to version guard floatx Created new test to verify version guard of floatx support
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| `float8_e4m3_tensor`, `float8_e4m3_row`, | ||
| - **Floating point X-bit quantization:** | ||
| - **Floating point X-bit quantization:** (in torchao <= 0.14.1, not supported in torchao >= 0.15.0) |
There was a problem hiding this comment.
Can we also provide a recommendation on what needs to be done if a user wants to use X-bit ops in torchao >= 0.15.0?
There was a problem hiding this comment.
they will need to downgrade to 0.14.1 or lower, we are removing this because we didn't see much usage for this feature
There was a problem hiding this comment.
Can we make that clear in the error message then?
they will need to downgrade to 0.14.1 or lower
There was a problem hiding this comment.
There's an error message for this in line 541 for if the user is trying to use the floatx in version >0.14.1. Is there a better place for it?
| ) | ||
| self._test_quant_type(quantization_config, expected_slice, model_id) | ||
|
|
||
| def test_floatx_quantization(self): |
There was a problem hiding this comment.
I don't think floatX is used much so we can avoid this test.
There was a problem hiding this comment.
Added a skip flag for this. We can also just remove it entirely if you think that's better.
Summary: TorchAO removing floatx support, added version guard in quantization_config.py Altered tests in test_torchao.py to version guard floatx Created new test to verify version guard of floatx support
4 participants
Motivation: TorchAO removing floatx support
Changes summary: added version guard in quantization_config.py
added version guard in quantization tests in test_torchao.py
added new test for version guarding in test_torchao.py (test_floatx_quantization)
Testing:
pytest tests/quantization/torchao/test_torchao.py -k test_floatx_quantization