ENH: Updates for upcoming BNB Int8 release

[matthewdouglas]

This PR contains changes to PEFT that relate to an upcoming 0.45.0 release that we are planning for bitsandbytes. Primarily the changes are related to bitsandbytes#1401.

1. Bitsandbytes will introduce a public API function for LLM.int8() weight dequantization. We use this API when available, and otherwise use a functionally equivalent implementation. Apart from its relative simplicity, this will also resolve issues such as Merge LoRA into 405B #2065.
2. The memory_efficient_backward kwarg for Linear8bitLt will be removed in the upcoming release, as per a longstanding deprecation notice. This PR removes usage of the kwarg for compatibility.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Thanks for this PR. I think that if we depend on bnb 0.45 anyway, we can completely remove the is_bnb_4bit_available function, since it appears to stem from a time when bnb implemented 8bit quant but not yet 4bit quant. Also, the is_bnb_available check should perform a version check for >= v0.45.0.

Also, I ran pytest tests/test_common_gpu.py -k test_8bit_lora_mixed_adapter_batches_lora on my machine (using shared_library_cuda_ubuntu-latest_x86_64_12.4.1 and this bnb branch) and it passed for me.

When running all bnb tests with pytest tests/ -k "bnb or bitsandbytes" -v I only got these errors from the bnb regression tests:

FAILED tests/bnb/test_bnb_regression.py::test_opt_350m_8bit - AssertionError: Tensor-likes are not close!
FAILED tests/bnb/test_bnb_regression.py::test_opt_350m_8bit_threshold - AssertionError: Tensor-likes are not close!

which I guess is expected and we should update the regression artifact (or remove those tests altogether -- are they still needed?).

Furthermore, there are the PEFT regression tests, some of which involve bnb, of which one is failing:

$ pytest tests/regression/test_regression.py -s --regression -k bnb
...
FAILED tests/regression/test_regression.py::TestOpt8bitBnb::test_lora_8bit - AssertionError: assert False

which is also not unexpected and we can update the regression artifact this change is released.

[BenjaminBossan]

Thanks for this PR to update bnb 8bit in PEFT. The changes LGTM and after the internal discussion, this should be good to merge (some regression tests will need adjusting once bnb v0.45.0 is released).