Integration of BitNet into bitsandbytes

[ParagEkbote]

Feature request

After the release of the bitnet architecture, I believe that the architecture and its methods could be integrated into the bnb library to further improve the accessibility of LLMs using quantization.

While there are some performance issues in the initial experiments, integration in the bnb library can help drive further improvements and optimizations to the quantization methods . The experiment methods also seem to be available on the HF repository of the experiment, which could make the integration easier.

Reference: https://huggingface.co/blog/1_58_llm_extreme_quantization
https://huggingface.co/1bitLLM/bitnet_b1_58-xl/blob/main/modeling_bitnet.py

Motivation

There are several capable open source LLMs like Arcee-SuperNova, LLama3.2-70B, etc that cannot be used by normal users without having lots of GPU compute.

By adding bitnet , these LLMs can be adopted on a larger scale by users that can take advantage of its capabilities.

Your contribution

With ample support and guidance, I could help in the integration.

[ParagEkbote]

Closing the issue since there are on-going efforts in the transformers and nanotron library for the same.