ROCm Backend Status Tracker

[pnunna93]

This issue summarizes and tracks the current status of ROCm enablement for BitsandBytes.

• Enable ROCm backend for bitsandbytes - Supported from ROCm 6.1
  BitsandBytes Enablement on ROCm #1207
  Updated CUDA backend to work seamlessly for ROCm. Added hipified versions of CUDA kernels and ops which allows to use optimized HIP kernels for AMD GPUs. Cmake build updated for ROCm and also enabled key features in bitsandbytes functional and autograd api
• Update ROCm install instructions in official bitsandbytes documentation
  cleanup docs-build breaking install instructs #1244
  Bitsandbytes can be built for ROCm backend by setting COMPUTE_BACKEND to 'hip' in the cmake command. The instructions to get supported dockers and steps to build ROCm enabled bitsandbytes are present in the installation doc of multi-backend-refactor branch.
• Provide AMD GPU to public CI
  AMD GPU has been successfully added into Huggingface CI for ROCm backend testing
• Enable ROCm build in public CI
  Create build job for ROCm #1255
  Added a new CI job to build bitsandbytes with ROCm backend for push/pull requests to main branch, releases and manually triggered jobs
• Provide packaging/wheels support for ROCm
  Enable bitsandbytes packaging for ROCm #1299
  Updated shared libs and wheels build jobs for ROCm. The new updates will create ROCm specific .so files, which will be packaged into bitsandbytes wheels. In addition to this, the documentation is updated with steps to install bitsandbytes through newly built packages.
• Update Debug Messages/Diagnostic Functions for ROCm
  Update diagnostic functions for ROCm #1333
  The diagnostic functions which aid users with debug messages related to installation failures are updated for ROCm. Also, added additional messages for igemmlt failures and a fix to clean up runner disk space for CI jobs.
• Enable packaging for ROCm 6.2
  Enable packaging for ROCm 6.2 #1367
  Wheels packaging is enabled for ROCm 6.2. Users can use pip install directly for ROCm 6.2 after this change.

The completed changes are in multi-backend-refactor branch and will be merged into main once multi-backend design is finalized.

cc: @Titus-von-Koeller @sunway513 @amathews-amd

[Titus-von-Koeller]

@pnunna93

PyTorch 2.4 is built with ROCm 6.1, we should be sure that the bundled ROCm binaries are compatible with the newest PyTorch release that just came out.

[Titus-von-Koeller]

@pnunna93 I'm planning to do the alpha release with prebuilt wheels in the beginning of next week. Do you think you'll have the last item on this list checked by then? I think it would be important to prioritize this so we can move forward and get more user feedback.

[pnunna93]

@Titus-von-Koeller Sure, I have been ironing out some details and got most of the info I need. I will start testing and open a PR early next week.