Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add CPU benchmark #1353

Closed
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
2621e1a
docs: cleanup compilation instrs for multi-backend
Titus-von-Koeller Jul 27, 2024
87f88af
Enable loading prequantized weights with bf16/fp16/fp32 quant_storage…
matthewdouglas Jul 29, 2024
5cc91b1
Bump pytest from 8.3.1 to 8.3.2 in the minor-patch group (#1296)
dependabot[bot] Jul 29, 2024
a96d2f0
Merge branch 'main' into fsdp-load-prequantized
matthewdouglas Jul 29, 2024
3a6911f
Merge pull request #1295 from matthewdouglas/fsdp-load-prequantized
matthewdouglas Jul 29, 2024
b64cbe3
add job to upload wheels to continuous pre-release (#1282)
Titus-von-Koeller Jul 29, 2024
2e03d34
release 0.43.3
Titus-von-Koeller Jul 30, 2024
4be1883
packaging: bump permissions for continuous release step
Titus-von-Koeller Jul 31, 2024
6d714a5
Embedding4bit and Embedding8bit implementation (#1292)
galqiwi Aug 6, 2024
a4875fc
Bugfix: Load correct nocublaslt library variant when BNB_CUDA_VERSION…
matthewdouglas Aug 14, 2024
7476f6b
docs: update rocm installation instructions
Titus-von-Koeller Aug 18, 2024
fe8b3bc
add missing test requirement
Titus-von-Koeller Aug 21, 2024
6ae9859
docs: fix pre-commit instructions
Titus-von-Koeller Aug 21, 2024
a685654
Enable certain CUDA kernels to accept specified cuda stream (#1330)
jeejeelee Aug 22, 2024
432a4f4
Initial support for ppc64le (#1316)
mgiessing Aug 22, 2024
6bef412
Cuda source cleanup , refactor and fixes (#1328)
abhilash1910 Aug 26, 2024
2a8cc8d
docs: tweaks for multi-backend preview release prep
Titus-von-Koeller Aug 26, 2024
d300136
docs: get started on detailed multi-backend guide
Titus-von-Koeller Aug 29, 2024
e467453
actions: update permissions for pr docs publishing
Titus-von-Koeller Aug 30, 2024
17da4f6
Update for VS2022 17.11 compatibility with CUDA < 12.4 (#1341)
matthewdouglas Sep 9, 2024
a0da01e
correct version to next dev after last release
Titus-von-Koeller Sep 11, 2024
c6426ce
Bump the minor-patch group with 3 updates (#1362)
dependabot[bot] Sep 19, 2024
97b112c
Update matplotlib requirement from ~=3.9.1 to ~=3.9.2 in the major gr…
dependabot[bot] Sep 19, 2024
abb0c32
docs: add internal reference to multi-backend guide (#1352)
Titus-von-Koeller Sep 19, 2024
8fc7892
Add `move_to_device` kwarg to the optimizer's `load_state_dict` (#1344)
koute Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: update rocm installation instructions
Titus-von-Koeller committed Aug 18, 2024
commit 7476f6b61a11c430e9366459a913308bf9b662a4
20 changes: 15 additions & 5 deletions docs/source/installation.mdx
Original file line number Diff line number Diff line change
@@ -146,15 +146,25 @@ Please follow these steps to install bitsandbytes with device-specific backend s
bitsandbytes is fully supported from ROCm 6.1 onwards (currently in alpha release).

> [!TIP]
> If you already installed ROCm and PyTorch, skip Docker steps below and please check that the torch version matches your ROCm install. To install torch for a specific ROCm version, please refer to step 3 of wheels install in [Installing PyTorch for ROCm](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#using-wheels-package) guide.
> If you would like to install ROCm and PyTorch on bare metal, skip Docker steps and refer to our official guides at [ROCm installation overview](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/install-overview.html#rocm-install-overview) and [Installing PyTorch for ROCm](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/3rd-party/pytorch-install.html#using-wheels-package) (Step 3 of wheels build for quick installation). Please make sure to get PyTorch wheel for the installed ROCm version.

```bash
# Create a docker container with latest pytorch. It comes with ROCm and pytorch preinstalled
docker pull rocm/pytorch:latest
docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/pytorch:latest
# Create a docker container with latest ROCm image, which includes ROCm libraries
docker pull rocm/dev-ubuntu-22.04:6.1.2-complete
docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/dev-ubuntu-22.04:6.1.2-complete
apt-get update && apt-get install -y git && cd home

# Install pytorch compatible with above ROCm version
pip install torch --index-url https://download.pytorch.org/whl/rocm6.1/

# Install bitsandbytes from PyPI
# (This is supported on Ubuntu 22.04, Python 3.10, ROCm 6.1.0/6.1.1/6.1.2 and gpu arch - gfx90a, gfx942, gfx1100
# Please install from source if your configuration doesn't match with these)
pip install bitsandbytes

# Install bitsandbytes from source
# Clone bitsandbytes repo, ROCm backend is currently enabled on multi-backend-refactor branch
git clone --depth 1 -b multi-backend-refactor https://github.com/TimDettmers/bitsandbytes.git && cd bitsandbytes/
git clone --depth 1 -b multi-backend-refactor https://github.com/bitsandbytes-foundation/bitsandbytes.git && cd bitsandbytes/

# Install dependencies
pip install -r requirements-dev.txt