breaks for 4 dimensions

[HDCharles]

System Info

A100 cuda 12.1

Package Version Editable project location

---

absl-py 2.0.0
accelerate 0.26.1
aiohttp 3.9.1
aiosignal 1.3.1
annotated-types 0.6.0
antlr4-python3-runtime 4.9.3
anyio 4.0.0
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
async-timeout 4.0.3
attrs 23.1.0
Babel 2.13.1
beautifulsoup4 4.12.2
bitsandbytes 0.42.0
bleach 6.1.0
certifi 2022.12.7
cffi 1.16.0
chardet 5.2.0
charset-normalizer 2.1.1
click 8.1.7
colorama 0.4.6
comm 0.2.0
DataProperty 1.0.1
datasets 2.16.1
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.7
distro 1.9.0
einops 0.7.0
exceptiongroup 1.1.3
executing 2.0.1
fastjsonschema 2.19.0
filelock 3.9.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
h11 0.14.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.20.2
idna 3.4
ipykernel 6.26.0
ipython 8.17.2
ipywidgets 8.1.1
isoduration 20.11.0
jedi 0.19.1
Jinja2 3.1.2
joblib 1.3.2
json5 0.9.14
jsonlines 4.0.0
jsonpointer 2.4
jsonschema 4.19.2
jsonschema-specifications 2023.11.1
jupyter 1.0.0
jupyter_client 8.6.0
jupyter-console 6.6.3
jupyter_core 5.5.0
jupyter-events 0.9.0
jupyter-lsp 2.2.0
jupyter_server 2.10.1
jupyter_server_terminals 0.4.4
jupyterlab 4.0.8
jupyterlab-pygments 0.2.2
jupyterlab_server 2.25.1
jupyterlab-widgets 3.0.9
lm-eval 0.3.0 /home/cdhernandez/local/lm-evaluation-harness
MarkupSafe 2.1.3
matplotlib-inline 0.1.6
mbstrdecoder 1.1.3
mistune 3.0.2
mpmath 1.2.1
multidict 6.0.4
multiprocess 0.70.15
nbclient 0.9.0
nbconvert 7.11.0
nbformat 5.9.2
nest-asyncio 1.5.8
networkx 3.0rc1
ninja 1.10.2
nltk 3.8.1
notebook 7.0.6
notebook_shim 0.2.3
numexpr 2.8.8
numpy 1.24.1
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.19.3
nvidia-nvjitlink-cu12 12.1.105
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
openai 1.7.2
overrides 7.4.0
packaging 23.2
pandas 2.1.4
pandocfilters 1.5.0
parso 0.8.3
pathvalidate 3.2.0
peft 0.7.1
pexpect 4.8.0
Pillow 9.3.0
pip 23.3.1
platformdirs 4.0.0
portalocker 2.8.2
prometheus-client 0.18.0
prompt-toolkit 3.0.41
psutil 5.9.6
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 14.0.2
pyarrow-hotfix 0.6
pybind11 2.11.1
pycountry 23.12.11
pycparser 2.21
pydantic 2.5.3
pydantic_core 2.14.6
Pygments 2.16.1
pytablewriter 1.2.0
python-dateutil 2.8.2
python-json-logger 2.0.7
pytorch-triton 2.2.0+e28a256d71
pytz 2023.3.post1
PyYAML 6.0.1
pyzmq 25.1.1
qtconsole 5.5.1
QtPy 2.4.1
referencing 0.31.0
regex 2023.12.25
requests 2.31.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rouge-score 0.1.2
rpds-py 0.12.0
sacrebleu 1.5.0
safetensors 0.4.1
scikit-learn 1.3.2
scipy 1.11.4
segment-anything 1.0
Send2Trash 1.8.2
sentencepiece 0.1.99
setuptools 68.0.0
six 1.16.0
sniffio 1.3.0
soupsieve 2.5
sqlitedict 2.1.0
stack-data 0.6.3
sympy 1.11.1
tabledata 1.3.3
tcolorpy 0.1.4
terminado 0.18.0
threadpoolctl 3.2.0
tinycss2 1.2.1
tokenizers 0.15.0
tomli 2.0.1
torch 2.3.0.dev20240110+cu121
torchao 0.0.1
torchaudio 2.2.0.dev20240110+cu121
torchvision 0.18.0.dev20240110+cu121
tornado 6.3.3
tqdm 4.66.1
tqdm-multiprocess 0.0.11
traitlets 5.13.0
transformers 4.36.2
typepy 1.3.2
types-python-dateutil 2.8.19.14
typing_extensions 4.8.0
tzdata 2023.4
uri-template 1.3.0
urllib3 1.26.13
wcwidth 0.2.10
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.4
wheel 0.41.2
widgetsnbextension 4.0.9
xxhash 3.4.1
yarl 1.9.4
zstandard 0.22.0

Reproduction

https://github.com/TimDettmers/bitsandbytes/blob/main/bitsandbytes/functional.py#L445

if dims>3 it just leaves rows empty which causes an error if to_order isn't "row" or "column"

repro with

mod=bitsandbytes.nn.Linear8bitLt(1280, 3840, bias=True, has_fp16_weights=False).to(0).cuda()
mod(torch.randn((400,14,14,1280) , device='cuda', dtype=torch.float16))

UnboundLocalError: local variable 'rows' referenced before assignment

note even after fixing that there are further issues

AttributeError: 'NoneType' object has no attribute 'dtype'

Expected behavior

not breaking

[matthewdouglas]

As of v0.45.0, due to #1401, we'll instead raise:
AssertionError: Only two or three dimensional matrices are supported for argument A.

A workaround for dim>3 could be to fold the additional dimensions first.