Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix bug with some LoRA variants when applied to bitsandbytes NF4 quantized models #7577

Merged
merged 2 commits into from
Jan 21, 2025
Merged

Fix bug with some LoRA variants when applied to bitsandbytes NF4 quantized models #7577

merged 2 commits into from
Jan 21, 2025

Conversation

RyanJDick
Copy link
Collaborator

@RyanJDick RyanJDick commented Jan 20, 2025 •
edited
Loading

Summary

Fixes a bug with some LoRA variants (LoKR, LoHA and some other obscure variants - LoRA is not affected) when they are applied to bitsandbytes NF4 quantized models. Unit tests were added to exercise the affected case.

Related Issues / Discussions

QA Instructions

  • Test LoKR on full FLUX dev
  • Test LoKR on GGUF quantized FLUX dev
  • Test LoKR on BnB NF4 quantized FLUX dev

Regression tests:

  • Test LoRA on full FLUX dev
  • Test LoRA on GGUF quantized FLUX dev
  • Test LoRA on BnB NF4 quantized FLUX dev
  • Test FLUX control LoRA on BnB NF4 quantized FLUX dev
  • Test a LoRA on MPS system (to confirm that missing bitsandbytes package is handled)

Merge Plan

No special isntructions.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files backend PRs that change backend files python-tests PRs that change python tests labels Jan 20, 2025
@RyanJDick RyanJDick marked this pull request as ready for review January 20, 2025 20:24
This was referenced Jan 20, 2025
Closed
Closed
@psychedelicious psychedelicious enabled auto-merge (rebase) January 21, 2025 22:10
@RyanJDick @psychedelicious
Add unit tests for LoKR patch layers. The new tests trigger a bug whe…
7724bd5 
…n LoKR layers are applied to BnB-quantized layers (also impacts several other LoRA variant types).
@RyanJDick @psychedelicious
Fix bug with some LoRA variants when applied to a BnB NF4 quantized m…
fa7c36a 
…odel. Note the previous commit which added a unit test to trigger this bug.
@psychedelicious psychedelicious force-pushed the ryan/fix-lora-variants-on-bnb-quantized branch from 77a86ff to fa7c36a Compare January 21, 2025 22:10
@psychedelicious psychedelicious merged commit f88c1ba into main Jan 21, 2025
14 of 15 checks passed
@psychedelicious psychedelicious deleted the ryan/fix-lora-variants-on-bnb-quantized branch January 21, 2025 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hipsterusername hipsterusername hipsterusername approved these changes

@psychedelicious psychedelicious psychedelicious approved these changes

@lstein lstein Awaiting requested review from lstein lstein is a code owner

@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

@brandonrising brandonrising Awaiting requested review from brandonrising brandonrising is a code owner

Assignees
No one assigned
Labels
backend PRs that change backend files python PRs that change python files python-tests PRs that change python tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[bug]: Certain FLUX LoRAs install, but fail to run [enhancement]: Support of LoKR (type of LORA) for FLUX
3 participants
@RyanJDick @psychedelicious @hipsterusername