Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] Staged 2of4 example #1238

Merged
merged 4 commits into from
Mar 11, 2025
Merged

Conversation

kylesayrs
Copy link
Collaborator

Purpose

  • Fix staged 2of4 example

Background

  • When [Callbacks] Remove pre_initialize_structure #1160 landed, this change introduced a bug in the recipe container which meant that the recipe was not recompiled after appending. This caused sgpt to initialize twice and gptq to never initialize, leading to a sparsity-only quantization config
  • At some point, a changed was introduced which causes previous stages to become reconstructed after recipe recompilation. This means that without resetting the session in between stages, previous stages will initialize twice.
    • In order to avoid this issue, this PR introduces session.reset() in between stages
      • This change has the consequence of creating recipe.yaml files which do not have the full recipe history. However, I believe this is acceptable for the time being, as the stage runner and this work flow will be removed in the next release.

Signed-off-by: Kyle Sayers <[email protected]>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

@kylesayrs kylesayrs changed the title [Bugfix] [Bugfix] Staged 2of4 example Mar 10, 2025
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so I understand, we're removing the functionality to track past stages? Which I think landed not too long ago

@dsikka dsikka added the ready When a PR is ready for review label Mar 10, 2025
@kylesayrs
Copy link
Collaborator Author

@dsikka That functionality breaks only for this example

@kylesayrs kylesayrs enabled auto-merge (squash) March 10, 2025 23:30
@kylesayrs kylesayrs merged commit 8290679 into main Mar 11, 2025
8 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/fix-quantization-modifier branch March 11, 2025 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants