Skip to content

Conversation

@shrijayan
Copy link

This pull request introduces support for Gemma 2 models in the llm2vec repository by implementing the following enhancements:

Key Changes:

  1. Updated Transformers Version Constraint:

    • Modified dependencies to allow transformers==4.45.2, ensuring compatibility with the latest Gemma 2 configurations.
  2. Added Support for Gemma2Config:

    • Integrated Gemma2Config to enable seamless usage of Gemma 2 models within the llm2vec framework.
    • Ensured backward compatibility with existing model configurations.
  3. Enhanced GemmaBiModel and GemmaBiForMNTP:

    • Updated GemmaBiModel and GemmaBiForMNTP to handle Gemma 2 architecture properly.
    • Refactored encoding logic to leverage new features from Gemma 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant