[Feature] Support Google Gemma Models #72
Description
🚀 Feature Request
Describe the feature you'd like
A clear and concise description of the new functionality.
Google Gemma uses a softcap in attention computation which needs to be supported in skylight.
Why is it needed?
Explain the problem it solves or the value it adds.
Need more model support
Additional context (optional)
Any links, references, screenshots, or related issues.
https://github.com/huggingface/transformers/blob/main/src/transformers/models/gemma/modeling_gemma.py