Fix issue with new caching mechanism in transformers and bump versions - #313
Merged
Conversation
abdulfatir
changed the title
abdulfatir
changed the title
Contributor
Author
|
Modified asserts to fix type issues introduced by transformers v4.54. |
This was referenced Aug 4, 2025
There was a problem hiding this comment.
Pull Request Overview
This PR addresses a compatibility issue with transformers v4.54 by removing problematic is_encoder_decoder = False settings and updates package versions. The changes ensure Chronos-Bolt works correctly with the new caching mechanism introduced in transformers v4.54.
- Remove
is_encoder_decoder = Falseconfiguration for encoder and decoder in ChronosBolt - Improve assertion checks with better type validation and error handling
- Update package version to 1.5.3 and bump dependency versions (numpy to 2.0, datasets to 3.0)
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/chronos/chronos_bolt.py | Removes problematic is_encoder_decoder = False settings that break compatibility with transformers v4.54 |
| src/chronos/chronos.py | Improves assertion checks with better type validation and formatting |
| pyproject.toml | Updates package version and dependency versions, adds ruff configuration |
Comments suppressed due to low confidence (2)
pyproject.toml:29
- numpy~=2.0 may not be compatible with all systems and dependencies. Consider using a more conservative version constraint like numpy>=1.21,<3 to ensure broader compatibility.
test = ["pytest~=8.0", "numpy~=2.0"]
pyproject.toml:34
- datasets~=3.0 may not exist yet. The datasets library was at version 2.x as of early 2024. Consider using a more conservative constraint like datasets>=2.18,<3 until version 3.0 is confirmed to be available.
"datasets~=3.0",
shchur
approved these changes
Aug 5, 2025
| evaluation = [ | ||
| "gluonts[pro]~=0.15", | ||
| "numpy>=1.21,<3", | ||
| "datasets>=2.18,<4", |
Contributor
There was a problem hiding this comment.
Why do we need to cap datasets<4?
Contributor
Author
There was a problem hiding this comment.
Mainly because I haven't tested if it works with datasets==4. This mainly concerns training/evaluation so should not conflict with the main package anyway.
| encoder_config = copy.deepcopy(config) | ||
| encoder_config.is_decoder = False | ||
| encoder_config.use_cache = False | ||
| encoder_config.is_encoder_decoder = False |
Contributor
There was a problem hiding this comment.
Just to double check,
- Will this also work with older
transformersversions? - Do we need to also apply this fix to the Chronos model in AutoGluon?
Contributor
Author
There was a problem hiding this comment.
- I checked with 4.48 which is our lower bound in this repo. It does work there.
- At some point, I believe so, although currently AG uses
"transformers[sentencepiece]": ">=4.38.0,<4.50".
3 tasks
zlin101
pushed a commit
to zlin101/chronos-forecasting
that referenced
this pull request
Mar 3, 2026
amazon-science#313) *Issue #, if available:* Fixes amazon-science#310 and closes amazon-science#302 *Description of changes:* This PR fixes an issue related to the new caching mechanism for T5 introduced in `transformers==4.54`. [Prior versions set](https://github.com/huggingface/transformers/blob/v4.53.3/src/transformers/models/t5/modeling_t5.py#L1328) `encoder_config.is_encoder_decoder = False` when initializing encoder and decoder. Following transformers, we also initialized Chronos-Bolt in the same way. However, in v4.54 this line [has been removed](https://github.com/huggingface/transformers/blob/3fd456b200ba434e567412cc4517309482653f60/src/transformers/models/t5/modeling_t5.py#L1301) and [new logic has been added](https://github.com/huggingface/transformers/blob/3fd456b200ba434e567412cc4517309482653f60/src/transformers/models/t5/modeling_t5.py#L494) which relies on `is_encoder_decoder` [being True](https://github.com/huggingface/transformers/blob/3fd456b200ba434e567412cc4517309482653f60/src/transformers/models/t5/modeling_t5.py#L1007). This causes Chronos-Bolt to break as described in amazon-science#310. This PR removes `is_encoder_decoder = False` for both encoder and decoder which fixes the issue. I re-ran our mini eval in the CI and got the same results for v4.54 and v4.48 (our current lower bound). This PR also bumps package versions. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: Fixes #310 and closes #302
Description of changes: This PR fixes an issue related to the new caching mechanism for T5 introduced in
transformers==4.54. Prior versions setencoder_config.is_encoder_decoder = Falsewhen initializing encoder and decoder. Following transformers, we also initialized Chronos-Bolt in the same way. However, in v4.54 this line has been removed and new logic has been added which relies onis_encoder_decoderbeing True. This causes Chronos-Bolt to break as described in #310. This PR removesis_encoder_decoder = Falsefor both encoder and decoder which fixes the issue. I re-ran our mini eval in the CI and got the same results for v4.54 and v4.48 (our current lower bound).This PR also bumps package versions.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.