Skip to content

⏪ revert back to 3-layer micro model #497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

⏪ revert back to 3-layer micro model #497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
87b339d
⏪ revert back to 3-layer micro model
prashantgupta24 Oct 1, 2025
7174818
🐛 download the right revision
prashantgupta24 Oct 1, 2025
5aeeb88
🎨 fmt
prashantgupta24 Oct 1, 2025
38acec8
🐛 download the right model
prashantgupta24 Oct 1, 2025
49c5077
🚧 flip condition so that we download the right model
prashantgupta24 Oct 1, 2025
e90387e
Merge remote-tracking branch 'upstream/main' into switch-to-3-layer-m…
prashantgupta24 Oct 1, 2025
b194d0a
⚡️ add hf_cache for older revision
prashantgupta24 Oct 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,15 +174,15 @@ jobs:
key: ${{ runner.os }}-hf-model-${{ env.model_key }}

- name: "Download HF models"
if: ( steps.changed-src-files.outputs.any_changed == 'true' && steps.cache_restore.outputs.cache-hit != 'true' )
if: ( steps.changed-src-files.outputs.any_changed == 'true' && steps.cache_restore.outputs.cache-hit == 'true' )
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

temporary change to get GH to download the older revision

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means you will have both revisions:

  • 4 layer model from the cache
  • 3 layer model downloaded with revision

Making all the tests work. If you bypass the cache, I reckon some tests will break like in PR #499

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the 3 layer model is verified, I don't think we need the 4 layer anymore, so we should only be using one revision of the model at all places

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally the fix has to be that we delete the old cache and populate it with the 3 layer revision and merge this PR right?

Copy link
Collaborator

@ckadner ckadner Oct 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it needs more code/test changes to make sure we don't use the latest/main revision of the tiny granite model anymore/anywhere.

Kicking of a test run here that is not also still using the old cached model: #502

Copy link
Collaborator

@ckadner ckadner Oct 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup:

E huggingface_hub.errors.LocalEntryNotFoundError: Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online, pass 'local_files_only=False' as input.

head_call_error = OfflineModeIsEnabled("Cannot access file since 'local_files_only=True' as been set. (repo_id: ibm-ai-platform/micro-g3.3-8b-instruct-1b, repo_type: model, revision: main, filename: config.json)") force_download = False, local_files_only = True

Also, need to not use the same revision key to download the FP8 model 🙄
ValueError: Unrecognized model in ibm-ai-platform/micro-g3.3-8b-instruct-1b-FP8

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#503

run: |
# We are caching HF models (HF_HUB_CACHE) for reliability rather than speed, since HF downloads are flaky for concurrent jobs.
# Be careful when adding models to the cache here, as the GHA cache is limited to 10 GB.
# If a new model is added here, a new hash key is generated. The previous cache blob can then
# be removed by an admin or can be left to expire after 7 days.

download_tinygranite() {
python -c "from transformers import pipeline, AutoTokenizer; pipeline('text-generation', model='$1'); tokenizer=AutoTokenizer.from_pretrained('$1')"
python -c "from transformers import pipeline; pipeline('text-generation', model='$1', revision='2714578f54cfb744ece40df9326ee0b47e879e03');"
}
download_roberta_large() {
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('$1')"
Expand Down
Loading