Matryoshka Representation Learning for Open CLIP #299
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hey Aditya thanks for the PR with MRL -- however if you want to make MRL an option it would be good to have a flag so that this PR does not affect default training and its not hard coded. Also, we'd want to do experiments to verify MRL works in this setting, which I can help you get set up on the UW compute -- we have a subset of LAION there. |
|
Sure thing Mitchell. I was hard coding it so that Rom could use it to train a model. I can probably cleanup the code and make it a flag so that it does not break training. I don't think we will be merging this PR anyway. I can run things on LAION subset at UW with the generalized or hard coded variant. |
|
sure can you convert to draft in the meantime? |
|
Done. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR hardcodes 4 granularities of dimensions to train CLIP on.
While I hope the default hparams will work, it might be good to train it for slightly longer epochs.