Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: support precomputed image embeddings #99

Open
rom1504 opened this issue May 26, 2022 · 0 comments
Open

Feature: support precomputed image embeddings #99

rom1504 opened this issue May 26, 2022 · 0 comments

Comments

@rom1504
Copy link
Collaborator

rom1504 commented May 26, 2022

Implementing that would make it possible to efficiently train a lit-style clip model. Which is pretty useful when a decent visual encoder is available and the goal is to map it to captions (eg multilingual ones)

https://github.com/lucidrains/DALLE2-pytorch/blob/main/train_diffusion_prior.py has a good data loader for embedding+caption, this can be reused.

We may work on this, not asking for someone else to do it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant