Code

Add beam search and fix generate#392

Code

Closed

gpucce wants to merge 51 commits intomlfoundations:mainmlfoundations/open_clip:mainfrom gpucce:add_gen_beamgpucce/open_clip:add_gen_beamCopy head branch name to clipboard

+208-24

Commits

Commits on Dec 20, 2022

• Add coca trained (mlfoundations#307)

  Show description for 1b86601
  

  4 peopleauthored
  1b86601

  Copy full SHA for 1b86601

Commits on Dec 21, 2022

• Add coca to CI

  rom1504

  authored
  29fa332

  Copy full SHA for 29fa332

• Add coca to CI pr

  rom1504

  authored
  911c737

  Copy full SHA for 911c737

• simplify encode_iamge (mlfoundations#313)

  Show description for b4881bc
  

  gpucce

  and

  rom1504

  authored
  b4881bc

  Copy full SHA for b4881bc

• Add cls mask (mlfoundations#312)

  Show description for 50bc599
  

  3 peopleauthored
  50bc599

  Copy full SHA for 50bc599

Commits on Dec 22, 2022

• Ignore pad tokens in captioning loss (mlfoundations#316)

  Show description for 279e088
  

  gpucce

  and

  gpucce

  authored
  279e088

  Copy full SHA for 279e088

• add generate to coca model (mlfoundations#314)

  Show description for dee1ea5
  

  gpucce

  and

  gpucce

  authored
  dee1ea5

  Copy full SHA for dee1ea5

Commits on Jan 6, 2023

• use TextEncoder in coca encode_image (mlfoundations#321)

  Show description for 30a73d4
  

  3 peopleauthored
  30a73d4

  Copy full SHA for 30a73d4

• Merge branch 'main' into coca

  rom1504

  authored
  f616050

  Copy full SHA for f616050

• add beam search,fix image_latent,image_emb calculated once

  Soonhwan-Kwon

  committed
  9e37e4c

  Copy full SHA for 9e37e4c

• fix to using forward with output_dict

  Soonhwan-Kwon

  committed
  24167f7

  Copy full SHA for 24167f7

• fix typo image_emb to image_embs

  Soonhwan-Kwon

  committed
  7879799

  Copy full SHA for 7879799

Commits on Jan 8, 2023

• fix import error

  Soonhwan-Kwon

  committed
  fbdae97

  Copy full SHA for fbdae97

• add past_key_values to loss

  Soonhwan-Kwon

  committed
  283523a

  Copy full SHA for 283523a

• remove HF transformer version dependencies

  Soonhwan-Kwon

  committed
  6ff339f

  Copy full SHA for 6ff339f

Commits on Jan 9, 2023

• Get some basic PEP changes out of the way

  rwightman

  committed
  061482b

  Copy full SHA for 061482b

Commits on Jan 10, 2023

• remove changes related to past_key_values

  Soonhwan-Kwon

  committed
  f640696

  Copy full SHA for f640696

• repeat_interleave

  gpucce

  committed
  e0910f2

  Copy full SHA for e0910f2

• return full sequences

  gpucce

  committed
  1a8245f

  Copy full SHA for 1a8245f

• much much simpler

  gpucce

  committed
  dc8c128

  Copy full SHA for dc8c128

Commits on Jan 14, 2023

• Merge pull request #1 from gpucce/beamsearch

  Show description for d0469ee
  Soonhwan-Kwon

  authored
  d0469ee

  Copy full SHA for d0469ee

Commits on Jan 21, 2023

• Add tests bis (mlfoundations#355)

  Show description for d0bd09e
  gpucce

  authored
  d0bd09e

  Copy full SHA for d0bd09e

• Merge branch 'main' into coca

  rom1504

  authored
  ef80b7b

  Copy full SHA for ef80b7b

• train.py: fix is_clip when doing distributed (mlfoundations#364)

  iejMac

  authored
  2ab47b7

  Copy full SHA for 2ab47b7

Commits on Jan 22, 2023

• add README (mlfoundations#365)

  Show description for c0e5950
  iejMac

  authored
  c0e5950

  Copy full SHA for c0e5950

• Merge branch 'main' into coca

  rom1504

  authored
  9ab881e

  Copy full SHA for 9ab881e

• remove output_dict argument (mlfoundations#368)

  Show description for 3f5b0fb
  gpucce

  authored
  3f5b0fb

  Copy full SHA for 3f5b0fb

• do same thing for _encode_image (mlfoundations#366)

  Show description for de343fb
  iejMac

  authored
  de343fb

  Copy full SHA for de343fb

• Merge branch 'coca' into coca

  Soonhwan-Kwon

  authored
  db18668

  Copy full SHA for db18668

Commits on Jan 23, 2023

• remove output_tokens in forward

  Soonhwan-Kwon

  committed
  db68298

  Copy full SHA for db68298

• fix output_dict in forward to be in model, for clip consistency

  Soonhwan-Kwon

  committed
  b67fc90

  Copy full SHA for b67fc90

• CoCa/forward: remove unused output_dict param

  

  iejMac

  authored and

  rwightman

  committed
  88aa6ce

  Copy full SHA for 88aa6ce

Commits on Jan 24, 2023

• Merge branch 'coca' into coca

  Soonhwan-Kwon

  authored
  11c80ed

  Copy full SHA for 11c80ed

• Revert "do same thing for _encode_image (mlfoundations#366)"

  Show description for 3b66f37
  

  gpucce

  authored and

  rwightman

  committed
  3b66f37

  Copy full SHA for 3b66f37

• refactor

  

  gpucce

  authored and

  rwightman

  committed
  cdb91dd

  Copy full SHA for cdb91dd

• white space

  

  gpucce

  authored and

  rwightman

  committed
  58eb5bd

  Copy full SHA for 58eb5bd

• remove extra layer norm

  

  gpucce

  authored and

  rwightman

  committed
  cbd66ed

  Copy full SHA for cbd66ed

• move to_logits into decoder

  

  gpucce

  authored and

  rwightman

  committed
  bf6ef3e

  Copy full SHA for bf6ef3e

• leave for later

  

  gpucce

  authored and

  rwightman

  committed
  03dfeab

  Copy full SHA for 03dfeab

Commits on Jan 25, 2023

• better torchscript

  

  gpucce

  authored and

  rwightman

  committed
  15d6223

  Copy full SHA for 15d6223

• annotate hf too

  

  gpucce

  authored and

  rwightman

  committed
  9beb0d4

  Copy full SHA for 9beb0d4

Commits on Jan 27, 2023

• Add CoCa-ViT-L/14 config (mlfoundations#379)

  iejMac

  authored
  fde2aee

  Copy full SHA for fde2aee

• Merge branch 'coca' of https://github.com/Soonhwan-Kwon/open_clip into soo_coca

  gpucce

  committed
  9408f16

  Copy full SHA for 9408f16

• adjust to coca changes

  gpucce

  committed
  6180441

  Copy full SHA for 6180441

• add_cls option

  gpucce

  committed
  9670540

  Copy full SHA for 9670540

• cleaner

  gpucce

  committed
  8879326

  Copy full SHA for 8879326

• Merge branch 'main' into coca

  rom1504

  authored
  24e454d

  Copy full SHA for 24e454d

Commits on Jan 28, 2023

• Remove dead LN code, refactor attn_pool conditional for more clarity, minor formatting tweaks

  rwightman

  committed
  f7c566b

  Copy full SHA for f7c566b

• Merge branch 'coca' into soo_coca

  gpucce

  committed
  0bb6389

  Copy full SHA for 0bb6389

Commits on Jan 29, 2023

• Merge branch 'main' into soo_coca

  gpucce

  committed
  ab481f0

  Copy full SHA for ab481f0

• fixes

  gpucce

  committed
  ac91008

  Copy full SHA for ac91008