Code

Mixture of Experts#115

Code

Merged

kernelmachine merged 63 commits intomainmlfoundations/open_lm:mainfrom moemlfoundations/open_lm:moeCopy head branch name to clipboardDec 18, 2023

+306-9

Commits

Commits on Nov 21, 2023

• update

  

  Suchin Gururangan

  committed
  8fcc9b9

  Copy full SHA for 8fcc9b9

Commits on Nov 25, 2023

• works on one gpu

  

  Suchin Gururangan

  committed
  7093f83

  Copy full SHA for 7093f83

Commits on Nov 26, 2023

• added moe params

  

  Suchin Gururangan

  committed
  b6ba69f

  Copy full SHA for b6ba69f

Commits on Nov 27, 2023

• works on multiple gpus

  

  Suchin Gururangan

  committed
  8b4c230

  Copy full SHA for 8b4c230

• update

  

  Suchin Gururangan

  committed
  45c1849

  Copy full SHA for 45c1849

• eval works

  

  Suchin Gururangan

  committed
  66359d9

  Copy full SHA for 66359d9

• update

  

  Suchin Gururangan

  committed
  7466d60

  Copy full SHA for 7466d60

• update

  

  Suchin Gururangan

  committed
  440f9f7

  Copy full SHA for 440f9f7

• update

  

  Suchin Gururangan

  committed
  5c37fdb

  Copy full SHA for 5c37fdb

• update

  

  Suchin Gururangan

  committed
  429a780

  Copy full SHA for 429a780

• update

  

  Suchin Gururangan

  committed
  930174f

  Copy full SHA for 930174f

• removed experiment dir

  

  Suchin Gururangan

  committed
  a302c66

  Copy full SHA for a302c66

• removed experiments dir

  

  Suchin Gururangan

  committed
  6483ef2

  Copy full SHA for 6483ef2

• removed custom fsdp

  

  Suchin Gururangan

  committed
  d0ea8dc

  Copy full SHA for d0ea8dc

• update

  

  Suchin Gururangan

  committed
  4eb3dd0

  Copy full SHA for 4eb3dd0

• update

  

  Suchin Gururangan

  committed
  a46a2fe

  Copy full SHA for a46a2fe

Commits on Nov 29, 2023

• added expert gradient norm

  

  Suchin Gururangan

  committed
  f5421f6

  Copy full SHA for f5421f6

Commits on Dec 12, 2023

• update

  

  Suchin Gururangan

  committed
  e254615

  Copy full SHA for e254615

• update

  

  Suchin Gururangan

  committed
  0912ba8

  Copy full SHA for 0912ba8

• Update model.py

  kernelmachine

  authored
  0a4d0a3

  Copy full SHA for 0a4d0a3

• added load balancing loss

  

  Suchin Gururangan

  committed
  53905bc

  Copy full SHA for 53905bc

• Merge branch 'moe' of github.com:mlfoundations/open_lm into moe

  

  Suchin Gururangan

  committed
  a4d0efc

  Copy full SHA for a4d0efc

• update

  

  Suchin Gururangan

  committed
  52e2cfd

  Copy full SHA for 52e2cfd

Commits on Dec 13, 2023

• update

  

  Suchin Gururangan

  committed
  ebacdbe

  Copy full SHA for ebacdbe

• update

  

  Suchin Gururangan

  committed
  f404b0d

  Copy full SHA for f404b0d

• update

  

  Suchin Gururangan

  committed
  2efdf17

  Copy full SHA for 2efdf17

Commits on Dec 14, 2023

• update

  

  Suchin Gururangan

  committed
  88156ca

  Copy full SHA for 88156ca

• update

  

  Suchin Gururangan

  committed
  e783e9e

  Copy full SHA for e783e9e

• update

  

  Suchin Gururangan

  committed
  f027203

  Copy full SHA for f027203

Commits on Dec 15, 2023

• fixed merge conflicts

  

  Suchin Gururangan

  committed
  5c70426

  Copy full SHA for 5c70426

• removed unnecessary dir

  

  Suchin Gururangan

  committed
  a6be666

  Copy full SHA for a6be666

• Remove the now ignored directory experiments

  

  Suchin Gururangan

  committed
  39124e7

  Copy full SHA for 39124e7

• update

  

  Suchin Gururangan

  committed
  6196cbc

  Copy full SHA for 6196cbc

• update

  

  Suchin Gururangan

  committed
  560250a

  Copy full SHA for 560250a

• update

  

  Suchin Gururangan

  committed
  c904356

  Copy full SHA for c904356

• update

  

  Suchin Gururangan

  committed
  525083c

  Copy full SHA for 525083c

• update

  

  Suchin Gururangan

  committed
  25e6948

  Copy full SHA for 25e6948

• update

  

  Suchin Gururangan

  committed
  4849f2a

  Copy full SHA for 4849f2a

• update

  

  Suchin Gururangan

  committed
  4c41b61

  Copy full SHA for 4c41b61

• black formatting

  

  Suchin Gururangan

  committed
  5c32f64

  Copy full SHA for 5c32f64

• removed custom moe dir

  

  Suchin Gururangan

  committed
  78ac356

  Copy full SHA for 78ac356

• update

  

  Suchin Gururangan

  committed
  24cbb9b

  Copy full SHA for 24cbb9b

• update

  

  Suchin Gururangan

  committed
  ec8cd0f

  Copy full SHA for ec8cd0f

• update

  

  Suchin Gururangan

  committed
  427bc39

  Copy full SHA for 427bc39

• update

  

  Suchin Gururangan

  committed
  64e512b

  Copy full SHA for 64e512b

• update

  

  Suchin Gururangan

  committed
  cbfa187

  Copy full SHA for cbfa187

• update

  

  Suchin Gururangan

  committed
  90648c9

  Copy full SHA for 90648c9

• update

  

  Suchin Gururangan

  committed
  32cba9d

  Copy full SHA for 32cba9d

• update

  

  Suchin Gururangan

  committed
  da9d4a7

  Copy full SHA for da9d4a7

• Merge branch 'main' into moe

  kernelmachine

  authored
  5741fde

  Copy full SHA for 5741fde

• Merge branch 'main' of github.com:mlfoundations/open_lm into moe

  

  Suchin Gururangan

  committed
  a70609b

  Copy full SHA for a70609b

• update

  

  Suchin Gururangan

  committed
  e722226

  Copy full SHA for e722226

• Merge branch 'moe' of github.com:mlfoundations/open_lm into moe

  

  Suchin Gururangan

  committed
  5980bf8

  Copy full SHA for 5980bf8

Commits on Dec 16, 2023

• update

  

  Suchin Gururangan

  committed
  ce3c838

  Copy full SHA for ce3c838

• update

  

  Suchin Gururangan

  committed
  8b7c77d

  Copy full SHA for 8b7c77d

• update

  

  Suchin Gururangan

  committed
  8daf040

  Copy full SHA for 8daf040

• update

  

  Suchin Gururangan

  committed
  dd552d8

  Copy full SHA for dd552d8

• update

  

  Suchin Gururangan

  committed
  1c9ff6f

  Copy full SHA for 1c9ff6f

• update

  

  Suchin Gururangan

  committed
  9e2aa30

  Copy full SHA for 9e2aa30

Commits on Dec 18, 2023

• update

  

  Suchin Gururangan

  committed
  5e78109

  Copy full SHA for 5e78109

• update

  

  Suchin Gururangan

  committed
  7ea0385

  Copy full SHA for 7ea0385

• update

  

  Suchin Gururangan

  committed
  7b68747

  Copy full SHA for 7b68747

• Merge branch 'main' into moe

  kernelmachine

  authored
  a50fed2

  Copy full SHA for a50fed2