Code

wip - Mg/pytorch source builds#506

Code

Draft

malcolmgreaves wants to merge 63 commits intomainNVIDIA/bionemo-framework:mainfrom mg/pytorch_source_buildsNVIDIA/bionemo-framework:mg/pytorch_source_buildsCopy head branch name to clipboard

+1,209-5

Commits

Commits on Dec 5, 2024

• wip

  malcolmgreaves

  committed
  52f1b4a

  Copy full SHA for 52f1b4a

• wip

  malcolmgreaves

  committed
  ac53484

  Copy full SHA for ac53484

• wip

  malcolmgreaves

  committed
  6b3837f

  Copy full SHA for 6b3837f

• wip

  malcolmgreaves

  committed
  967b97f

  Copy full SHA for 967b97f

• build python from source

  malcolmgreaves

  committed
  5f4003b

  Copy full SHA for 5f4003b

• fix

  malcolmgreaves

  committed
  ad3edd5

  Copy full SHA for ad3edd5

• try this to see if it caches better

  malcolmgreaves

  committed
  b60893a

  Copy full SHA for b60893a

• undo

  malcolmgreaves

  committed
  080a954

  Copy full SHA for 080a954

• use curl

  malcolmgreaves

  committed
  64b7e90

  Copy full SHA for 64b7e90

• needs curl

  malcolmgreaves

  committed
  9e3af13

  Copy full SHA for 9e3af13

• wip

  malcolmgreaves

  committed
  d8a7f8b

  Copy full SHA for d8a7f8b

• wip adding more dependencies

  malcolmgreaves

  committed
  bbb7a9c

  Copy full SHA for bbb7a9c

• full py version

  malcolmgreaves

  committed
  2fb633b

  Copy full SHA for 2fb633b

• wip

  malcolmgreaves

  committed
  ab33f3b

  Copy full SHA for ab33f3b

• wip

  malcolmgreaves

  committed
  989ec72

  Copy full SHA for 989ec72

• parse pyproject.toml for dependencies and put into unified requirements.txt file for install in compatibility images

  malcolmgreaves

  committed
  6233f70

  Copy full SHA for 6233f70

• wip

  malcolmgreaves

  committed
  8dab6b3

  Copy full SHA for 8dab6b3

• DEBIAN_FRONTEND=noninteractive

  malcolmgreaves

  committed
  4e3c113

  Copy full SHA for 4e3c113

• install pip

  malcolmgreaves

  committed
  a70dc7c

  Copy full SHA for a70dc7c

• fixed: install pip and symlink for normal names

  malcolmgreaves

  committed
  3956692

  Copy full SHA for 3956692

• wip

  malcolmgreaves

  committed
  c8d6029

  Copy full SHA for c8d6029

• wip

  malcolmgreaves

  committed
  0fa06eb

  Copy full SHA for 0fa06eb

• apex has wrong build deps

  malcolmgreaves

  committed
  6d3d800

  Copy full SHA for 6d3d800

• ubuntu 20.04, cuda 12.1.1, use right index for pip torch install

  malcolmgreaves

  committed
  f398dc1

  Copy full SHA for f398dc1

• set CUDA_HOME and install sub-package code

  malcolmgreaves

  committed
  f141d43

  Copy full SHA for f141d43

• cache pytorch instaall

  malcolmgreaves

  committed
  66084f2

  Copy full SHA for 66084f2

• wip

  malcolmgreaves

  committed
  e72c2d3

  Copy full SHA for e72c2d3

• wip

  malcolmgreaves

  committed
  e3d1182

  Copy full SHA for e3d1182

• fixing apex install...

  malcolmgreaves

  committed
  c951117

  Copy full SHA for c951117

• compute capability

  malcolmgreaves

  committed
  c4a3ffd

  Copy full SHA for c4a3ffd

• wip

  malcolmgreaves

  committed
  98fc4b7

  Copy full SHA for 98fc4b7

• fixing

  malcolmgreaves

  committed
  8417911

  Copy full SHA for 8417911

• wip

  malcolmgreaves

  committed
  d144495

  Copy full SHA for d144495

• upping cuda arch versions: I think bfloat16 is only supported at 8.0+

  malcolmgreaves

  committed
  23d3afb

  Copy full SHA for 23d3afb

• ensure correct cuda is on PATH and remove wrong ones

  malcolmgreaves

  committed
  4096b26

  Copy full SHA for 4096b26

• fix TE bug that prevented cuda 12.1.1 installation

  Show description for c343e13
  malcolmgreaves

  committed
  c343e13

  Copy full SHA for c343e13

• wip

  malcolmgreaves

  committed
  070960a

  Copy full SHA for 070960a

• wip

  malcolmgreaves

  committed
  8c8c5cf

  Copy full SHA for 8c8c5cf

• no pipefail

  malcolmgreaves

  committed
  785984a

  Copy full SHA for 785984a

• attempting to fix this install..

  malcolmgreaves

  committed
  ff322d7

  Copy full SHA for ff322d7

• install yq & get rid of bionemo-geometric

  malcolmgreaves

  committed
  f80a725

  Copy full SHA for f80a725

• wip

  malcolmgreaves

  committed
  731534a

  Copy full SHA for 731534a

• wip

  malcolmgreaves

  committed
  93e38c8

  Copy full SHA for 93e38c8

• wip

  malcolmgreaves

  committed
  b124d28

  Copy full SHA for b124d28

• wip

  malcolmgreaves

  committed
  9c263a2

  Copy full SHA for 9c263a2

• need xz

  malcolmgreaves

  committed
  ae827a6

  Copy full SHA for ae827a6

• Patch to fix CUDA 12.1.1 support in Transformer Engine w/o updating because they broke their API and there's no more te.pytorch module :(

  malcolmgreaves

  committed
  4b4c538

  Copy full SHA for 4b4c538

• fix git patch application

  malcolmgreaves

  committed
  e06ba17

  Copy full SHA for e06ba17

• don't need patch anymore...was using wrong commit

  malcolmgreaves

  committed
  aecccb0

  Copy full SHA for aecccb0

• max_jobs=-1 for all, declare ARG closer to use for improved caching

  malcolmgreaves

  committed
  4dd9980

  Copy full SHA for 4dd9980

• add missing overrides dep

  malcolmgreaves

  committed
  29e8870

  Copy full SHA for 29e8870

• Autocast down when not on ampere

  malcolmgreaves

  committed
  9fe3681

  Copy full SHA for 9fe3681

• add setuptools

  malcolmgreaves

  committed
  f6e25e5

  Copy full SHA for f6e25e5

• cuda 12.3 image build

  malcolmgreaves

  committed
  f4171d4

  Copy full SHA for f4171d4

• rename

  malcolmgreaves

  committed
  35c5145

  Copy full SHA for 35c5145

• wip

  malcolmgreaves

  committed
  fab385f

  Copy full SHA for fab385f

• wip

  malcolmgreaves

  committed
  3b52154

  Copy full SHA for 3b52154

• standardize

  malcolmgreaves

  committed
  0298a43

  Copy full SHA for 0298a43

• change base

  malcolmgreaves

  committed
  bf7a618

  Copy full SHA for bf7a618

• wip trying out pytorch build from source

  malcolmgreaves

  committed
  3b13926

  Copy full SHA for 3b13926

• torch and hf-inference dockerfiles

  malcolmgreaves

  committed
  f3a8aa0

  Copy full SHA for f3a8aa0

Commits on Dec 6, 2024

• wip

  malcolmgreaves

  committed
  aba036d

  Copy full SHA for aba036d

• wip

  malcolmgreaves

  committed
  1c6f7ed

  Copy full SHA for 1c6f7ed