Make context builder / base images arm64 compatible #1490
Conversation
nnarayen
force-pushed
the
nikhil/update-context-builder-arm
branch
6 times, most recently
from
e679265 to
24f86e6
Compare
| @@ -2,6 +2,16 @@ name: Build Context Builder Image | |||
|
|
|||
| on: | |||
| workflow_dispatch: | |||
| inputs: | |||
There was a problem hiding this comment.
Building arm64 images on a github runner is slow, so I wanted to keep the option of just building for amd64 fast for now
| @@ -59,6 +59,7 @@ jobs: | |||
| push: ${{ github.event_name != 'pull_request' }} | |||
| tags: baseten/truss-context-builder:v${{ needs.detect-version-changed.outputs.new_version }} | |||
| labels: ${{ steps.meta.outputs.labels }} | |||
| platforms: linux/amd64,linux/arm64 | |||
There was a problem hiding this comment.
Production context builders ideally are built for both, and since nobody is really waiting on these it's ok to be slower
| @@ -2,21 +2,32 @@ | |||
| # for creating docker build context out of a Truss. | |||
| # Build that image as: | |||
| # docker buildx build . -f context_builder.Dockerfile --platform=linux/amd64 -t baseten/truss-context-builder | |||
| FROM python:3.9-slim | |||
| FROM python:3.9-slim AS builder | |||
There was a problem hiding this comment.
Making this a multi stage build takes it from ~500mb to ~100mb
| @@ -1 +1,3 @@ | |||
| torch>=2.0.1 | |||
| blake3>=0.3.3 | |||
There was a problem hiding this comment.
Without the changes made in this PR (specifically to truss/templates/base.Dockerfile.jinja), this test would now fail on Macbooks
There was a problem hiding this comment.
What part inside the truss model exactly needs blake now?
There was a problem hiding this comment.
For this test nothing, but I just put it here as an example of a dependency that is trickier to build on arm64. In 'real' models, I believe it's the control patching flow
|
|
||
| RUN apt-get update \ | ||
| && apt-get install --yes --no-install-recommends curl tar \ | ||
| && apt-get install --yes --no-install-recommends curl build-essential golang-go \ |
There was a problem hiding this comment.
We add the tooling needing for arm64 images unconditionally, but since it's a multi stage image, these deps aren't needed in the final image
nnarayen
force-pushed
the
nikhil/update-context-builder-arm
branch
from
24f86e6 to
619c1a0
Compare
| @@ -1 +1,3 @@ | |||
| torch>=2.0.1 | |||
| blake3>=0.3.3 | |||
There was a problem hiding this comment.
What part inside the truss model exactly needs blake now?
nnarayen
force-pushed
the
nikhil/update-context-builder-arm
branch
from
619c1a0 to
ba7efb5
Compare
🚀 What
Changes our context builder + base images to be arm64 compatible. Modified a unit test that now runs locally on mac:
And verified in local dev/staging via context builder
0.9.71rc008that models can still build + run.Before multi stage
After multi stage
💻 How
🔬 Testing