Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker buildx bake experiment for builds #705

Closed
wants to merge 37 commits into from
Closed

Docker buildx bake experiment for builds #705

wants to merge 37 commits into from

Conversation

costrouc
Copy link
Member

@costrouc costrouc commented Mar 12, 2024

I have attempted to avoid any tricks or advanced features of hcl within bake. There are many optimizations we could make from here. Most importantly this does not break anything!

Docker buildx bake has several advantages:

  • allows us to represent the dependencies between dockerfiles
  • performs parallel builds (at the layer level)
  • allows us to cleanly express the build matricies that each product has
  • have a single maintainable place for managing all docker build configuration

This PR builds everything within a single runner and no complex github action workflows. All is captured within the docker-bake.hcl. It is possible to easily exclude certain builds etc and only build certain ones.

This builds:
- product-base
- product-base-pro
- connect
- connect-content-init
- package-manager
- r-session-complete
- workbench
- workbench-for-google-cloud-workstations
- workbench-for-microsoft-azure-ml

@costrouc
Copy link
Member Author

The action that matters is Build DockerImages with Bake

@atheriel
Copy link
Collaborator

This is interesting, I hadn't heard of docker buildx bake.

@costrouc costrouc requested review from a team as code owners April 3, 2024 15:40
@costrouc costrouc requested review from tylfin and jmwoliver and removed request for a team April 3, 2024 15:40
@dbkegley
Copy link
Contributor

dbkegley commented Apr 3, 2024

FWIW we did a bunch of testing with docker buildx bake for Connect and found that the inline layer caching was unreliable for multi-stage builds. We think it was related to this issue which was just recently fixed. We have not re-tested yet though.

https://github.com/rstudio/connect/issues/23961#issuecomment-1613775719

@ianpittwood ianpittwood marked this pull request as draft April 5, 2024 18:15
@ianpittwood
Copy link
Collaborator

FWIW we did a bunch of testing with docker buildx bake for Connect and found that the inline layer caching was unreliable for multi-stage builds. We think it was related to this issue which was just recently fixed. We have not re-tested yet though.

rstudio/connect#23961 (comment)

We weren't previously using caching for these builds so I'm not concerned with anything like that at the moment. Thanks for the heads up though!

@ianpittwood ianpittwood force-pushed the feat-buildx branch 2 times, most recently from a90f046 to 8174e77 Compare April 5, 2024 18:57
@ianpittwood
Copy link
Collaborator

Merged in #718

@bschwedler bschwedler deleted the feat-buildx branch October 30, 2024 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants