Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Cache and Docker CI #142

Merged
merged 7 commits into from
Nov 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions .github/workflows/cont_int.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,21 +126,6 @@ jobs:
echo "PYTHONPATH=$(pwd):$PYTHONPATH" >> ~/.bashrc
echo "PATH=$(pwd):$PATH" >> ~/.bashrc
echo "export rmgpy_path=$(pwd)" >> ~/.bashrc


- name: Cache RMS
id: cache-rms
uses: actions/cache@v2
env:
CACHE_NUMBER: 0
with:
path: |
/usr/share/miniconda3/envs/rmg_env/share/julia/artifacts/
/usr/share/miniconda3/envs/rmg_env/share/julia/packages/
/usr/share/miniconda3/envs/rmg_env/share/julia/registries/
/usr/share/miniconda3/envs/rmg_env/share/julia/compiled/
key:
${{ runner.os }}-julia-${{ env.CACHE_NUMBER }}-rms

- name: Install RMS Julia
run: julia -e 'using Pkg; Pkg.add(PackageSpec(name="PyCall",rev="master"));Pkg.build("PyCall");Pkg.add(PackageSpec(name="ReactionMechanismSimulator",rev="main")); using ReactionMechanismSimulator;'
Expand Down
109 changes: 78 additions & 31 deletions .github/workflows/docker_build.yml
Original file line number Diff line number Diff line change
@@ -1,62 +1,109 @@
name: Docker Image Build and Push

#Ensure that top-level permissions are not set to write-all
#https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions


on:
schedule:
- cron: "0 0 */7 * *" # every 7 days
push:
branches:
- main
- main
pull_request:
types: [opened, synchronize, reopened]
branches:
- main
- main
paths:
- './Dockerfile'
- "./Dockerfile"

permissions:
contents: read
pull-requests: write

jobs:
ci-check:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: true
# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: false
swap-storage: false
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4.1.1
# Add steps for CI checks of the Dockerfile
- name: Login to Docker Hub
uses: docker/[email protected]
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/[email protected]
- name: Build RMG-Py
uses: docker/build-push-action@v2
uses: docker/build-push-action@v5.0.0
with:
context: .
file: ./devtools/Dockerfile_rmgpy
push: false
tags: ${{ secrets.DOCKERHUB_USERNAME }}/rmgpy:latest
- name: Build T3
uses: docker/build-push-action@v2
uses: docker/build-push-action@v5.0.0
with:
context: .
file: ./Dockerfile
push: false

tags: ${{ secrets.DOCKERHUB_USERNAME }}/thetandemtool-t3:latest

build-and-push:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Build and push RMG-Py
uses: docker/build-push-action@v2
with:
context: .
file: ./devtools/Dockerfile_rmgpy
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/rmgpy:latest
- name: Build and push T3
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/thetandemtool-t3:latest

- name: Checkout
uses: actions/[email protected]
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: true

# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: false
swap-storage: false
- name: Login to Docker Hub
uses: docker/[email protected]
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/[email protected]
- name: Build and push RMG-Py
uses: docker/[email protected]
with:
context: .
file: ./devtools/Dockerfile_rmgpy
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/rmgpy:latest
- name: Build and push T3
uses: docker/[email protected]
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ secrets.DOCKERHUB_USERNAME }}/thetandemtool-t3:latest
27 changes: 6 additions & 21 deletions .github/workflows/update-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ jobs:
run: |
cd RMG-Py
conda activate rmg_env
make clean
make
echo "PYTHONPATH=$(pwd):$PYTHONPATH" >> $GITHUB_ENV
echo "PATH=$(pwd):$PATH" >> $GITHUB_ENV
Expand All @@ -98,28 +99,12 @@ jobs:
- name: Install PyCall RMG_ENV
run: python -c "import julia; julia.install(); import diffeqpy; diffeqpy.install()"

- name: Cache RMS
id: cache-rms
uses: actions/cache@v2
env:
CACHE_NUMBER: 0
with:
path: |
/usr/share/miniconda3/envs/rmg_env/share/julia/artifacts/
/usr/share/miniconda3/envs/rmg_env/share/julia/packages/
/usr/share/miniconda3/envs/rmg_env/share/julia/registries/
/usr/share/miniconda3/envs/rmg_env/share/julia/compiled/
key:
${{ runner.os }}-julia-${{ env.CACHE_NUMBER }}-rms

- name: Install RMS Julia
run: julia -e 'using Pkg; Pkg.add(PackageSpec(name="ReactionMechanismSimulator",rev="main"));using ReactionMechanismSimulator'

- name: Link Python-JL
run: |
ln -sfn $(which python-jl) $(which python)
cd ..

run: julia -e 'using Pkg; Pkg.add(PackageSpec(name="PyCall",rev="master"));Pkg.build("PyCall");Pkg.add(PackageSpec(name="ReactionMechanismSimulator",rev="main")); using ReactionMechanismSimulator;'

- name: Install PyCall RMG_ENV
run: python -c "import julia; julia.install(); import diffeqpy; diffeqpy.install()"

- name: Install ARC
run: |
cd ARC
Expand Down
2 changes: 0 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
# Using the a base image of rmgpy:latest from DockerHub - Not the official version but a custom version that is smaller in size
FROM --platform=linux/amd64 laxzal/rmgpy:latest


USER rmguser


# Installing ARC
# Change directory to Code
WORKDIR /home/rmguser/Code
Expand Down
Loading