Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[no-relnote] Configure all GitHub actions as reusable workflow #915

Merged
merged 1 commit into from
Feb 12, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copyright 2025 NVIDIA CORPORATION
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: CI Pipeline

on:
push:
branches:
- "pull-request/[0-9]+"
- main
- release-*

jobs:
code-scanning:
uses: ./.github/workflows/code_scanning.yaml

variables:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.version.outputs.version }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the following work?

Suggested change
version: ${{ steps.version.outputs.version }}
version: ${{ github.sha.substring(0,8) }}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it doesn't

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a matter of interest, what error does it give? Do we have a reference of which functions are available to us? We do use startsWith, for example.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

steps:
- name: Generate Commit Short SHA
id: version
run: echo "version=$(echo $GITHUB_SHA | cut -c1-8)" >> "$GITHUB_OUTPUT"

golang:
uses: ./.github/workflows/golang.yaml

image:
uses: ./.github/workflows/image.yaml
needs: [variables, golang, code-scanning]
secrets: inherit
with:
version: ${{ needs.variables.outputs.version }}
build_multi_arch_images: ${{ github.ref_name == 'main' || startsWith(github.ref_name, 'release-') }}

e2e-test:
needs: [image, variables]
secrets: inherit
uses: ./.github/workflows/e2e.yaml
with:
version: ${{ needs.variables.outputs.version }}
5 changes: 1 addition & 4 deletions .github/workflows/code_scanning.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,14 @@
name: "CodeQL"

on:
workflow_call: {}
pull_request:
types:
- opened
- synchronize
branches:
- main
- release-*
push:
branches:
- main
- release-*

jobs:
analyze:
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@ name: End-to-end Tests

on:
workflow_call:
inputs: {}
inputs:
version:
required: true
type: string
secrets:
AWS_ACCESS_KEY_ID:
required: true
Expand Down Expand Up @@ -67,8 +70,8 @@ jobs:

- name: Run e2e tests
env:
IMAGE_NAME: ghcr.io/${LOWERCASE_REPO_OWNER}/container-toolkit
VERSION: ${COMMIT_SHORT_SHA}
IMAGE_NAME: ghcr.io/nvidia/container-toolkit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do move e2e.yaml to a central repo, we would have to have this as an input.

(Out of scope for this PR).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree

VERSION: ${{ inputs.version }}
SSH_KEY: ${{ secrets.AWS_SSH_KEY }}
E2E_SSH_USER: ${{ secrets.E2E_SSH_USER }}
E2E_SSH_HOST: ${{ steps.holodeck_public_dns_name.outputs.result }}
Expand Down
5 changes: 1 addition & 4 deletions .github/workflows/golang.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,14 @@
name: Golang

on:
workflow_call: {}
pull_request:
types:
- opened
- synchronize
branches:
- main
- release-*
push:
branches:
- main
- release-*

jobs:
check:
Expand Down
44 changes: 20 additions & 24 deletions .github/workflows/image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,14 @@
name: image

on:
push:
branches:
- "pull-request/[0-9]+"
- main
- release-*
workflow_call:
inputs:
version:
required: true
type: string
build_multi_arch_images:
required: true
type: string

jobs:
packages:
Expand All @@ -46,20 +49,25 @@ jobs:
- ispr: true
target: centos8-ppc64le
fail-fast: false

steps:
- uses: actions/checkout@v4
name: Check out code

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:master

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: build ${{ matrix.target }} packages
run: |
sudo apt-get install -y coreutils build-essential sed git bash make
echo "Building packages"
./scripts/build-packages.sh ${{ matrix.target }}

- name: 'Upload Artifacts'
uses: actions/upload-artifact@v4
with:
Expand All @@ -84,24 +92,15 @@ jobs:
steps:
- uses: actions/checkout@v4
name: Check out code
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What version is checked out here? Should we align this with the short sha?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would make sense in a follow-up PR. The defaults are ok, but moving forward when writing more reusable workflows, it will make more sense

https://github.com/actions/checkout?tab=readme-ov-file#usage

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shure. Could you please clarify which verison is checked out by default.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    # The branch, tag or SHA to checkout. When checking out the repository that
    # triggered a workflow, this defaults to the reference or SHA for that event.
    # Otherwise, uses the default branch.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is, if this is triggered by a workflow call, is the SHA that is checked out the current version in the PR, or the SHA of the default branch.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the SHA of the commit that triggered the action, unregarding of the branch (that's why copy-pr-bot works)

- name: Calculate build vars
id: vars
run: |
echo "COMMIT_SHORT_SHA=${GITHUB_SHA:0:8}" >> $GITHUB_ENV
echo "LOWERCASE_REPO_OWNER=$(echo "${GITHUB_REPOSITORY_OWNER}" | awk '{print tolower($0)}')" >> $GITHUB_ENV

BUILD_MULTI_ARCH_IMAGES="true"
if [[ "${{ matrix.ispr }}" == "true" ]]; then
BUILD_MULTI_ARCH_IMAGES="false"
fi
echo "PUSH_ON_BUILD=true" >> $GITHUB_ENV
echo "BUILD_MULTI_ARCH_IMAGES=${BUILD_MULTI_ARCH_IMAGES}" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:master

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Get built packages
uses: actions/download-artifact@v4
with:
Expand All @@ -115,16 +114,13 @@ jobs:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build image
env:
IMAGE_NAME: ghcr.io/${LOWERCASE_REPO_OWNER}/container-toolkit
VERSION: ${COMMIT_SHORT_SHA}
IMAGE_NAME: ghcr.io/nvidia/container-toolkit
VERSION: ${{ inputs.version }}
PUSH_ON_BUILD: "true"
BUILD_MULTI_ARCH_IMAGES: ${{ inputs.build_multi_arch_images }}
run: |
echo "${VERSION}"
make -f deployments/container/Makefile build-${{ matrix.dist }}

test:
name: End-to-end Tests
uses: ./.github/workflows/e2e.yaml
secrets: inherit
needs: image