Skip to content

Commit ab96d91

Browse files
andistormandistorm
committed
Add CI workflow for building new documentation
Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> fix f-string Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> Strip install in ci to save space Signed-off-by: Kai-Uwe Hermann <kai-uwe.hermann@pionix.de> Enable MinSizeRel build if Debug is not explicitly requested during test build Signed-off-by: Kai-Uwe Hermann <kai-uwe.hermann@pionix.de> Rework trailbook configuration * Remove arg INSTANCE_NAME from add_trailbook() * Add CMake var TRAILBOOK_everest_INSTANCE_NAME to allow configuring the instance name again * Add check to not overwrite existing instance accidently * Add CMAKE var TRAILBOOK_OVERWRITE_EXISTING_INSTANCE to allow forcing overwrite of instance Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> Add --exclude-dir parameter to create_snapshot script Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> Use --exclude-dir arg for create_snapshot script Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> add debug output Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> Revert "add debug output" This reverts commit 1639086. do not create snapshot.yaml Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> fix staticpages extension for sphinx v7 Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> update artifact dir Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> minor changes Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> activate snapshot.yml creation again Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> Forward SSH KEY Signed-off-by: Andreas Heinrich <andreas.heinrich@rwth-aachen.de> squashed
1 parent bbb8f17 commit ab96d91

20 files changed

Lines changed: 1167 additions & 122 deletions

‎.ci/build-kit/scripts/build_docs.sh‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/sh
2+
3+
TRAILBOOK_everest_IS_RELEASE=${TRAILBOOK_everest_IS_RELEASE:-"OFF"}
4+
TRAILBOOK_everest_INSTANCE_NAME=${TRAILBOOK_everest_INSTANCE_NAME:-"nightly"}
5+
TRAILBOOK_everest_OVERWRITE_EXISTING_INSTANCE=${TRAILBOOK_everest_OVERWRITE_EXISTING_INSTANCE:-"OFF"}
6+
7+
mkdir -p ~/.ssh
8+
ssh-keyscan github.com >> ~/.ssh/known_hosts
9+
chmod 600 ~/.ssh/known_hosts
10+
11+
cmake \
12+
-B "$EXT_MOUNT/build" \
13+
-S "$EXT_MOUNT/source" \
14+
-G Ninja \
15+
-D EVC_ENABLE_CCACHE=ON \
16+
-D EVEREST_ENABLE_COMPILE_WARNINGS=ON \
17+
-D EVEREST_BUILD_DOCS=ON \
18+
-D TRAILBOOK_everest_DOWNLOAD_ALL_VERSIONS=ON \
19+
-D TRAILBOOK_everest_IS_RELEASE=$TRAILBOOK_everest_IS_RELEASE \
20+
-D TRAILBOOK_everest_INSTANCE_NAME="$TRAILBOOK_everest_INSTANCE_NAME" \
21+
-D TRAILBOOK_everest_OVERWRITE_EXISTING_INSTANCE=$TRAILBOOK_everest_OVERWRITE_EXISTING_INSTANCE
22+
retVal=$?
23+
if [ $retVal -ne 0 ]; then
24+
echo "Configuring failed with return code $retVal"
25+
exit $retVal
26+
fi
27+
28+
ninja -C "$EXT_MOUNT/build" trailbook_everest
29+
retVal=$?
30+
if [ $retVal -ne 0 ]; then
31+
echo "Compiling failed with return code $retVal"
32+
exit $retVal
33+
fi

.github/workflows/bazel_build_and_test.yaml renamed to .github/workflows/job_bazel-build-test.yaml

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,16 @@
1-
name: Bazel Build
1+
name: Bazel Build And Test
22
run-name: ${{ github.actor }} is building with bazel
33
on:
4-
pull_request: {}
5-
merge_group: {}
4+
workflow_call:
5+
inputs:
6+
runner:
7+
description: 'Which runner to use'
8+
required: false
9+
default: 'ubuntu-24.04'
10+
type: string
611
jobs:
712
bazel-build-and-test:
8-
runs-on: ubuntu-22.04
13+
runs-on: ${{ inputs.runner }}
914
steps:
1015
- run: echo branch name is ${{ github.ref }}
1116
- name: Checkout

‎.github/workflows/job_build-build-kit.yml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
name: Build the build-kit
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
runner:
7+
description: 'Which runner to use'
8+
required: false
9+
default: 'ubuntu-24.04'
10+
type: string
11+
build_kit_docker_directory:
12+
description: 'Directory in the repository where the build kit Dockerfile is located'
13+
required: false
14+
default: '.ci/build-kit/docker'
15+
type: string
16+
base_image_tag_everest_ci:
17+
description: 'The tag of the everest-ci base image to use for building the build-kit'
18+
required: true
19+
type: string
20+
outputs:
21+
build_kit_artifact_name:
22+
description: 'The name of the build-kit artifact'
23+
value: ${{ jobs.build-build-kit.outputs.build_kit_artifact_name }}
24+
build_kit_image_tag:
25+
description: 'The tag of the built build-kit image'
26+
value: ${{ jobs.build-build-kit.outputs.build_kit_image_tag }}
27+
28+
jobs:
29+
build-build-kit:
30+
name: Build the build-kit
31+
runs-on: ${{ inputs.runner }}
32+
env:
33+
BUILD_KIT_ARTIFACT_NAME: build-kit
34+
BUILD_KIT_IMAGE_NAME: local/build-kit-${{ github.event.repository.name }}
35+
BUILD_ARGS: |
36+
BASE_IMAGE_TAG=${{ inputs.base_image_tag_everest_ci }}
37+
outputs:
38+
build_kit_image_tag: ${{ steps.set-outputs.outputs.tag }}
39+
build_kit_artifact_name: ${{ env.BUILD_KIT_ARTIFACT_NAME }}
40+
steps:
41+
- name: Checkout Dockerfile
42+
uses: actions/checkout@v4
43+
with:
44+
repository: ${{ github.repository }}
45+
path: source
46+
ref: ${{ github.ref }}
47+
token: ${{ github.token}}
48+
fetch-depth: 0
49+
- name: Docker Meta
50+
id: meta
51+
uses: docker/metadata-action@v5
52+
with:
53+
images: ${{ env.BUILD_KIT_IMAGE_NAME }}
54+
sep-tags: ","
55+
- name: Setup Docker buildx
56+
uses: docker/setup-buildx-action@v3
57+
- name: Build
58+
uses: docker/build-push-action@v6
59+
with:
60+
context: source/${{ inputs.build_kit_docker_directory }}
61+
push: false
62+
cache-from: type=gha
63+
cache-to: type=gha,mode=max
64+
build-args: ${{ env.BUILD_ARGS }}
65+
tags: ${{ steps.meta.outputs.tags }}
66+
labels: ${{ steps.meta.outputs.labels }}
67+
outputs: type=docker,dest=build-kit.tar
68+
- name: Upload build-kit image
69+
uses: actions/upload-artifact@v4
70+
with:
71+
name: ${{ env.BUILD_KIT_ARTIFACT_NAME }}
72+
path: build-kit.tar
73+
- name: Set output tag
74+
id: set-outputs
75+
shell: python3 {0}
76+
run: |
77+
import os
78+
tags = "${{ steps.meta.outputs.tags }}".split(",")
79+
if len(tags) == 0:
80+
print("No tags found!❌")
81+
exit(1)
82+
tag = tags[0]
83+
with open(os.environ["GITHUB_OUTPUT"], "a") as f:
84+
f.write(f"tag={tag}\n")
85+
print(f"Set tag={tag}")

‎.github/workflows/job_build-cmake-gcc.yml‎

Lines changed: 245 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
name: Build with CMake and GCC
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
runner:
7+
description: 'Which runner to use'
8+
required: false
9+
default: 'ubuntu-24.04'
10+
type: string
11+
ref_everest_ci:
12+
description: 'The reference of the everest-ci repository to checkout'
13+
required: true
14+
type: string
15+
is_fork:
16+
description: 'Whether the current repository is a fork'
17+
required: true
18+
type: string
19+
build-kit-artifact-name:
20+
description: 'The name of the build-kit artifact to download'
21+
required: true
22+
type: string
23+
build_kit_image_tag:
24+
description: 'The tag of the build-kit image to use for building the project'
25+
required: true
26+
type: string
27+
build_kit_scripts_directory:
28+
description: 'Directory in the repository where the build kit scripts are located'
29+
required: false
30+
default: '.ci/build-kit/scripts'
31+
type: string
32+
ctest_report_path:
33+
description: 'The path to the ctest report, relative to the github workspace'
34+
required: false
35+
default: 'build/Testing/Temporary/LastTest.log'
36+
type: string
37+
coverage_report_path:
38+
description: 'The path to the coverage report, relative to the github workspace'
39+
required: false
40+
default: 'build/gcovr-coverage'
41+
type: string
42+
coverage_xml_path:
43+
description: 'The path to the coverage xml, relative to the github workspace'
44+
required: false
45+
default: 'build/coverage.xml'
46+
type: string
47+
artifact_deploy_target_repo:
48+
description: 'Repository to deploy artifacts to'
49+
required: true
50+
type: string
51+
wheels_path:
52+
description: 'The path to the wheels directory, relative to the github workspace'
53+
required: false
54+
default: 'wheels'
55+
type: string
56+
secrets:
57+
coverage_deploy_token:
58+
description: 'The token to use to deploy the coverage report'
59+
required: true
60+
SA_GITHUB_SSH_KEY:
61+
description: 'The ssh key to use for git operations'
62+
required: false
63+
outputs:
64+
ctest_report_artifact_name:
65+
description: 'The name of the ctest report artifact uploaded'
66+
value: ${{ jobs.build-cmake-gcc.outputs.ctest_report_artifact_name }}
67+
coverage_report_artifact_name:
68+
description: 'The name of the coverage report artifact uploaded'
69+
value: ${{ jobs.build-cmake-gcc.outputs.coverage_report_artifact_name }}
70+
coverage_xml_artifact_name:
71+
description: 'The name of the coverage xml artifact uploaded'
72+
value: ${{ jobs.build-cmake-gcc.outputs.coverage_xml_artifact_name }}
73+
dist_artifact_name:
74+
description: 'The name of the dist artifact uploaded'
75+
value: ${{ jobs.build-cmake-gcc.outputs.dist_artifact_name }}
76+
wheels_artifact_name:
77+
description: 'The name of the wheels artifact uploaded'
78+
value: ${{ jobs.build-cmake-gcc.outputs.wheels_artifact_name }}
79+
80+
jobs:
81+
build-cmake-gcc:
82+
name: Build with CMake and GCC, Unit Tests and Install
83+
runs-on: ${{ inputs.runner }}
84+
env:
85+
CTEST_REPORT_ARTIFACT_NAME: ctest-report
86+
COVERAGE_REPORT_ARTIFACT_NAME: coverage-report
87+
COVERAGE_XML_ARTIFACT_NAME: coverage-xml
88+
DIST_ARTIFACT_NAME: dist
89+
WHEELS_ARTIFACT_NAME: wheels
90+
BUILD_KIT_IMAGE: ${{ inputs.build_kit_image_tag }}
91+
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
92+
outputs:
93+
ctest_report_artifact_name: ${{ env.CTEST_REPORT_ARTIFACT_NAME }}
94+
coverage_report_artifact_name: ${{ env.COVERAGE_REPORT_ARTIFACT_NAME }}
95+
coverage_xml_artifact_name: ${{ env.COVERAGE_XML_ARTIFACT_NAME }}
96+
dist_artifact_name: ${{ env.DIST_ARTIFACT_NAME }}
97+
wheels_artifact_name: ${{ env.WHEELS_ARTIFACT_NAME }}
98+
steps:
99+
- name: Setup SSH Agent, optional with SSH key
100+
env:
101+
SSH_KEY: ${{ secrets.SA_GITHUB_SSH_KEY }}
102+
run: |
103+
ssh-agent -a $SSH_AUTH_SOCK > /dev/null
104+
if [ -z "${{ env.SSH_KEY }}" ]; then
105+
echo "No SSH key provided, skipping SSH key setup"
106+
exit 0
107+
fi
108+
mkdir -p ~/.ssh
109+
echo "${{ env.SSH_KEY }}" > ~/.ssh/id_ed25519
110+
chmod 600 ~/.ssh/id_ed25519
111+
# Check if github.com is already in known_hosts, if not, add it
112+
if ! grep -q "^github.com " ~/.ssh/known_hosts; then
113+
ssh-keyscan github.com >> ~/.ssh/known_hosts
114+
fi
115+
ssh-add ~/.ssh/id_ed25519
116+
- name: Checkout local github actions
117+
uses: actions/checkout@v4
118+
with:
119+
repository: everest/everest-ci
120+
ref: ${{ inputs.ref_everest_ci }}
121+
path: everest-ci
122+
- name: Format branch name for cache key
123+
run: |
124+
BRANCH_NAME_FOR_CACHE="${GITHUB_REF_NAME//-/_}"
125+
echo "branch_name_for_cache=${BRANCH_NAME_FOR_CACHE}" >> "$GITHUB_ENV"
126+
- name: Setup cache
127+
uses: actions/cache@v4
128+
with:
129+
path: cache
130+
key: compile-${{ env.branch_name_for_cache }}-${{ github.sha }}
131+
restore-keys: |
132+
compile-${{ env.branch_name_for_cache }}-
133+
compile-
134+
- name: Checkout repository
135+
uses: actions/checkout@v4
136+
with:
137+
path: source
138+
- name: Setup run scripts
139+
run: |
140+
mkdir scripts
141+
rsync -a source/${{ inputs.build_kit_scripts_directory }}/ scripts
142+
- name: Download build-kit image
143+
uses: actions/download-artifact@v5
144+
with:
145+
name: ${{ inputs.build-kit-artifact-name }}
146+
- name: Load build-kit image
147+
run: |
148+
docker load -i build-kit.tar
149+
docker image tag ${{ env.BUILD_KIT_IMAGE }} build-kit
150+
- name: Compile
151+
run: |
152+
docker run \
153+
--mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
154+
--env SSH_AUTH_SOCK=/ssh-agent \
155+
--volume "${{ github.workspace }}:/ext" \
156+
--name compile-container \
157+
build-kit run-script compile
158+
- name: Run unit tests
159+
id: run_unit_tests
160+
run: |
161+
docker run \
162+
--mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
163+
--env SSH_AUTH_SOCK=/ssh-agent \
164+
--volume "${{ github.workspace }}:/ext" \
165+
--name unit-test-container \
166+
build-kit run-script run_unit_tests
167+
- name: Archive test results
168+
if: ${{ always() && (steps.run_unit_tests.outcome == 'success' || steps.run_unit_tests.outcome == 'failure') }}
169+
uses: actions/upload-artifact@v4
170+
with:
171+
if-no-files-found: error
172+
name: ${{ env.CTEST_REPORT_ARTIFACT_NAME }}
173+
path: ${{ inputs.ctest_report_path }}
174+
# - name: Run coverage
175+
# id: run_coverage
176+
# run: |
177+
# docker run \
178+
# --mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
179+
# --env SSH_AUTH_SOCK=/ssh-agent \
180+
# --volume "${{ github.workspace }}:/ext" \
181+
# --name coverage-container \
182+
# build-kit run-script run_coverage
183+
# - name: Archive coverage report
184+
# if: ${{ always() && (steps.run_coverage.outcome == 'success' || steps.run_coverage.outcome == 'failure') }}
185+
# uses: actions/upload-artifact@v4
186+
# with:
187+
# if-no-files-found: error
188+
# name: ${{ env.COVERAGE_REPORT_ARTIFACT_NAME }}
189+
# path: ${{ inputs.coverage_report_path }}
190+
# - name: Archive coverage xml
191+
# if: ${{ always() && (steps.run_coverage.outcome == 'success' || steps.run_coverage.outcome == 'failure') }}
192+
# uses: actions/upload-artifact@v4
193+
# with:
194+
# if-no-files-found: error
195+
# name: ${{ env.COVERAGE_XML_ARTIFACT_NAME }}
196+
# path: ${{ inputs.coverage_xml_path}}
197+
# - name: Deploy html coverage report
198+
# id: deploy_coverage_report
199+
# if: ${{ always() && ( steps.run_coverage.outcome == 'success' || steps.run_coverage.outcome == 'failure' ) && inputs.is_fork == 'false' }}
200+
# uses: ./everest-ci/github-actions/deploy-ci-artifact
201+
# with:
202+
# target_repo: ${{ inputs.artifact_deploy_target_repo }}
203+
# github_token: ${{ secrets.coverage_deploy_token }}
204+
# artifact_name: coverage-report
205+
# artifact_directory: ${{ inputs.coverage_report_path }}
206+
# - name: Write summary coverage
207+
# if: ${{ always() && (steps.run_coverage.outcome == 'success' || steps.run_coverage.outcome == 'failure') }}
208+
# run: |
209+
# echo "Coverage report deployed to: [everest.github.io](https://everest.github.io/${{ steps.deploy_coverage_report.outputs.deployed_path }})" >> $GITHUB_STEP_SUMMARY
210+
- name: Create dist
211+
id: create_dist
212+
run: |
213+
docker run \
214+
--mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
215+
--env SSH_AUTH_SOCK=/ssh-agent \
216+
--volume "${{ github.workspace }}:/ext" \
217+
--name dist-container \
218+
build-kit run-script install
219+
- name: Tar dist dir and keep permissions
220+
if: ${{ always() && (steps.create_dist.outcome == 'success' || steps.create_dist.outcome == 'failure') }}
221+
run: |
222+
tar -czf dist.tar.gz dist
223+
- name: Upload dist artifact
224+
if: ${{ always() && (steps.create_dist.outcome == 'success' || steps.create_dist.outcome == 'failure') }}
225+
uses: actions/upload-artifact@v4.6.2
226+
with:
227+
if-no-files-found: error
228+
path: dist.tar.gz
229+
name: ${{ env.DIST_ARTIFACT_NAME }}
230+
- name: Create wheels
231+
id: create_wheels
232+
run: |
233+
docker run \
234+
--mount type=bind,source=$SSH_AUTH_SOCK,target=/ssh-agent \
235+
--env SSH_AUTH_SOCK=/ssh-agent \
236+
--volume "${{ github.workspace }}:/ext" \
237+
--name wheels-container \
238+
build-kit run-script install_wheels
239+
- name: Upload wheels artifact
240+
if: ${{ always() && (steps.create_wheels.outcome == 'success' || steps.create_wheels.outcome == 'failure') }}
241+
uses: actions/upload-artifact@v4.6.2
242+
with:
243+
if-no-files-found: error
244+
path: ${{ inputs.wheels_path }}
245+
name: ${{ env.WHEELS_ARTIFACT_NAME }}

0 commit comments

Comments
 (0)