Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: caching #657

Merged
merged 55 commits into from
Aug 13, 2024
Merged
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
c1a8bc8
ci: enable cache
ggera Jun 13, 2024
0786a36
ci: speedup pipeline with needs
ggera Jun 14, 2024
10bb9be
ci: test try-runtime with github actions
ggera Jun 14, 2024
2d1d7ed
ci: Add self-hosted runner
ggera Jun 20, 2024
c492ccc
ci: run jobs in container
ggera Jun 20, 2024
8c24d38
fix: github action
ggera Jun 20, 2024
e6738b6
ci: use DinD
ggera Jun 20, 2024
d1e1adb
ci: try-runtime gh action
ggera Jun 27, 2024
506a2f0
ci: upgrade caching version
ggera Jun 27, 2024
3681ced
ci: integration tests
ggera Jun 27, 2024
c35236e
ci: add tests gh action
ggera Jun 27, 2024
5b42fff
ci: srtool build
ggera Jun 28, 2024
42348a9
ci: tirgger on workflow_dispatch
ggera Jul 1, 2024
4c0ba3e
ci: use default cargo location
ggera Jul 1, 2024
225b872
Merge branch 'develop' of github.com:KILTprotocol/mashnet-node into c…
ggera Jul 18, 2024
dd91860
ci: address PR comments
ggera Jul 24, 2024
60be123
Merge branch 'develop' into ci-cache
ggera Jul 24, 2024
37b5ce1
Merge branch 'develop' of github.com:KILTprotocol/mashnet-node into c…
ggera Jul 24, 2024
95d1402
Merge branch 'ci-cache' of github.com:KILTprotocol/mashnet-node into …
ggera Jul 24, 2024
7377a4e
ci: prevent cancel-in-progress
ggera Jul 24, 2024
d453946
chore: merge develop
ggera Jul 29, 2024
3faf1d4
chore: resolve PR comments
ggera Jul 29, 2024
f15cbfd
Merge branch 'develop' of github.com:KILTprotocol/mashnet-node into c…
ggera Jul 30, 2024
a27face
chore: merge develop
ggera Jul 30, 2024
3c0bab3
ci-skip-rust
ggera Jul 30, 2024
7982893
chore: test skip
ggera Jul 30, 2024
a04f9ae
Merge branch 'develop' of github.com:KILTprotocol/mashnet-node into c…
ggera Jul 30, 2024
39a6191
chore: test ci-skip-rust
ggera Jul 30, 2024
7eb9671
ci-skip-rust;ci-skip-integration-tests
ggera Jul 30, 2024
1f87ef7
ci-skip-rust;ci-skip-integration-tests
ggera Jul 30, 2024
6a14c76
chore: test maximise build space
ggera Aug 1, 2024
eecdd08
ci-skip-integration-tests
ggera Aug 1, 2024
d5f972a
ci-skip-rust;ci-skip-integration-tests
ggera Aug 1, 2024
b925762
ci-skip-rust;ci-skip-integration-tests
ggera Aug 1, 2024
874802f
ci-skip-rust;ci-skip-integration-tests
ggera Aug 1, 2024
74a39ef
ci-skip-rust;ci-skip-integration-tests
ggera Aug 1, 2024
9a2417f
chore: commit message is not available on PR
ggera Aug 1, 2024
8419714
add check skip flags
ggera Aug 6, 2024
b2917a8
address PR comments
ggera Aug 7, 2024
af9c1a9
chore: formatting
ggera Aug 8, 2024
4a17731
chore: use actions/setup-node
ggera Aug 8, 2024
7ceefa6
ci-skip-rust
ggera Aug 8, 2024
bb1e606
ci-skip-rust
ggera Aug 8, 2024
499b0c6
ci-skip-rust
ggera Aug 8, 2024
6d71c3d
ci-skip-rust
ggera Aug 8, 2024
2e2dfa2
ci-skip-rust
ggera Aug 8, 2024
b35afb1
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
1f90326
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
980854e
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
f74de15
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
635f8dc
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
9a4fcc2
ci-skip-rust;ci-skip-integration-tests
ggera Aug 8, 2024
5f3baf1
chore: minor change
ggera Aug 12, 2024
749ca69
docs: Add GitHub docs autogen (#684)
ChrisChinchilla Aug 13, 2024
d8bdf80
Include metadata in artifact
ggera Aug 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/check-fmt.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Check fmt

on:
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true

jobs:
get-head-commit-msg:
runs-on: ubuntu-latest
outputs:
head-commit-msg: ${{ steps.step-head-commit.outputs.headCommitMsg }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Get Head Commit Message
id: step-head-commit
run: echo "headCommitMsg=$(git show -s --format=%s)" >> "$GITHUB_OUTPUT"

check-fmt:
runs-on: ubuntu-latest
needs: get-head-commit-msg
if: ${{ !contains(needs.get-head-commit-msg.outputs.head-commit-msg, 'ci-skip-rust') }}
container:
image: paritytech/ci-unified:bullseye-1.74.0
steps:
- uses: actions/checkout@v4

- name: Run cargo fmt
run: cargo fmt -- --check

- name: Run taplo
run: taplo fmt --check
55 changes: 55 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: Integration tests
ntn-x2 marked this conversation as resolved.
Show resolved Hide resolved

on:
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true

jobs:
get-head-commit-msg:
ntn-x2 marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
outputs:
head-commit-msg: ${{ steps.step-head-commit.outputs.headCommitMsg }}

steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Get Head Commit Message
id: step-head-commit
run: echo "headCommitMsg=$(git show -s --format=%s)" >> "$GITHUB_OUTPUT"

integration-tests:
runs-on: ubuntu-latest
needs: get-head-commit-msg
if: ${{ !contains(needs.get-head-commit-msg.outputs.head-commit-msg, 'ci-skip-integration-tests') }}
env:
ntn-x2 marked this conversation as resolved.
Show resolved Hide resolved
working-dir: ./integration-tests/chopsticks
CI: "true"
defaults:
run:
working-directory: ${{ env.working-dir }}
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
node-version-file: "${{ env.working-dir }}/.nvmrc"

- name: Install dependencies
run: yarn --immutable

- name: Type Checking
run: yarn ts-check

- name: Linting
run: yarn lint

- name: Test Suite
run: yarn test:CI

79 changes: 79 additions & 0 deletions .github/workflows/srtool-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Srtool build

on:
push:
branches:
- develop
- master
- 'refs/tags/[0-9]+.[0-9]+.[0-9]+*'

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true

env:
SUBWASM_VERSION: v0.21.3

jobs:
build-wasm:
runs-on: ubuntu-latest

strategy:
matrix:
runtime: [peregrine, spiritnet]

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Srtool build
id: srtool_build
uses: chevdor/[email protected]
env:
PARACHAIN_PALLET_ID: "0x50"
AUTHORIZE_UPGRADE_PREFIX: "0x02"
AUTHORIZE_UPGRADE_CHECK_VERSION: "true"
with:
chain: ${{ matrix.runtime }}
runtime_dir: runtimes/${{ matrix.runtime }}

- name: Summary
run: |
echo '${{ steps.srtool_build.outputs.json }}' | jq > ${{ matrix.runtime }}-srtool-digest.json
cat ${{ matrix.runtime }}-srtool-digest.json
echo "Compressed Runtime: ${{ steps.srtool_build.outputs.wasm_compressed }}"

- name: Install subwasm
run: |
wget https://github.com/chevdor/subwasm/releases/download/${{ env.SUBWASM_VERSION }}/subwasm_linux_amd64_${{ env.SUBWASM_VERSION }}.deb
sudo dpkg -i subwasm_linux_amd64_${{ env.SUBWASM_VERSION }}.deb
subwasm --version

- name: Show Runtime information
shell: bash
run: |
subwasm info --json ${{ steps.srtool_build.outputs.wasm_compressed }} > ${{ matrix.runtime }}-compressed-info.json

- name: Extract the metadata
shell: bash
run: |
subwasm meta --format=json+scale ${{ steps.srtool_build.outputs.wasm_compressed }}

- name: Check the metadata diff
shell: bash
run: |
subwasm get -o ${{ matrix.runtime }}-live.wasm wss://${{ matrix.runtime }}.kilt.io
subwasm diff --no-color ${{ matrix.runtime }}-live.wasm ${{ steps.srtool_build.outputs.wasm_compressed }} || \
echo "Subwasm call failed, check the logs. This is likely because ${{ matrix.runtime }} is not known by subwasm" | \
tee ${{ matrix.runtime }}-diff.txt

- name: Archive Subwasm results
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.runtime }}-runtime
path: |
${{ matrix.runtime }}-compressed-info.json
${{ matrix.runtime }}-diff.txt
${{ steps.srtool_build.outputs.wasm_compressed }}
${{ matrix.runtime }}-srtool-digest.json

65 changes: 65 additions & 0 deletions .github/workflows/try-runtime.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Try-runtime

on:
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true

env:
TRY_RUNTIME_CLI_VERSION_TAG: v0.7.0
CARGO_HOME: ./.cargo

jobs:
get-head-commit-msg:
runs-on: ubuntu-latest
outputs:
head-commit-msg: ${{ steps.step-head-commit.outputs.headCommitMsg }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Get Head Commit Message
id: step-head-commit
run: echo "headCommitMsg=$(git show -s --format=%s)" >> "$GITHUB_OUTPUT"


test-try-runtime:
strategy:
matrix:
runtime: [peregrine, spiritnet]
runs-on: ubuntu-latest
needs: get-head-commit-msg
if: ${{ !contains(needs.get-head-commit-msg.outputs.head-commit-msg, 'ci-skip-rust') }}
container:
image: paritytech/ci-unified:bullseye-1.74.0
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up caching for Cargo
uses: actions/cache@v4
with:
path: ${{ secrets.CARGO_CACHE_PATH }}
key: cargo-try-runtime-${{ matrix.runtime }}-${{ github.ref_name }}-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
cargo-try-runtime-${{ matrix.runtime }}-${{ github.ref_name }}-
- name: Run try-runtime
run: |
echo "Running ${{ matrix.runtime }} runtime migration check"
echo "---------- Downloading try-runtime CLI ----------"
curl -sL https://github.com/paritytech/try-runtime-cli/releases/download/${{ env.TRY_RUNTIME_CLI_VERSION_TAG }}/try-runtime-x86_64-unknown-linux-musl -o try-runtime
chmod +x ./try-runtime
echo "Using try-runtime-cli version:"
./try-runtime --version
echo "---------- Building ${{ matrix.runtime }} runtime ----------"
cargo build --release --locked -p ${{ matrix.runtime }}-runtime --features try-runtime
ntn-x2 marked this conversation as resolved.
Show resolved Hide resolved
echo "---------- Executing on-runtime-upgrade for ${{ matrix.runtime }} ----------"
./try-runtime --runtime ./target/release/wbuild/${{ matrix.runtime }}-runtime/${{ matrix.runtime }}_runtime.compact.compressed.wasm \
on-runtime-upgrade --disable-spec-version-check --checks=all live --uri wss://${{ matrix.runtime }}.kilt.io
Loading
Loading