Skip to content

Commit 3fafa0c

Browse files
levalleux-ludoclaude
levalleux-ludo
and
claude
committed
ci: promote subgraphs from a schedule instead of waiting for the sync
Waiting for a freshly deployed version to be indexed inside the deploy job does not hold: boson-testing-base took roughly 6 hours, the job hit its timeout, and the 'latest' tag stayed on the previous version even though the new one ended up indexed and healthy. The wait is now split off from the deployment: - the deploy job watches the new version for 5 minutes, long enough to catch a deployment that cannot index at all, then ends successfully and flags the deployed version in a SUBGRAPH_PENDING_<ENV>_<NETWORK> repository variable - a new 'Promote deployed subgraphs' schedule checks every flagged deployment every 30 minutes and, once it is synced and healthy, moves the 'latest' tag onto it and deletes the version the tag was pointing to Nothing holds a runner for hours any more, and no promotion can hit the 6 hour job limit. A network is flagged, checked and cleared on its own, so a subgraph that fails to index never affects the others. 'promote-on-goldsky.ts' makes a single check and reports 'promoted', 'pending' or 'failed' on its 'outcome' step output, which is what clears the flag. A transient failure - Goldsky unreachable, say - writes no outcome, so the deployment stays flagged and is retried on the next tick. The deploy and promote scripts share 'goldsky.ts', and the workflows share a 'setup-subgraph' composite action. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 3227730 commit 3fafa0c

9 files changed

Lines changed: 711 additions & 418 deletions

File tree

‎.github/actions/setup-subgraph/action.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: "Set up the subgraph workspace"
2+
description: "Install the monorepo dependencies and, when the built subgraph is needed, build them. The repository must already be checked out."
3+
4+
inputs:
5+
build:
6+
description: "Build the monorepo. Only the deployment needs it, as 'goldsky subgraph deploy' uploads the ./build directory - tagging an already deployed version does not."
7+
required: false
8+
default: "true"
9+
10+
runs:
11+
using: "composite"
12+
steps:
13+
- uses: actions/setup-node@v3
14+
with:
15+
# Must stay on the Node (and so npm) version pinned by the root
16+
# package.json volta config: npm 11 and npm 10 resolve the root
17+
# typescript override differently, and no single lock file
18+
# satisfies both, so `npm ci` fails on whichever is out of step.
19+
node-version: "22"
20+
cache: "npm"
21+
- name: Turbo Cache
22+
if: inputs.build == 'true'
23+
uses: actions/cache@v3
24+
with:
25+
path: .turbo
26+
key: turbo-${{ runner.os }}-${{ github.sha }}
27+
restore-keys: |
28+
turbo-${{ runner.os }}
29+
- name: Install dependencies
30+
shell: bash
31+
run: npm ci
32+
- name: Build
33+
if: inputs.build == 'true'
34+
shell: bash
35+
run: npm run build -- --cache-dir=".turbo"

‎.github/workflows/deploy-prod-subgraph.yaml‎

Lines changed: 50 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -7,58 +7,65 @@ on:
77
description: "The subgraph version to deploy. For example, 1.34.0. The given number should correspond to an existing tag @bosonprotocol/subgraph@<VERSION>"
88
required: true
99

10-
# Deployments wait for the new version to be synced, so a run can last hours: queue
11-
# overlapping runs instead of cancelling one in the middle of a tag move
12-
concurrency:
13-
group: deploy-prod-subgraph
14-
cancel-in-progress: false
10+
permissions:
11+
contents: read
1512

13+
# Every network is deployed by its own job, so that a network which fails to deploy does
14+
# not hold back the others. Indexing the new version takes far longer than a job may run,
15+
# so no job waits for it: each deployment is flagged in a repository variable, and the
16+
# 'Promote deployed subgraphs' schedule moves the 'latest' tag once the version is indexed.
1617
jobs:
17-
deploy:
18-
name: Deploy ${{ matrix.label }} subgraph
18+
deploy-ethereum:
19+
name: Deploy Ethereum subgraph
1920
runs-on: ubuntu-latest
20-
# Each network gets its own runner, so the whole job budget is available for a
21-
# single GOLDSKY_SYNC_TIMEOUT_MINUTES wait plus the install and build steps
22-
timeout-minutes: 350
23-
strategy:
24-
# A network that fails to sync must not cancel the others
25-
fail-fast: false
26-
matrix:
27-
include:
28-
- network: ethereum
29-
label: Ethereum
30-
- network: base
31-
label: Base
21+
timeout-minutes: 45
22+
# Do not let two runs deploy the same subgraph at the same time, and never cancel one
23+
# mid-deployment
24+
concurrency:
25+
group: subgraph-deploy-production-ethereum
26+
cancel-in-progress: false
3227
steps:
3328
- uses: actions/checkout@v3
3429
with:
3530
token: ${{ secrets.BSNORG_ACTIONS_SECRET }}
3631
ref: "@bosonprotocol/subgraph@${{ inputs.version }}"
3732
fetch-depth: "0"
38-
- uses: actions/setup-node@v3
39-
with:
40-
# Must stay on the Node (and so npm) version pinned by the root
41-
# package.json volta config: npm 11 and npm 10 resolve the root
42-
# typescript override differently, and no single lock file
43-
# satisfies both, so `npm ci` fails on whichever is out of step.
44-
node-version: "22"
45-
cache: "npm"
46-
- name: Turbo Cache
47-
id: turbo-cache
48-
uses: actions/cache@v3
33+
- uses: ./.github/actions/setup-subgraph
34+
# Deploys the new version and watches it for a few minutes, long enough to catch a
35+
# deployment that cannot index at all. The 'latest' tag is left on the version
36+
# currently served: moving it is the promotion schedule's business.
37+
- id: deploy
38+
name: Deploy subgraph
39+
run: cd ./packages/subgraph && npm run deploy:production:ethereum
40+
env:
41+
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_PRODUCTION }}
42+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
43+
- name: Flag the deployment for promotion
44+
env:
45+
GH_TOKEN: ${{ secrets.BSNORG_ACTIONS_SECRET }}
46+
run: gh variable set SUBGRAPH_PENDING_PRODUCTION_ETHEREUM --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"
47+
48+
deploy-base:
49+
name: Deploy Base subgraph
50+
runs-on: ubuntu-latest
51+
timeout-minutes: 45
52+
concurrency:
53+
group: subgraph-deploy-production-base
54+
cancel-in-progress: false
55+
steps:
56+
- uses: actions/checkout@v3
4957
with:
50-
path: .turbo
51-
key: turbo-${{ runner.os }}-${{ github.sha }}
52-
restore-keys: |
53-
turbo-${{ runner.os }}
54-
- run: npm ci
55-
- id: build
56-
run: npm run build -- --cache-dir=".turbo"
57-
# GOLDSKY_WAIT_FOR_SYNC makes the 'latest' tag move - and the previously tagged
58-
# version get deleted - only once the new version is synced and healthy
59-
- name: Deploy subgraph
60-
run: cd ./packages/subgraph && npm run deploy:production:${{ matrix.network }}
58+
token: ${{ secrets.BSNORG_ACTIONS_SECRET }}
59+
ref: "@bosonprotocol/subgraph@${{ inputs.version }}"
60+
fetch-depth: "0"
61+
- uses: ./.github/actions/setup-subgraph
62+
- id: deploy
63+
name: Deploy subgraph
64+
run: cd ./packages/subgraph && npm run deploy:production:base
6165
env:
6266
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_PRODUCTION }}
63-
GOLDSKY_WAIT_FOR_SYNC: "true"
64-
GOLDSKY_SYNC_TIMEOUT_MINUTES: "300"
67+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
68+
- name: Flag the deployment for promotion
69+
env:
70+
GH_TOKEN: ${{ secrets.BSNORG_ACTIONS_SECRET }}
71+
run: gh variable set SUBGRAPH_PENDING_PRODUCTION_BASE --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"

‎.github/workflows/deploy-staging-subgraph.yaml‎

Lines changed: 49 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -5,57 +5,63 @@ on:
55
release:
66
types: [published]
77

8-
# Deployments wait for the new version to be synced, so a run can last hours: queue
9-
# overlapping runs instead of cancelling one in the middle of a tag move
10-
concurrency:
11-
group: deploy-staging-subgraph
12-
cancel-in-progress: false
8+
permissions:
9+
contents: read
1310

11+
# Every network is deployed by its own job, so that a network which fails to deploy does
12+
# not hold back the others. Indexing the new version takes far longer than a job may run,
13+
# so no job waits for it: each deployment is flagged in a repository variable, and the
14+
# 'Promote deployed subgraphs' schedule moves the 'latest' tag once the version is indexed.
1415
jobs:
15-
deploy:
16+
deploy-sepolia:
1617
if: github.event_name == 'workflow_dispatch' || startsWith(github.event.release.tag_name, '@bosonprotocol/subgraph@')
17-
name: Deploy ${{ matrix.label }} subgraph
18+
name: Deploy Sepolia subgraph
1819
runs-on: ubuntu-latest
19-
# Each network gets its own runner, so the whole job budget is available for a
20-
# single GOLDSKY_SYNC_TIMEOUT_MINUTES wait plus the install and build steps
21-
timeout-minutes: 350
22-
strategy:
23-
# A network that fails to sync must not cancel the others
24-
fail-fast: false
25-
matrix:
26-
include:
27-
- network: sepolia
28-
label: Sepolia
29-
- network: base
30-
label: Base Sepolia
20+
timeout-minutes: 45
21+
# Do not let two runs deploy the same subgraph at the same time, and never cancel one
22+
# mid-deployment
23+
concurrency:
24+
group: subgraph-deploy-staging-sepolia
25+
cancel-in-progress: false
3126
steps:
3227
- uses: actions/checkout@v3
3328
with:
3429
token: ${{ secrets.BSNORG_ACTIONS_SECRET }}
35-
- uses: actions/setup-node@v3
36-
with:
37-
# Must stay on the Node (and so npm) version pinned by the root
38-
# package.json volta config: npm 11 and npm 10 resolve the root
39-
# typescript override differently, and no single lock file
40-
# satisfies both, so `npm ci` fails on whichever is out of step.
41-
node-version: "22"
42-
cache: "npm"
43-
- name: Turbo Cache
44-
id: turbo-cache
45-
uses: actions/cache@v3
30+
- uses: ./.github/actions/setup-subgraph
31+
# Deploys the new version and watches it for a few minutes, long enough to catch a
32+
# deployment that cannot index at all. The 'latest' tag is left on the version
33+
# currently served: moving it is the promotion schedule's business.
34+
- id: deploy
35+
name: Deploy subgraph
36+
run: cd ./packages/subgraph && npm run deploy:staging:sepolia
37+
env:
38+
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_STAGING }}
39+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
40+
- name: Flag the deployment for promotion
41+
env:
42+
GH_TOKEN: ${{ secrets.BSNORG_ACTIONS_SECRET }}
43+
run: gh variable set SUBGRAPH_PENDING_STAGING_SEPOLIA --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"
44+
45+
deploy-base:
46+
if: github.event_name == 'workflow_dispatch' || startsWith(github.event.release.tag_name, '@bosonprotocol/subgraph@')
47+
name: Deploy Base Sepolia subgraph
48+
runs-on: ubuntu-latest
49+
timeout-minutes: 45
50+
concurrency:
51+
group: subgraph-deploy-staging-base
52+
cancel-in-progress: false
53+
steps:
54+
- uses: actions/checkout@v3
4655
with:
47-
path: .turbo
48-
key: turbo-${{ runner.os }}-${{ github.sha }}
49-
restore-keys: |
50-
turbo-${{ runner.os }}
51-
- run: npm ci
52-
- id: build
53-
run: npm run build -- --cache-dir=".turbo"
54-
# GOLDSKY_WAIT_FOR_SYNC makes the 'latest' tag move - and the previously tagged
55-
# version get deleted - only once the new version is synced and healthy
56-
- name: Deploy subgraph
57-
run: cd ./packages/subgraph && npm run deploy:staging:${{ matrix.network }}
56+
token: ${{ secrets.BSNORG_ACTIONS_SECRET }}
57+
- uses: ./.github/actions/setup-subgraph
58+
- id: deploy
59+
name: Deploy subgraph
60+
run: cd ./packages/subgraph && npm run deploy:staging:base
5861
env:
5962
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_STAGING }}
60-
GOLDSKY_WAIT_FOR_SYNC: "true"
61-
GOLDSKY_SYNC_TIMEOUT_MINUTES: "300"
63+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
64+
- name: Flag the deployment for promotion
65+
env:
66+
GH_TOKEN: ${{ secrets.BSNORG_ACTIONS_SECRET }}
67+
run: gh variable set SUBGRAPH_PENDING_STAGING_BASE --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"

‎.github/workflows/deploy-testing-subgraph.yaml‎

Lines changed: 48 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -13,56 +13,61 @@ on:
1313
GH_TOKEN:
1414
required: true
1515

16-
# Deployments wait for the new version to be synced, so a run can last hours: queue
17-
# overlapping runs instead of cancelling one in the middle of a tag move
18-
concurrency:
19-
group: deploy-testing-subgraph
20-
cancel-in-progress: false
16+
permissions:
17+
contents: read
2118

19+
# Every network is deployed by its own job, so that a network which fails to deploy does
20+
# not hold back the others. Indexing the new version takes far longer than a job may run,
21+
# so no job waits for it: each deployment is flagged in a repository variable, and the
22+
# 'Promote deployed subgraphs' schedule moves the 'latest' tag once the version is indexed.
2223
jobs:
23-
deploy:
24-
name: Deploy ${{ matrix.label }} subgraph
24+
deploy-sepolia:
25+
name: Deploy Sepolia subgraph
2526
runs-on: ubuntu-latest
26-
# Each network gets its own runner, so the whole job budget is available for a
27-
# single GOLDSKY_SYNC_TIMEOUT_MINUTES wait plus the install and build steps
28-
timeout-minutes: 350
29-
strategy:
30-
# A network that fails to sync must not cancel the others
31-
fail-fast: false
32-
matrix:
33-
include:
34-
- network: sepolia
35-
label: Sepolia
36-
- network: base
37-
label: Base Sepolia
27+
timeout-minutes: 45
28+
# Do not let two runs deploy the same subgraph at the same time, and never cancel one
29+
# mid-deployment
30+
concurrency:
31+
group: subgraph-deploy-testing-sepolia
32+
cancel-in-progress: false
3833
steps:
3934
- uses: actions/checkout@v3
4035
with:
4136
token: ${{ secrets.GH_TOKEN }}
42-
- uses: actions/setup-node@v3
43-
with:
44-
# Must stay on the Node (and so npm) version pinned by the root
45-
# package.json volta config: npm 11 and npm 10 resolve the root
46-
# typescript override differently, and no single lock file
47-
# satisfies both, so `npm ci` fails on whichever is out of step.
48-
node-version: "22"
49-
cache: "npm"
50-
- name: Turbo Cache
51-
id: turbo-cache
52-
uses: actions/cache@v3
37+
- uses: ./.github/actions/setup-subgraph
38+
# Deploys the new version and watches it for a few minutes, long enough to catch a
39+
# deployment that cannot index at all. The 'latest' tag is left on the version
40+
# currently served: moving it is the promotion schedule's business.
41+
- id: deploy
42+
name: Deploy subgraph
43+
run: cd ./packages/subgraph && npm run deploy:testing:sepolia
44+
env:
45+
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_TESTING }}
46+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
47+
- name: Flag the deployment for promotion
48+
env:
49+
GH_TOKEN: ${{ secrets.GH_TOKEN }}
50+
run: gh variable set SUBGRAPH_PENDING_TESTING_SEPOLIA --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"
51+
52+
deploy-base:
53+
name: Deploy Base Sepolia subgraph
54+
runs-on: ubuntu-latest
55+
timeout-minutes: 45
56+
concurrency:
57+
group: subgraph-deploy-testing-base
58+
cancel-in-progress: false
59+
steps:
60+
- uses: actions/checkout@v3
5361
with:
54-
path: .turbo
55-
key: turbo-${{ runner.os }}-${{ github.sha }}
56-
restore-keys: |
57-
turbo-${{ runner.os }}
58-
- run: npm ci
59-
- id: build
60-
run: npm run build -- --cache-dir=".turbo"
61-
# GOLDSKY_WAIT_FOR_SYNC makes the 'latest' tag move - and the previously tagged
62-
# version get deleted - only once the new version is synced and healthy
63-
- name: Deploy subgraph
64-
run: cd ./packages/subgraph && npm run deploy:testing:${{ matrix.network }}
62+
token: ${{ secrets.GH_TOKEN }}
63+
- uses: ./.github/actions/setup-subgraph
64+
- id: deploy
65+
name: Deploy subgraph
66+
run: cd ./packages/subgraph && npm run deploy:testing:base
6567
env:
6668
GOLDSKY_API_KEY: ${{ secrets.GOLDSKY_API_KEY_TESTING }}
67-
GOLDSKY_WAIT_FOR_SYNC: "true"
68-
GOLDSKY_SYNC_TIMEOUT_MINUTES: "300"
69+
GOLDSKY_FATAL_ERROR_WATCH_MINUTES: "5"
70+
- name: Flag the deployment for promotion
71+
env:
72+
GH_TOKEN: ${{ secrets.GH_TOKEN }}
73+
run: gh variable set SUBGRAPH_PENDING_TESTING_BASE --repo "$GITHUB_REPOSITORY" --body "${{ steps.deploy.outputs.version }}"

0 commit comments

Comments
 (0)