Skip to content

Commit 3862405

Browse files
mdelapenyaclaude
mdelapenya
and
claude
authored
feat(metrics): track daily GitHub clones on the usage-metrics site (#3920)
* feat(metrics): track daily GitHub clones on the usage-metrics site GitHub keeps only 14 days of clone traffic. Add a `clones` subcommand to the usage-metrics collector that reads the Traffic API through `gh` and upserts one row per day into docs/usage-metrics/clones.csv, a daily workflow that pushes the CSV to a monthly `YYYY-MM-clones` branch and opens a pull request to main once the month is over, and a new "GitHub Clones" dashboard page with daily, 7-day rolling average and weekly charts. The CSV is seeded with the 14 days available today. The workflow reads the Traffic API with the TRAFFIC_TOKEN secret, a fine-grained PAT with Administration: read on this repository, because the traffic endpoints need push access that GITHUB_TOKEN cannot get. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix(metrics): rebuild the clones data branch from main and add view toggles Address the review of the GitHub clones feature. The daily workflow now rebuilds the `YYYY-MM-clones` branch on top of main on every run instead of merging main into a long-lived branch, so the branch never conflicts with main and never reverts fixes made there. The previous month is finalised with `-until <last day>` before its pull request opens, and the in-progress UTC day is never stored. The workflow gains a concurrency group and a lowercase PR title that satisfies the conventions check, and both the workflow and clones.csv are excluded from module detection. The collector gains `-until`, validates the CSV header, writes through a temporary file, treats any 403 that is not a rate limit as an auth error, and pins its endpoint path in a test. The dashboard replaces the fixed daily and weekly charts with one chart that switches between day, week, month and year, and computes the 7-day average and the 14-day card over calendar days rather than rows. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 90bbb19 commit 3862405

10 files changed

Lines changed: 1258 additions & 26 deletions

File tree

‎.github/workflows/github-clones.yml‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
name: Update GitHub Clones Metrics
2+
3+
on:
4+
schedule:
5+
# Run daily at 3 AM UTC. GitHub keeps only 14 days of clone traffic,
6+
# so a daily run leaves a wide margin before data is lost.
7+
- cron: '0 3 * * *'
8+
workflow_dispatch:
9+
10+
permissions:
11+
contents: write
12+
pull-requests: write
13+
14+
# Every run pushes to shared data branches; never let two runs overlap.
15+
concurrency:
16+
group: github-clones
17+
cancel-in-progress: false
18+
19+
env:
20+
CSV: docs/usage-metrics/clones.csv
21+
22+
jobs:
23+
collect-clones:
24+
runs-on: ubuntu-latest
25+
timeout-minutes: 30
26+
steps:
27+
- name: Checkout main
28+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
29+
with:
30+
# Data branches are always rebuilt on top of main, whatever ref triggered the run.
31+
ref: main
32+
33+
- name: Set up Go
34+
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
35+
with:
36+
go-version-file: 'usage-metrics/go.mod'
37+
38+
- name: Resolve dates and branches
39+
id: dates
40+
run: |
41+
TODAY=$(date -u +%Y-%m-%d)
42+
YESTERDAY=$(date -u -d "$TODAY -1 day" +%Y-%m-%d)
43+
MONTH=$(date -u +%Y-%m)
44+
PREV_LAST_DAY=$(date -u -d "$MONTH-01 -1 day" +%Y-%m-%d)
45+
PREV_MONTH=${PREV_LAST_DAY%-*}
46+
{
47+
echo "today=$TODAY"
48+
echo "yesterday=$YESTERDAY"
49+
echo "branch=$MONTH-clones"
50+
echo "prev_branch=$PREV_MONTH-clones"
51+
echo "prev_month=$PREV_MONTH"
52+
echo "prev_last_day=$PREV_LAST_DAY"
53+
} >> "$GITHUB_OUTPUT"
54+
echo "Data branch: $MONTH-clones (previous: $PREV_MONTH-clones, last day $PREV_LAST_DAY)"
55+
56+
- name: Configure git and fetch the data branches
57+
env:
58+
BRANCH: ${{ steps.dates.outputs.branch }}
59+
PREV_BRANCH: ${{ steps.dates.outputs.prev_branch }}
60+
run: |
61+
git config user.name "github-actions[bot]"
62+
git config user.email "github-actions[bot]@users.noreply.github.com"
63+
for b in "$BRANCH" "$PREV_BRANCH"; do
64+
if git ls-remote --exit-code --heads origin "$b" > /dev/null; then
65+
git fetch --depth=1 origin "+refs/heads/$b:refs/remotes/origin/$b"
66+
fi
67+
done
68+
69+
- name: Finalise the previous month and open its pull request
70+
env:
71+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
72+
# Fine-grained PAT with "Administration: read" on this repository.
73+
# The traffic API needs push access, which GITHUB_TOKEN cannot be granted.
74+
TRAFFIC_TOKEN: ${{ secrets.TRAFFIC_TOKEN }}
75+
PREV_BRANCH: ${{ steps.dates.outputs.prev_branch }}
76+
PREV_MONTH: ${{ steps.dates.outputs.prev_month }}
77+
PREV_LAST_DAY: ${{ steps.dates.outputs.prev_last_day }}
78+
TODAY: ${{ steps.dates.outputs.today }}
79+
run: |
80+
if ! git rev-parse --verify --quiet "origin/$PREV_BRANCH" > /dev/null; then
81+
echo "No branch $PREV_BRANCH on origin, nothing to finalise"
82+
exit 0
83+
fi
84+
85+
EXISTING_PR=$(gh pr list --head "$PREV_BRANCH" --state all --json number --jq '.[0].number')
86+
if [ -n "$EXISTING_PR" ]; then
87+
echo "PR #$EXISTING_PR already exists for $PREV_BRANCH"
88+
exit 0
89+
fi
90+
91+
# Rebuild the branch on main, restore its CSV, and upsert the final numbers for the
92+
# month's last days: the API window still covers them during the first days of the new month.
93+
git checkout -B "$PREV_BRANCH" origin/main
94+
git checkout "origin/$PREV_BRANCH" -- "$CSV"
95+
(cd usage-metrics && GH_TOKEN="$TRAFFIC_TOKEN" go run collect.go clones -csv "../$CSV" -until "$PREV_LAST_DAY")
96+
git add "$CSV"
97+
if git diff --staged --quiet; then
98+
echo "$PREV_BRANCH has no data beyond main, nothing to publish"
99+
exit 0
100+
fi
101+
git commit -m "chore(metrics): update github clones ($TODAY)"
102+
git push --force-with-lease -u origin "$PREV_BRANCH"
103+
104+
gh pr create \
105+
--head "$PREV_BRANCH" \
106+
--base main \
107+
--title "chore(metrics): update github clones for $PREV_MONTH" \
108+
--body "Automated update of GitHub clone traffic for $PREV_MONTH, collected daily by the Update GitHub Clones Metrics workflow. Merging it publishes the data at https://golang.testcontainers.org/usage-metrics/clones/."
109+
110+
- name: Update the current month
111+
env:
112+
TRAFFIC_TOKEN: ${{ secrets.TRAFFIC_TOKEN }}
113+
BRANCH: ${{ steps.dates.outputs.branch }}
114+
PREV_BRANCH: ${{ steps.dates.outputs.prev_branch }}
115+
YESTERDAY: ${{ steps.dates.outputs.yesterday }}
116+
TODAY: ${{ steps.dates.outputs.today }}
117+
run: |
118+
# The branch is rebuilt from main on every run: one commit, only the CSV differs.
119+
git checkout -B "$BRANCH" origin/main
120+
if git rev-parse --verify --quiet "origin/$BRANCH" > /dev/null; then
121+
git checkout "origin/$BRANCH" -- "$CSV"
122+
elif git rev-parse --verify --quiet "origin/$PREV_BRANCH" > /dev/null; then
123+
# New month: start from last month's rows so this branch stays a superset of main.
124+
git checkout "origin/$PREV_BRANCH" -- "$CSV"
125+
fi
126+
127+
# Never store the in-progress UTC day; it is completed on tomorrow's run.
128+
(cd usage-metrics && GH_TOKEN="$TRAFFIC_TOKEN" go run collect.go clones -csv "../$CSV" -until "$YESTERDAY")
129+
git add "$CSV"
130+
if git diff --staged --quiet; then
131+
echo "No new clone data for $BRANCH"
132+
exit 0
133+
fi
134+
git commit -m "chore(metrics): update github clones ($TODAY)"
135+
git push --force-with-lease -u origin "$BRANCH"

‎docs/css/usage-metrics.css‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,49 @@ body:has(.stats-grid) .md-content__inner > article {
133133
margin-bottom: 10px;
134134
}
135135

136+
.chart-header {
137+
display: flex;
138+
align-items: center;
139+
justify-content: space-between;
140+
flex-wrap: wrap;
141+
gap: 12px;
142+
margin-bottom: 20px;
143+
}
144+
145+
.chart-header .chart-title {
146+
margin-bottom: 0;
147+
}
148+
149+
.granularity-toggle {
150+
display: inline-flex;
151+
border: 2px solid #e0e0e0;
152+
border-radius: 8px;
153+
overflow: hidden;
154+
}
155+
156+
.granularity-button {
157+
background: white;
158+
border: none;
159+
padding: 6px 14px;
160+
font-size: 0.9rem;
161+
color: #666;
162+
cursor: pointer;
163+
transition: background-color 0.2s, color 0.2s;
164+
}
165+
166+
.granularity-button + .granularity-button {
167+
border-left: 2px solid #e0e0e0;
168+
}
169+
170+
.granularity-button:hover {
171+
background: #f5f5f5;
172+
}
173+
174+
.granularity-button.active {
175+
background: #667eea;
176+
color: white;
177+
}
178+
136179
.metrics-info {
137180
text-align: center;
138181
margin-top: 40px;

0 commit comments

Comments
 (0)