Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very many epochs/* tags accumulating, can they be garbage collected? #28764

Open
foolip opened this issue Apr 30, 2021 · 3 comments
Open

Very many epochs/* tags accumulating, can they be garbage collected? #28764

foolip opened this issue Apr 30, 2021 · 3 comments

Comments

@foolip
Copy link
Member

foolip commented Apr 30, 2021

#26122 introduced tags for pushes to the epochs/* branches, in order to have a record of what those branches had pointed to in the past. That's resulted in over 2000 such tags since then, which are pretty prominent in git log:

commit c2df3bee8330f8058bd15a59e51567d423ed3a81 (tag: merge_pr_28741, tag: epochs/twelve_hourly/2021-04-30_12H, tag: epochs/three_hourly/2021-04-30_12H, tag: epochs/six_hourly/2021-04-30_12H, origin/master, origin/epochs/twelve_hourly, origin/epochs/three_hourly, origin/epochs/six_hourly, origin/HEAD)
Author: Antonio Sartori <[email protected]>
Date:   Fri Apr 30 13:12:21 2021 +0200

    Fix the Stash Lock to be unique (#28741)
    
    Co-authored-by: Antonio Sartori <[email protected]>

It's not a big problem, but I wonder if these tags have helped debug issues, @clopez?

In the issue the challenge was identifying the commit that triggered the taskcluster run on epochs/daily for October 12, 2020.

The commit should be one listed by ./wpt rev-list --epoch 1d --max-count=365. Looking at./wpt rev-list --epoch 1d --max-count=365 | xargs git log --no-walk --pretty=fuller I think it was 1b2893a, and clicking through to https://community-tc.services.mozilla.com/tasks/MRVToMMNQjOG7npZAfF0jQ confirms it.

Being able to list the right commits after the fact was an intentional part of how ./wpt rev-list was implemented, but it's not documented how to figure this out anywhere.

@clopez did you recall if you looked for documentation about this anywhere? If you did, then maybe we can put documentation there, and stop creating the tags?

@foolip foolip added the infra label Apr 30, 2021
@clopez
Copy link
Contributor

clopez commented May 7, 2021

I knew about the command ./wpt rev-list, but for some reason I had difficulties finding the exact commit that caused #26097
Not sure if it could be related to different time-zones. Or maybe it was related because i didn't knew that it accepted a --max-count=365 parameter so I was trying to pass something like ./wpt rev-list --epoch 4d

It's not a big problem, but I wonder if these tags have helped debug issues, @clopez?

I didn't had a need to use them, since there wasn't any issue reported with the CI and WebKitGTK builds since then.

If no one else is finding this useful, then I agree it is a good idea to revert #26122 and document ./wpt rev-list

@clopez
Copy link
Contributor

clopez commented May 7, 2021

BTW; I'm quickly checking if all commits annotated via tags are reported by ./wpt rev-list and it seems a few are not.

# get list of commit hashes for each epoch tag
git tag|grep epochs/twelve_hourly|while read x; do git log -1 --pretty=%H $x; done >/tmp/twelve_hourly
git tag|grep epochs/six_hourly|while read x; do git log -1 --pretty=%H $x; done >/tmp/six_hourly
git tag|grep epochs/three_hourly|while read x; do git log -1 --pretty=%H $x; done >/tmp/three_hourly
git tag|grep epochs/daily|while read x; do git log -1 --pretty=%H $x; done >/tmp/daily
git tag|grep epochs/weekly|while read x; do git log -1 --pretty=%H $x; done >/tmp/weekly
# get list of commit hashes for each epoch tag from wpt rev-list
./wpt rev-list --epoch 1d --max-count=9999 >/tmp/daily.wpt
./wpt rev-list --epoch 1w --max-count=9999 >/tmp/weekly.wpt
./wpt rev-list --epoch 12h --max-count=9999 >/tmp/twelve_hourly.wpt
./wpt rev-list --epoch 6h --max-count=9999 >/tmp/six_hourly.wpt
./wpt rev-list --epoch 3h --max-count=9999 >/tmp/three_hourly.wpt
# compare to see if any tag is not reported by wpt rev-list
for epoch in daily weekly twelve_hourly six_hourly three_hourly; do cat /tmp/${epoch}| while read commit; do grep -q -o ${commit} /tmp/${epoch}.wpt || echo "${epoch} ${commit}"; done; done

Result:

daily b6fc05c48b13142e45ad0d409ed14bfc23262060
daily 2797f44946e26757effe22bac5573071a919b1cf
twelve_hourly b6fc05c48b13142e45ad0d409ed14bfc23262060
twelve_hourly 2797f44946e26757effe22bac5573071a919b1cf
six_hourly 9cdaa1607511824df0c9ae77f6b60d0b23f24d04
six_hourly 2797f44946e26757effe22bac5573071a919b1cf
three_hourly 030a772430d57102296d3d3f811cd4ff9f9bcd79
three_hourly b74014abe0da655dd4675b1ddce4364eabc0b3de

Those have an epoch tag but doesn't seem to appear in the wpt rev-list for such epoch

@foolip
Copy link
Member Author

foolip commented Jul 6, 2022

I've sent web-platform-tests/rfcs#117 to drop these tags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants