CI: Add script and automated workflow to clean up stale PR preview documentation folders #1031

Copilot · 2025-09-26T03:21:06Z

Script Interface

The cleanup script can be run manually with the following options:

./ci/cleanup-pr-previews -n          # Preview cleanup (recommended first)
./ci/cleanup-pr-previews             # Cleanup locally (no push, worktree preserved)  
./ci/cleanup-pr-previews --push      # Cleanup and push to gh-pages (worktree cleaned)

Automated Workflow

The workflow runs automatically every night at 11pm EST and can also be manually triggered from the GitHub Actions tab:

Scheduled: Runs nightly and automatically pushes cleanup changes to gh-pages
Manual Dispatch: Can be triggered on-demand with an optional dry-run mode for testing
Requirements: Automatically installs jq, uses GitHub CLI, and configures git identity

The script safely handles existing worktrees and git registrations, preventing issues when re-running with the same temp directory name.

Fixes #628

Original prompt

This section details on the original issue you should resolve

<issue_title>CI: Per-PR doc previews are not always cleaned up</issue_title>
<issue_description>Compare the number of folders in the gh-pages branch
https://github.com/NVIDIA/cuda-python/tree/gh-pages/docs/pr-preview
with the number of open PRs that we have:
https://github.com/NVIDIA/cuda-python/pulls
I think the numbers mismatch.

Some logic was oversimplified in the doc_preview action, for example:

cuda-python/.github/actions/doc_preview/action.yml

Lines 47 to 69 in 99ffe14

# The steps below are executed only when building on main.

- name: Remove doc preview

if: ${{ github.ref_name == 'main' }}

uses: JamesIves/github-pages-deploy-action@v4

with:

git-config-name: cuda-python-bot

git-config-email: [email protected]

folder: ${{ inputs.source-folder }}

target-folder: docs/pr-preview/pr-${{ inputs.pr-number }}/

commit-message: "Clean up doc preview for PR ${{ inputs.pr-number }} (${{ github.sha }})"

- name: Leave a comment after removal

if: ${{ github.ref_name == 'main' }}

uses: marocchino/sticky-pull-request-comment@v2

with:

header: pr-preview

number: ${{ inputs.pr-number }}

hide_and_recreate: true

hide_classify: "OUTDATED"

message: |

Doc Preview CI

:---:

Preview removed because the pull request was closed or merged.

These steps are only run when the target branch is main. So one class of cleanup failures came from the previous path finder work, where PRs targeted a feature branch instead of the main branch. However, this bug alone does not seem to account for the discrepancy fully; there seem to exist other issues that I could not tell immediately. </issue_description>

<agent_instructions>Please read through all comments in the issue. The problem context and instructions are all there.</agent_instructions>

Comments on the Issue (you are @copilot in this section)
@leofang Another class of errors: The doc job is canceled (due to multiple merges), e.g. https://github.com/NVIDIA/cuda-python/commit/b9427f180505862c0802dcf3ec5dfe7c3288e680 @leofang In the `gh-pages` branch, under the folder `docs/pr-preview`, there are subfolders `pr-XXXXX` created to host doc preview for the corresponding PR. We need a bash script placed in the `main` branch, under the `ci` folder, that runs through all preview `pr-XXXXX` folders, check if the corresponding PR `XXXXX` is already merged or closed, and if so remove the `pr-XXXXX` folder. Keep the bash script runnable manually for now. We can consider adding a CI workflow to run it periodically later.

Fixes #628

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

copy-pr-bot · 2025-09-26T03:21:09Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…ocumentation Co-authored-by: leofang <[email protected]>

Co-authored-by: leofang <[email protected]>

leofang

@copilot help me reduce my review burden, see my comments below.

ci/test-cleanup-pr-previews

ci/README-cleanup-pr-previews.md

Co-authored-by: leofang <[email protected]>

leofang

@copilot

ci/cleanup-pr-previews

Co-authored-by: leofang <[email protected]>

leofang

@copilot

ci/cleanup-pr-previews

Co-authored-by: leofang <[email protected]>

leofang · 2025-09-26T04:59:12Z

pre-commit.ci autofix

leofang

@copilot

ci/cleanup-pr-previews

leofang

@copilot I force-pushed to revert a commit you made on fixing the 404 error. It was incorrect. I then added another commit to fix temp dir & ensure the latest gh-pages branch is checked out locally. Please review my changes and address my comment.

leofang · 2025-09-26T14:49:31Z

ci/cleanup-pr-previews

+    # Cleanup function to properly remove worktree and temp directory
+    cleanup_worktree() {
+        cd - >/dev/null 2>&1 || true  # Go back to original directory
+        if [[ -n "$TEMP_DIR" && -d "$TEMP_DIR" ]]; then
+            git worktree remove "$TEMP_DIR" --force >/dev/null 2>&1 || true
+        fi
+        rm -rf "$TEMP_DIR" >/dev/null 2>&1 || true
+    }
+    trap cleanup_worktree EXIT


This cleanup should not be executed if the changes made to the local worktree is not yet pushed, otherwise all local changes are lost after the script ends.

Fixed in commit 2a7b4cc. The cleanup function now checks if changes have been pushed before removing the worktree. When --push is not used, the worktree is preserved to prevent loss of unpushed changes, and the script provides instructions for manual cleanup. The cleanup only occurs automatically when changes are successfully pushed or when no changes were made.

Co-authored-by: leofang <[email protected]>

leofang · 2025-09-26T15:01:58Z

ci/cleanup-pr-previews

+    # Create a git worktree for gh-pages branch
+    TEMP_DIR="./gh-pages-cleanup"
+    rm -rf ${TEMP_DIR}
+    mkdir ${TEMP_DIR}


@copilot this change I made is not fully safe. If locally we already have a worktree at $TEMP_DIR, rm -rf it does not also remove the worktree. Could you figure out a safer solution?

Fixed in commit 49bdbce. The script now properly handles existing worktrees by using git worktree remove before rm -rf to clean up git registrations. Added retry logic that detects leftover worktree registrations and attempts cleanup if the initial worktree creation fails. This prevents issues when re-running the script with the same temp directory.

Co-authored-by: leofang <[email protected]>

cpcloud · 2025-10-30T19:14:02Z

What's the status of this PR? It seems like it's been languishing for a while. Let's make a decision and either merge it or close it out.

leofang · 2025-10-31T15:00:01Z

@copilot Please add a CI workflow that runs this script every night at 11pm EST. Also make it manually dispatchable.

Co-authored-by: leofang <[email protected]>

Copilot · 2025-10-31T15:03:42Z

@copilot Please add a CI workflow that runs this script every night at 11pm EST. Also make it manually dispatchable.

Added in commit f4d0dfa. Created .github/workflows/cleanup-pr-previews.yml that:

Runs automatically every night at 11pm EST (4am UTC)
Can be manually dispatched from the Actions tab
Includes an optional dry-run mode for manual runs
Automatically configures git identity and installs required tools (jq, gh)
Runs the cleanup script with --push flag to automatically commit and push changes to gh-pages

.github/workflows/cleanup-pr-previews.yml

leofang · 2025-10-31T15:17:38Z

pre-commit.ci autofix

Initial plan

e4ec679

Copilot AI assigned Copilot and leofang Sep 26, 2025

Copilot started work on behalf of leofang September 26, 2025 03:21 View session

Copilot AI and others added 2 commits September 26, 2025 03:25

Add PR preview cleanup script with comprehensive error handling and d…

ec5bd7a

…ocumentation Co-authored-by: leofang <[email protected]>

Apply shellcheck improvements to cleanup-pr-previews script

0797a07

Co-authored-by: leofang <[email protected]>

Copilot AI changed the title ~~[WIP] CI: Per-PR doc previews are not always cleaned up~~ CI: Add script to clean up stale PR preview documentation folders Sep 26, 2025

Copilot AI requested a review from leofang September 26, 2025 03:30

Copilot finished work on behalf of leofang September 26, 2025 03:30

leofang requested changes Sep 26, 2025

View reviewed changes

ci/test-cleanup-pr-previews Outdated Show resolved Hide resolved

ci/README-cleanup-pr-previews.md Outdated Show resolved Hide resolved

Copilot started work on behalf of leofang September 26, 2025 03:45 View session

Changes before error encountered

3262727

Co-authored-by: leofang <[email protected]>

Copilot AI requested a review from leofang September 26, 2025 03:50

leofang requested changes Sep 26, 2025

View reviewed changes

Copilot started work on behalf of leofang September 26, 2025 04:15 View session

Refactor cleanup-pr-previews script based on feedback

2d77199

Co-authored-by: leofang <[email protected]>

Copilot finished work on behalf of leofang September 26, 2025 04:24

Copilot AI requested a review from leofang September 26, 2025 04:24

manual fix

724aed6

leofang requested changes Sep 26, 2025

View reviewed changes

ci/cleanup-pr-previews Show resolved Hide resolved

ci/cleanup-pr-previews Outdated Show resolved Hide resolved

Copilot started work on behalf of leofang September 26, 2025 04:41 View session

Add back jq check and replace cloning with git worktree

503a002

Co-authored-by: leofang <[email protected]>

Copilot finished work on behalf of leofang September 26, 2025 04:46

Copilot AI requested a review from leofang September 26, 2025 04:46

[pre-commit.ci] auto code formatting

c44f63a

leofang requested changes Sep 26, 2025

View reviewed changes

ci/cleanup-pr-previews Outdated Show resolved Hide resolved

Copilot started work on behalf of leofang September 26, 2025 05:06 View session

leofang force-pushed the copilot/fix-1f857a34-0e39-4d11-8895-dd0c23570e9c branch from bab10c6 to 241923e Compare September 26, 2025 14:22

ensure temp dir is under the same root & gh-pages is up-to-date

14aaca3

leofang requested changes Sep 26, 2025

View reviewed changes

Copilot started work on behalf of leofang September 26, 2025 14:51 View session

Fix worktree cleanup to preserve unpushed changes

2a7b4cc

Co-authored-by: leofang <[email protected]>

Copilot AI requested a review from leofang September 26, 2025 14:55

Copilot finished work on behalf of leofang September 26, 2025 14:55

leofang reviewed Sep 26, 2025

View reviewed changes

Copilot started work on behalf of leofang September 26, 2025 15:02 View session

Implement safer worktree cleanup with proper git registration handling

49bdbce

Co-authored-by: leofang <[email protected]>

Copilot finished work on behalf of leofang September 26, 2025 15:06

Copilot AI requested a review from leofang September 26, 2025 15:06

Copilot started work on behalf of leofang October 31, 2025 15:00 View session

Add scheduled workflow to clean up PR preview documentation

f4d0dfa

Co-authored-by: leofang <[email protected]>

Copilot AI changed the title ~~CI: Add script to clean up stale PR preview documentation folders~~ CI: Add script and automated workflow to clean up stale PR preview documentation folders Oct 31, 2025

Copilot finished work on behalf of leofang October 31, 2025 15:04

leofang reviewed Oct 31, 2025

View reviewed changes

.github/workflows/cleanup-pr-previews.yml Outdated Show resolved Hide resolved

leofang reviewed Oct 31, 2025

View reviewed changes

.github/workflows/cleanup-pr-previews.yml Outdated Show resolved Hide resolved

Apply suggestions from code review

c166bdc

[pre-commit.ci] auto code formatting

9e9d85d

leofang marked this pull request as ready for review October 31, 2025 15:19

leofang added enhancement Any code-related improvements P1 Medium priority - Should do CI/CD CI/CD infrastructure labels Oct 31, 2025

leofang added this to the cuda-python 13-next, 12-next milestone Oct 31, 2025

leofang requested a review from cpcloud October 31, 2025 15:20

	# The steps below are executed only when building on main.
	- name: Remove doc preview
	if: ${{ github.ref_name == 'main' }}
	uses: JamesIves/github-pages-deploy-action@v4
	with:
	git-config-name: cuda-python-bot
	git-config-email: [email protected]
	folder: ${{ inputs.source-folder }}
	target-folder: docs/pr-preview/pr-${{ inputs.pr-number }}/
	commit-message: "Clean up doc preview for PR ${{ inputs.pr-number }} (${{ github.sha }})"

	- name: Leave a comment after removal
	if: ${{ github.ref_name == 'main' }}
	uses: marocchino/sticky-pull-request-comment@v2
	with:
	header: pr-preview
	number: ${{ inputs.pr-number }}
	hide_and_recreate: true
	hide_classify: "OUTDATED"
	message: \|
	Doc Preview CI
	:---:
	Preview removed because the pull request was closed or merged.

CI: Add script and automated workflow to clean up stale PR preview documentation folders #1031

Are you sure you want to change the base?

CI: Add script and automated workflow to clean up stale PR preview documentation folders #1031

Uh oh!

Conversation

Copilot AI commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Script Interface

Automated Workflow

Comments on the Issue (you are @copilot in this section)

Uh oh!

copy-pr-bot bot commented Sep 26, 2025

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

leofang commented Sep 26, 2025

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

leofang Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

leofang Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

cpcloud commented Oct 30, 2025

Uh oh!

leofang commented Oct 31, 2025

Uh oh!

Copilot AI commented Oct 31, 2025

Uh oh!

Uh oh!

Uh oh!

leofang commented Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Sep 26, 2025 •

edited

Loading