-
Notifications
You must be signed in to change notification settings - Fork 45
feat: automate stale and abandon PR detection and labeling #41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pemamian
wants to merge
21
commits into
Universal-Commerce-Protocol:main
Choose a base branch
from
pemamian:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
242ac2a
feat: automate stale and abandon PR detection and labeling
pemamian 8b8e940
Merge pull request #1 from pemamian/stale-pr-detection
pemamian 2586322
chore: fix PyGithub deprecation warning in stale tracker
pemamian 2c22825
chore: address review comments on stale tracker by adding timeouts, o…
pemamian dbc66c6
Merge branch 'stale-pr-detection'
pemamian 9ea0bc9
chore: ignore draft pull requests in stale PR tracker
pemamian 20b7817
feat: improvements on stale cronjob.
pemamian 9d8fa9e
feat: initial version of label synchronization workflow
pemamian 6b606f7
feat: fix auth issue label syncer
pemamian 1d6bd6b
feat: fix permission issue for label sync
pemamian 98a2e4c
feat: getting rid of github toke pass.
pemamian c26b441
feat: additional labels needed for rules based labeling
pemamian 2924f59
feat: README and rules for rule based labeling.
pemamian c4d4c25
feat: implement automated PR triage routing and validation logic with…
pemamian b4e1173
Merge pull request #2 from pemamian/pr-rules-engine
pemamian b2437e0
feat: implement guardrail to prevent unauthorized removal of active r…
pemamian 12927e0
Merge branch 'main' of github.com:pemamian/python-sdk-peyman into pr-…
pemamian d8ca0c1
Merge pull request #4 from pemamian/pr-rules-engine
pemamian 752c7d1
feat: enforce label guardrails as the primary triage rule and add sup…
pemamian 81b9344
Merge branch 'main' into pr-rules-engine
pemamian a0a1169
Merge pull request #5 from pemamian/pr-rules-engine
pemamian File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| - name: status:needs-triage | ||
| color: 'FBCA04' | ||
| description: Signal that the PR is ready for human triage | ||
|
|
||
| - name: status:under-review | ||
| color: '1D76DB' | ||
|
|
||
| - name: status:stale-review | ||
| color: '6F7A8A' | ||
| description: Applied if a PR is waiting on a reviewer for too long | ||
|
|
||
| - name: status:blocked | ||
| color: 'D93F0B' | ||
|
|
||
| - name: status:ready-to-merge | ||
| color: '0E8A16' | ||
| description: Signaling to the DevOps team that it can be safely merged | ||
|
|
||
| - name: status:stale | ||
| color: '6F7A8A' | ||
| description: Automatically applied if a PR is blocked waiting for the author's response for 30 days | ||
|
|
||
| - name: status:merged | ||
| color: '6F42C1' | ||
| description: Signify the completion of its lifecycle and provide clear visibility for reporting | ||
|
|
||
| - name: area:payments | ||
| color: '0052CC' | ||
|
|
||
| - name: devops | ||
| color: 'FFD1DC' | ||
|
|
||
| - name: gov:needs-tc-review | ||
| color: 'D93F0B' | ||
|
|
||
| - name: gov:needs-gc-review | ||
| color: 'FDE5BE' | ||
|
|
||
| - name: gov:approved | ||
| color: 'C2E0C6' | ||
| description: Triggers the final code ownership checks | ||
|
|
||
| # ============================================================================== | ||
| # PROPOSED DYNAMIC & DOMAIN TRIAGE LABELS (For Team Review) | ||
| # ============================================================================== | ||
| - name: gov:gc-approved | ||
| color: 'B5D9F2' | ||
| description: Signifies Governance Council approved | ||
|
|
||
| - name: gov:tc-approved | ||
| color: 'C2E0C6' | ||
| description: Signifies Technical Council approved | ||
|
|
||
| - name: gov:maintainer-approved | ||
| color: 'D4C5F9' | ||
| description: Signifies Domain Expert / Maintainer approved | ||
|
|
||
| - name: status:review-needed-maintainers | ||
| color: 'FBCA04' | ||
| description: Missing standard maintainer reviews | ||
|
|
||
| - name: status:review-needed-payments-maintainers | ||
| color: 'FBCA04' | ||
| description: Missing Payments domain maintainer reviews | ||
|
|
||
| - name: status:payments-approved | ||
| color: 'D4E5FF' | ||
| description: Payments domain approvals met | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| name: Sync Labels | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - '.github/labels.yml' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| labeler: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| issues: write | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Sync labels | ||
| uses: EndBug/label-sync@v2 | ||
| with: | ||
| config-file: .github/labels.yml | ||
| delete-other-labels: false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| --- | ||
| name: PR Stale and Abandon Tracker | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '17 2 * * *' # Daily at 2:17 AM UTC to avoid peak scheduler hours | ||
| workflow_dispatch: # Allows manual trigger for testing | ||
|
|
||
| permissions: | ||
| pull-requests: write | ||
| issues: write | ||
| contents: read | ||
|
|
||
| jobs: | ||
| stale_tracker: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up uv | ||
| uses: astral-sh/setup-uv@v5 | ||
|
|
||
| - name: Trace and Mark Stale PRs | ||
| run: uv run .github/workflows/scripts/routing/pr-cron-stale-abandon.py | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| name: PR Triage Automation | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, ready_for_review, synchronize, labeled] | ||
| issue_comment: | ||
| types: [created] | ||
| check_suite: | ||
| types: [completed] | ||
|
|
||
| permissions: | ||
| pull-requests: write | ||
| issues: write | ||
| contents: read | ||
|
|
||
| jobs: | ||
| triage_realtime: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| if: github.event.pull_request.draft == false | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up uv | ||
| uses: astral-sh/setup-uv@v5 | ||
|
|
||
| - name: Validate Routing Configurations | ||
| run: uv run .github/workflows/scripts/routing/validate-routing.py | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Check and Update PR Labels | ||
| run: uv run .github/workflows/scripts/routing/pr-triage-automation.py | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| # UCP PR Triage & Review Routing Automation | ||
|
|
||
| This directory houses the modular, configuration-driven Python rules engine designed to automate pull request ingestion triage, blocked/resume lifecycles, dynamically scoped organizational reviews verification, and inactivity stale-PR scans. | ||
|
|
||
| --- | ||
|
|
||
| ## Directory Directory Tree Layout | ||
|
|
||
| ``` | ||
| scripts/routing/ | ||
| ├── UCP_PR_REVIEW_ROUTING.yml # Centralized rules configuration mapping | ||
| ├── pr-triage-automation.py # Real-time webhook trigger runner (run by GHA workflow) | ||
| ├── pr-cron-stale-abandon.py # Daily stale/abandon inactivity scan runner (run by cron GHA workflow) | ||
| ├── validate-routing.py # Pretty CLI validation tool checking YAML syntax and org group existence | ||
| ├── test_routing.py # Pretty unit test suite executing mocked verifications locally | ||
| └── triage/ | ||
| ├── github_api.py # Encapsulates dynamic PyGithub API calls and organization caching | ||
| ├── models.py # Standard shared dataclasses and strict LABEL_ Constants | ||
| └── rules.py # Core abstract BaseRule class and concrete check implementations | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 1. Centralized Review Routing Configuration | ||
|
|
||
| All folder-to-reviewer-mappings and label states are decoupled completely from the execution codebase and stored in [**`UCP_PR_REVIEW_ROUTING.yml`**](./UCP_PR_REVIEW_ROUTING.yml). This allows developers to flexibly configure or update rules without editing Python modules. | ||
|
|
||
| ### Configuration Rule Structure: | ||
| ```yaml | ||
| routing_rules: | ||
| - name: "Core Protocol & Spec" | ||
| patterns: | ||
| - "schemas/**/*.json" | ||
| - "spec/**/*.md" | ||
| review_requirements: | ||
| # Maps fully qualified GitHub team handles to approval thresholds and status labels: | ||
| "@Universal-Commerce-Protocol/tech-council": | ||
| threshold: "majority" # Require TC majority approval or status:tc-majority-approved | ||
| needs_review_label: "gov:needs-tc-review" | ||
| approved_label: "gov:tc-approved" | ||
| "@Universal-Commerce-Protocol/maintainers": | ||
| threshold: 1 | ||
| needs_review_label: "status:review-needed-maintainers" | ||
| approved_label: "gov:maintainer-approved" | ||
| ``` | ||
|
|
||
| * **`patterns`**: List of path glob filters determining if modifications in the PR match this rule. | ||
| * **`review_requirements`**: Maps dynamic, fully qualified organization team handles to: | ||
| * `threshold`: Integer count (e.g. `1`, `2`) or `"majority"`. | ||
| * `needs_review_label`: Staging label applied when approvals are below the threshold. | ||
| * `approved_label`: Calming tint approval label applied dynamically once approvals are satisfied. | ||
|
|
||
| --- | ||
|
|
||
| ## 2. Core Rules Scaffolding (`triage/rules.py`) | ||
|
|
||
| Triage logic is organized into specialized rule subclasses inheriting from `BaseRule`: | ||
|
|
||
| 1. **`FileRoutingRule`**: Compiles modified files against the configuration and dynamically applies the corresponding `needs_review_label` and removes the `approved_label`. | ||
| 2. **`ReviewerApprovalRule`**: Tracks active `pygithub` approvals against the resolved organization team members list: | ||
| * **Superpower Override**: If a designated superpower user (like Amit `amithanda`) approves, all TC and GC rules are satisfied instantly, transitioning the PR to `gov:approved` / `status:ready-to-merge`. | ||
| * **Label Application & Removal Guardrails**: | ||
| * Restricts `gov:tc-approved` and `status:tc-majority-approved` application. If applied by an unauthorized user outside Tech Council or DevOps, the script revokes the label. | ||
| * If an active `needs_review_label` is manually **unlabeled (removed)** by an unauthorized user while approvals are still pending, the engine **automatically re-applies the label** dynamically and posts a warning comment on the PR. | ||
| * **SDK Relaxed Mode**: Repositories matching `sdk` or `meeting-minutes` automatically default team thresholds to `1` to expedite SDK review cycles. | ||
| 3. **`LabelLifecycleRule`**: Resolves blocked feedback loops. Clears `Label.LABEL_BLOCKED` and restores `Label.LABEL_UNDER_REVIEW` when the author pushes a new commit or comments on the PR. | ||
| 4. **`StalePRRule`**: Scans active timestamps: | ||
| * Under-review PRs inactive for 30 days are labeled `status:stale-review` and `status:needs-triage`. | ||
| * Blocked PRs inactive for 37 days are labeled `status:abandon-candidate`. | ||
|
|
||
| --- | ||
|
|
||
| ## 3. Local Dry-Run & Validation Utilities | ||
|
|
||
| ### YAML and Taxonomy Validation: | ||
| Developers making changes to `UCP_PR_REVIEW_ROUTING.yml` can test and validate their configurations locally using: | ||
| ```bash | ||
| export GH_TOKEN="your_github_personal_access_token" | ||
| uv run .github/workflows/scripts/routing/validate-routing.py | ||
| ``` | ||
| This utility checks: | ||
| 1. **YAML Syntax**: Verifies structure correctness. | ||
| 2. **Taxonomy Matcher**: Cross-references labels with `.github/labels.yml` to prevent styling typos. | ||
| 3. **Dynamic Org Team Check**: Dynamically calls the API to verify that all configured dynamic handles actually exist in the active organization (gracefully skipped with a warning on local forks). | ||
|
|
||
| ### Triage dry-runs: | ||
| You can evaluate the rules engine output on any PR locally without committing live updates to GitHub by appending the `--dry-run` option flag: | ||
| ```bash | ||
| export GH_TOKEN="your_token" | ||
| uv run .github/workflows/scripts/routing/pr-triage-automation.py --dry-run | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 4. Local Unit Testing | ||
|
|
||
| The rules engine is backed by a mock-based unit test suite verifying all edge conditions. Execute tests locally using: | ||
| ```bash | ||
| export GH_TOKEN="your_token" | ||
| uv run .github/workflows/scripts/routing/test_routing.py | ||
| ``` | ||
| This outputs high-visibility boxed **Test Run Summary Results** with counts of executed, passed, failed, and errored test cases. |
66 changes: 66 additions & 0 deletions
66
.github/workflows/scripts/routing/UCP_PR_REVIEW_ROUTING.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # UCP PR Review Routing Configuration | ||
| # This file acts as the configuration map for pr-triage-automation script. | ||
| # It defines glob patterns for changed files and associates them with required reviewer sets and label states. | ||
|
|
||
| routing_rules: | ||
| - name: "Governance Files" | ||
| patterns: | ||
| - "LICENSE" | ||
| - "GOVERNANCE.md" | ||
| - "CONTRIBUTING.md" | ||
| - ".github/CODEOWNERS" | ||
| review_requirements: | ||
| "@Universal-Commerce-Protocol/pemamian": | ||
| threshold: 1 | ||
| needs_review_label: "gov:needs-gc-review" | ||
| approved_label: "gov:gc-approved" | ||
|
|
||
| - name: "Core Protocol & Spec" | ||
| patterns: | ||
| - "schemas/**/*.json" | ||
| - "spec/**/*.md" | ||
| review_requirements: | ||
| "@Universal-Commerce-Protocol/tech-council": | ||
| threshold: "majority" | ||
| needs_review_label: "gov:needs-tc-review" | ||
| approved_label: "gov:tc-approved" | ||
| "@Universal-Commerce-Protocol/maintainers": | ||
| threshold: 1 | ||
| needs_review_label: "status:review-needed-maintainers" | ||
| approved_label: "gov:maintainer-approved" | ||
|
|
||
| - name: "Infrastructure & Tooling" | ||
| patterns: | ||
| - ".github/workflows/**" | ||
| - ".gitignore" | ||
| - ".pre-commit-config.yaml" | ||
| - "pyproject.toml" | ||
| - "uv.lock" | ||
| review_requirements: | ||
| "@Universal-Commerce-Protocol/devops-maintainers": | ||
| threshold: 1 | ||
| needs_review_label: "status:needs-triage" | ||
| approved_label: "status:under-review" | ||
|
|
||
| - name: "SDK Code & Maintenance" | ||
| patterns: | ||
| - "src/**" | ||
| - "templates/**" | ||
| review_requirements: | ||
| "@Universal-Commerce-Protocol/maintainers": | ||
| threshold: 1 | ||
| needs_review_label: "status:needs-triage" | ||
| approved_label: "status:under-review" | ||
|
|
||
| - name: "Payments Custom Review Rules" | ||
| patterns: | ||
| - "src/components/payments/**" | ||
| review_requirements: | ||
| "@Universal-Commerce-Protocol/maintainers": | ||
| threshold: 1 | ||
| needs_review_label: "status:review-needed-payments-maintainers" | ||
| approved_label: "status:payments-approved" | ||
| "@Universal-Commerce-Protocol/tech-council": | ||
| threshold: 1 | ||
| needs_review_label: "gov:needs-tc-review" | ||
| approved_label: "gov:tc-approved" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.