Skip to content

Add AI-powered PR review system for Playwright + TypeScript#1

Merged
yhAutomationQA merged 9 commits into
mainfrom
feature/ai-pr-review
May 11, 2026
Merged

Add AI-powered PR review system for Playwright + TypeScript#1
yhAutomationQA merged 9 commits into
mainfrom
feature/ai-pr-review

Conversation

@yhAutomationQA
Copy link
Copy Markdown
Owner

@yhAutomationQA yhAutomationQA commented May 11, 2026

Description

Adds an AI PR review system that automatically analyzes pull request diffs for Playwright + TypeScript test framework code. Detects flaky patterns, poor locators, missing assertions, DRY violations, and architecture issues.

Files Added

File Purpose
.ai/skills/playwright-pr-review.md Skill definition with severity levels, 7-category review checklist, scoring formula
.ai/prompts/pr-review.prompt.md Prompt template with diff injection point
.github/PULL_REQUEST_TEMPLATE.md Structured PR template with flakiness + framework compliance checks
.github/scripts/ai-review.mjs AI API caller that sends diffs to OpenAI for analysis
.github/workflows/ai-pr-review.yml Full CI workflow: diff extraction, AI review, PR comment, check run, auto-labeling

How It Works

  1. PR opened to main — workflow triggers automatically
  2. Extracts the git diff (excluding binaries and lockfiles)
  3. Sends diff to OpenAI (GPT-4o) for analysis against Playwright + TypeScript best practices
  4. AI reviews the diff across 7 categories: flaky patterns, locator usage, assertions, anti-patterns, TypeScript issues, architecture violations, and DRY violations
  5. Posts a structured review as a PR comment with severity labels and quality score
  6. Creates a check run for branch protection rule compatibility
  7. Auto-labels the PR: passed / warning / failed
  8. No auto-merge — human approval is always required

Setup Required After Merge

Add an OPENAI_API_KEY secret in repository Settings > Secrets and variables > Actions, then enable the AI PR Review / review check in branch protection rules for main.

@github-actions github-actions Bot added the ai-pr-review-passed AI PR review passed (score >= 80) label May 11, 2026
@github-actions
Copy link
Copy Markdown

🤖 AI PR Review

AI review failed: OpenAI API error 404: {
"error": {
"message": "The model GPT-5.5 does not exist or you do not have access to it.",
"type": "invalid_request_error",
"param": null,
"code": "model_not_found"
}
}

AI PR Review

Review could not be completed.

Error: OpenAI API error 404: {
"error": {
"message": "The model GPT-5.5 does not exist or you do not have access to it.",
"type": "invalid_request_error",
"param": null,
"code": "model_not_found"
}
}

(AI review script failed -- see logs)


This review was generated automatically by the AI PR Review workflow. Human approval is required before merge.

@github-actions github-actions Bot added ai-pr-review-passed AI PR review passed (score >= 80) and removed ai-pr-review-passed AI PR review passed (score >= 80) labels May 11, 2026
@github-actions github-actions Bot added ai-pr-review-passed AI PR review passed (score >= 80) and removed ai-pr-review-passed AI PR review passed (score >= 80) labels May 11, 2026
@github-actions
Copy link
Copy Markdown

🤖 AI PR Review

AI review failed: OpenAI API error 404: {
"error": {
"message": "The model GPT-5.5 does not exist or you do not have access to it.",
"type": "invalid_request_error",
"param": null,
"code": "model_not_found"
}
}

AI PR Review

Review could not be completed.

Error: OpenAI API error 404: {
"error": {
"message": "The model GPT-5.5 does not exist or you do not have access to it.",
"type": "invalid_request_error",
"param": null,
"code": "model_not_found"
}
}

(AI review script failed -- see logs)


This review was generated automatically by the AI PR Review workflow. Human approval is required before merge.

@github-actions github-actions Bot added ai-pr-review-passed AI PR review passed (score >= 80) and removed ai-pr-review-passed AI PR review passed (score >= 80) labels May 11, 2026
@sonarqubecloud
Copy link
Copy Markdown

@yhAutomationQA yhAutomationQA self-assigned this May 11, 2026
@yhAutomationQA yhAutomationQA merged commit b0214dd into main May 11, 2026
14 checks passed
@yhAutomationQA yhAutomationQA deleted the feature/ai-pr-review branch May 11, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-pr-review-passed AI PR review passed (score >= 80)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant