Skip to content

Validate AI-suggested labels against repository labels #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
swissspidy merged 4 commits into from
Jan 20, 2026
Merged

Validate AI-suggested labels against repository labels #195

swissspidy merged 4 commits into from
Jan 20, 2026

Conversation

Copy link
Contributor

Copilot AI commented Jan 20, 2026 •
edited
Loading

Plan: Only use existing labels in issue triage workflow

  • Understand the current implementation in .github/workflows/reusable-issue-triage.yml
  • Identify the two "Apply labels" steps that need to be updated
  • Modify the "Apply labels" step in the triage-new-item job to filter labels against AVAILABLE_LABELS
  • Modify the "Apply labels" step in the triage-single-item job to filter labels against AVAILABLE_LABELS
  • Add null checks for AVAILABLE_LABELS to prevent errors
  • Request code review and address feedback
  • Implement case-insensitive label comparison
  • Optimize to use single-pass reduce for partitioning labels
  • Run security checks

Summary

Successfully implemented label validation to prevent AI hallucinations in the issue triage workflow. Both "Apply labels" steps now:

  • Parse and validate against AVAILABLE_LABELS with case-insensitive comparison
  • Filter out any labels that don't exist in the repository
  • Use efficient single-pass reduce to partition labels into valid/invalid
  • Log skipped invalid labels for debugging
  • Include proper null checks to prevent errors

No security vulnerabilities found.

Original prompt

This section details on the original issue you should resolve

<issue_title>Issue triage: only use existing labels</issue_title>
<issue_description>In .github/workflows/reusable-issue-triage.yml, the "Apply labels" step should skip labels that aren't in AVAILABLE_LABELS, to avoid AI hallucinations</issue_description>

Comments on the Issue (you are @copilot in this section)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@github-actions github-actions bot added enhancement New feature or request scope:meta labels Jan 20, 2026
Copilot AI changed the title [WIP] Fix apply labels step to skip non-existing labels Validate AI-suggested labels against repository labels Jan 20, 2026
Copilot AI requested a review from swissspidy January 20, 2026 10:08
@swissspidy swissspidy requested a review from Copilot January 20, 2026 10:18
Copilot AI reviewed Jan 20, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds validation to prevent AI-suggested labels from being applied if they don't exist in the repository's available labels, addressing the issue of AI hallucinating non-existent labels.

Changes:

  • Added label validation logic that filters AI suggestions against AVAILABLE_LABELS before applying them
  • Added logging to track when invalid labels are skipped
  • Added null safety checks for missing AVAILABLE_LABELS environment variable

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@swissspidy swissspidy marked this pull request as ready for review January 20, 2026 11:13
@swissspidy swissspidy requested a review from a team as a code owner January 20, 2026 11:13
@swissspidy swissspidy merged commit fe38626 into main Jan 20, 2026
12 checks passed
@swissspidy swissspidy deleted the copilot/fix-apply-labels-step branch January 20, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@swissspidy swissspidy Awaiting requested review from swissspidy

Assignees

@swissspidy swissspidy

Copilot code review Copilot

Labels

enhancement New feature or request scope:meta

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Issue triage: only use existing labels

2 participants

@swissspidy