Skip to content

Conversation

@neubig
Copy link
Contributor

@neubig neubig commented Oct 20, 2025

Summary

Fixes #826

The assign-reviews.yml workflow was assigning reviewers to PRs that already had reviewers assigned, causing duplicate assignments. This happened because the prompt was not explicit enough about how to check for existing reviewers using the GitHub API.

Changes

This PR updates the "Need Reviewers" section of the workflow prompt to:

  1. Add explicit GitHub API instructions: Clearly specify that the agent should use GET /repos/{owner}/{repo}/pulls/{pull_number} to fetch PR details
  2. Check both reviewer arrays: Explicitly check both requested_reviewers and requested_teams arrays
  3. Include concrete examples: Provide actual API response examples showing:
    • A PR that DOES NOT need reviewers (has entries in the arrays - should be skipped)
    • A PR that DOES need reviewers (empty arrays - should be processed)
  4. Emphasize the skip condition: Make it clear that if either array has any entries, the PR should be skipped

Example

The updated prompt now includes concrete examples like:

Example API response showing a PR that DOES NOT need reviewers (skip this):
{
  "requested_reviewers": [{"login": "someuser"}],
  "requested_teams": []
}

Example API response showing a PR that DOES need reviewers (process this):
{
  "requested_reviewers": [],
  "requested_teams": []
}

Testing

This is a configuration change to a GitHub workflow file. The change will be validated when the workflow runs next (scheduled daily at 12 PM UTC or can be triggered manually).

Impact

This change should prevent the workflow from:

  • Assigning duplicate reviewers to PRs that already have reviewers
  • Creating redundant assignment comments on PRs

The agent will now properly identify PRs that truly have no reviewers assigned and skip those that already have reviewers.

@neubig can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Base Image Docs / Tags
golang golang:1.21-bookworm Link
java eclipse-temurin:17-jdk Link
python nikolaik/python-nodejs:python3.12-nodejs22 Link

Pull (multi-arch manifest)

docker pull ghcr.io/openhands/agent-server:f41a0db-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-f41a0db-python \
  ghcr.io/openhands/agent-server:f41a0db-python

All tags pushed for this build

ghcr.io/openhands/agent-server:f41a0db-golang
ghcr.io/openhands/agent-server:v1.0.0a3_golang_tag_1.21-bookworm_binary
ghcr.io/openhands/agent-server:f41a0db-java
ghcr.io/openhands/agent-server:v1.0.0a3_eclipse-temurin_tag_17-jdk_binary
ghcr.io/openhands/agent-server:f41a0db-python
ghcr.io/openhands/agent-server:v1.0.0a3_nikolaik_s_python-nodejs_tag_python3.12-nodejs22_binary

The f41a0db tag is a multi-arch manifest (amd64/arm64); your client pulls the right arch automatically.

The workflow was assigning reviewers to PRs that already had reviewers
assigned. This happened because the prompt was not explicit enough about
how to check for existing reviewers using the GitHub API.

Changes:
- Added explicit instructions to check both 'requested_reviewers' and
  'requested_teams' arrays in the GitHub API response
- Included concrete API response examples showing PRs with and without
  reviewers
- Emphasized that PRs with any entries in either array should be skipped

This ensures the agent will properly detect when a PR already has
reviewers and avoid duplicate assignments.

Fixes #826

Co-authored-by: openhands <[email protected]>
The previous fix only checked requested_reviewers/requested_teams arrays,
but GitHub automatically removes reviewers from these arrays when they
submit a review (COMMENTED, APPROVED, etc.). This meant the workflow could
still assign duplicate reviewers if someone had already reviewed the PR.

Now the workflow checks three conditions:
1. requested_reviewers array (pending review requests)
2. requested_teams array (pending team review requests)
3. submitted reviews via /pulls/{number}/reviews endpoint

This catches the exact case from PR #520 where @enyst reviewed on Oct 17
(removed from requested_reviewers) but @neubig was assigned on Oct 20
because the workflow only saw an empty requested_reviewers array.

Co-authored-by: openhands <[email protected]>
@neubig neubig marked this pull request as ready for review October 20, 2025 22:23
@neubig neubig requested a review from simonrosenberg October 20, 2025 22:23
@blacksmith-sh
Copy link
Contributor

blacksmith-sh bot commented Oct 29, 2025

[Automatic Post]: This PR seems to be currently waiting for review. @simonrosenberg, could you please take a look when you have a chance?

4. A PR needs reviewers ONLY if ALL of these are true:
- The "requested_reviewers" array is empty (no pending review requests)
- The "requested_teams" array is empty (no pending team review requests)
- The reviews array is empty (no reviews have been submitted yet)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems a bit tricky, it's also possible to review a part of a PR and not the rest? 🤔

I'm not sure what is best:

  • PRs with two reviewers
  • PRs with a comment, and no more response/follow-up review

Probably two reviewers? 🤔

We can try it like this, see how it works, but idk, we might want to revisit.

Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a little doubt, but maybe we can try this prompt and see how we like the effect! Not sure how Simon thinks about it?

@enyst
Copy link
Collaborator

enyst commented Nov 1, 2025

Looks like this PR is needed:

In that case, the PR

  • had a reviewer (assigned)
  • it was approved, too
  • and yet, another reviewer was assigned.

@enyst enyst merged commit 7e6774f into main Nov 1, 2025
16 checks passed
@enyst enyst deleted the openhands/fix-duplicate-reviewer-assignment branch November 1, 2025 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatic review assignment workflow assigned duplicate reviewers

4 participants