Skip to content

Detect AI-Generated Pull Requests Using AI Agents and LLMs #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Cgarg9 opened this issue Mar 10, 2025 · 5 comments
Open

Detect AI-Generated Pull Requests Using AI Agents and LLMs #16

Cgarg9 opened this issue Mar 10, 2025 · 5 comments

Comments

@Cgarg9
Copy link
Collaborator

Cgarg9 commented Mar 10, 2025

Description:
To enhance repository security and maintain code authenticity, implement an AI-driven system that checks whether a Pull Request (PR) was generated using AI. The system should leverage AI agents and an LLM of your choice to analyze PR content, comments, and commit messages.

Tasks:

Select an LLM and set up an AI agent to evaluate PRs.
Define criteria for detecting AI-generated code (e.g., syntax patterns, comment structures, repetition).
Integrate the AI agent with GitHub Actions or another CI/CD pipeline for automated checks.
Provide a report or confidence score indicating AI-generated content probability.
Name the notebook ai_generated_pr_checker.ipynb.
Update the README with implementation details and references.

@Steve-Wayne
Copy link
Contributor

Steve-Wayne commented Mar 19, 2025

Sir do we have to build an ai agent ?? Can lang chain be a option do we have set a criteria on basis on amount of plagiarism

@Cgarg9
Copy link
Collaborator Author

Cgarg9 commented Mar 19, 2025

@Steve-Wayne langchain can also be used to build ai agent. Ideally i want to deploy it to the CI/CD so that it performs a plag check as soon as a PR is raised but for now you can build an agent which when provided with a file analyses it and returns a plag report. Set the threshold what you want for testing.

@Cgarg9
Copy link
Collaborator Author

Cgarg9 commented Mar 19, 2025

However do keep in mind if its not deployed to CI/CD i cant count it as hard so will merge PR with medium label at best.

@Steve-Wayne
Copy link
Contributor

If I build ai agent with langchain then is it possible to integrate it with ci/cd

@Cgarg9
Copy link
Collaborator Author

Cgarg9 commented Mar 20, 2025

@Steve-Wayne i will need to check on that. Ideally there should not be any issues but please check in their docs about deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants