-
Notifications
You must be signed in to change notification settings - Fork 79
feat(RubricEvaluation): implement rubric auto-grading #7941
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
OK
What is the behaviour if the grading is "in-progress"? i.e. when the instructor opens the submission before the grading completes.
That is fine. I suspect we can also remove this 'Evaluate Answers' button since individual/single 'Re-evaluate Answer' is sufficient for manual triggering, and autograded answers will be autograded on finalise anyway.
Autograding logic should go to Also add test cases once ready. |
682ad82
to
9120c4e
Compare
6a48ee7
to
6da72ec
Compare
app/services/course/assessment/answer/prompts/rubric_auto_grading_system_prompt.json
Outdated
Show resolved
Hide resolved
app/services/course/assessment/answer/prompts/rubric_auto_grading_user_prompt.json
Outdated
Show resolved
Hide resolved
client/app/bundles/course/assessment/submission/actions/index.js
Outdated
Show resolved
Hide resolved
client/app/bundles/course/assessment/submission/components/comment/CommentCard.jsx
Outdated
Show resolved
Hide resolved
client/app/bundles/course/discussion/topics/components/cards/CommentCard.tsx
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks mostly good, but there are some comments that need to be addressed. When making changes, remember to make them in the appropriate commits.
Also, there are endpoints called auto_feedback_count and publish_auto_feedback that allow instructor to publish all feedback from a given assessment at once, but currently these only affect programming file feedback by Codaveri.
A user would consider the draft comments generated as "automated feedback", so we should probably rename the button to something like "Publish Automated Programming Feedback" to disambiguate.
38038a7
to
07f5a9e
Compare
style(RubricPanelRow): make category explanation text wrap
…and draft comment generation - Add backend service for rubric auto-grading and AI-generated draft comments - Modify for frontend to enable graders to edit and publish AI-generated draft comments - Add tests and translations for new features
07f5a9e
to
ac24c46
Compare
Description
Implement auto-grading service for rubric-based response questions. OpenAI LLM is used to evaluate student's answers and automatically assign suitable grades for different categories. A draft comment for overall feedback is also automatically generated where teachers can edit and publish later.
Changes made
rubric_auto_grading_service
andrubric_llm_service
Views
Teacher can see suggested grades and comments when opening student's submission. Pressing
Re-evaluate Answer
will rerun the autograding job. This button is in loading state if the answer is currently autograded, with the rubric panel being disabled too.Comments can be modified and published in both the Submission Page and the Comment Center Page.
Notes