feat(report): credibility scoring pipeline for report-an-issue#35
Merged
Conversation
Replaces the bare "POST to GitHub" report action with a strict scoring pipeline that filters nonsense + destructive submissions before they reach the auto-fix queue, while respecting wisdom-of-the-crowd corroboration (3 independent reports on the same URL force verified). Pipeline: Zod parse → resolve reporter → hard filters (HF1-HF10) → Turnstile → dedup search → Haiku triage → score R+Q+C+A+P → bucket → GitHub issue Strict thresholds (locked in plan): <30 silent-reject no issue created 30-54 low-confidence issue + label, agent skips, 14d auto-close 55-74 needs-human issue + label, human review ≥75 verified-report issue + label, agent auto-fixes Overrides: destructive classification → forced needs-human regardless of score 3 corroborations on same URL → upgrade existing to verified AI failure → cap at needs-human (never silent-rejects legit report) severityHint=critical + score≥60 → promote to verified Mkan-specifics: - Adapter uses @/lib/auth + existing assertRateLimit + Upstash KV - Adds 'report' + 'report-tenant' buckets to existing rateLimiters map - HOST and GUEST roles get reputation bases of 14 and 10 respectively - Reuses existing Upstash infrastructure (already in mkan's deps) Anti-abuse measures: - 60s client-side cooldown mirrors HF9 (fixes triple-click case) - Symmetric success toast denies feedback to spammers - Anonymous requires Turnstile, capped at base reputation 4 - Per-tenant rate limit catches coordinated abuse Plan: /Users/abdout/.claude/plans/read-report-an-issue-glistening-wave.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The old report-issue tests covered the pre-pipeline bare-fetch action. The action is now a thin wrapper around runReportPipeline; the old contract (throwing on missing token, retrying on 422, posting an ack comment) lives inside the pipeline now and is covered by the canonical src/lib/report/__tests__ suite. Action tests now smoke-test the wiring: - input forwarding to pipeline - bucket-aware return shape (issueNumber surfaces only for verified-report) - failure pass-through Deleted tests/components/common/report-issue.test.tsx — the ReportIssue component is now an async server component that auth()s before rendering the client dialog. RTL can't render async server components; component behavior is covered by Playwright at the integration level. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the bare "POST to GitHub" report-an-issue action with a strict credibility scoring pipeline that filters nonsense + destructive submissions before they reach the auto-fix queue, while respecting wisdom-of-the-crowd corroboration.
Plan:
/Users/abdout/.claude/plans/read-report-an-issue-glistening-wave.mdPipeline
Strict thresholds
<3030-5455-74≥75Mkan-specifics
@/lib/auth+ existingassertRateLimit+ Upstash KV (zero new infrastructure)report+report-tenantbuckets to existingrateLimitersmapFiles
Required env
Upstash already configured.
Verification
pnpm typecheck— 0 errors.Bootstrap (run once after merge)
🤖 Generated with Claude Code