The corpus is built and reproducible (corpus/README.md, npm run corpus:fetch), but every machine unit in it predates 2025. That single gap blocks the two live questions: whether to re-weight categories by measured lift (#71), and how to recalibrate the score (#70).
What exists
| Source |
Era |
Task shape |
Units |
| RAID (Dugan et al. 2024, MIT) |
2024 |
in-domain continuation |
373 machine |
| HC3 (Guo et al. 2023, CC-BY-SA) |
Dec 2022 |
assistant answers, paired on question |
406 machine |
Both are external, which is the point — a corpus generated by the person evaluating the detector measures their prompting as much as the detector.
What is missing
A modern instruction-tuned model writing in the registers this skill actually targets: LinkedIn posts, blog drafts, investor emails, release notes, README prose. Nothing in the corpus is that. RAID writes recipes and encyclopedia entries; HC3 answers questions.
This matters more than it sounds because the genre's whole premise is that model habits shift. Anecdotally GPT-5.x cut em-dash usage sharply; if true, several rules here are already measuring a habit that has moved.
The design problem, stated honestly
Every option has a bias:
- Generate locally. Register-matched and current, but the prompts get written by someone who knows the pattern list, so the result partly measures the prompting. Also single-family unless several providers are available.
- Find a current public dataset. No self-generation bias, but nothing well-licensed covers 2026 models in marketing/professional registers yet.
- Collect in the wild. Real LinkedIn and blog text suspected of being AI-drafted. Highest external validity, no reliable ground truth — which defeats the corpus's core property that labels come from provenance rather than judgment.
- Solicit contributions. Ask users to submit AI-drafted text they wrote with a model, with the model named. Real provenance, real register, slow to accumulate. The false-positive form is the template.
Option 4 plus a clearly-labelled option 1 leg is probably the honest combination: attested contributions as the primary set, local generation as a labelled secondary with its bias stated in the README.
Acceptance
Whatever lands should satisfy the public-claim gate already written into corpus/README.md: n >= 100 per claim cell, more than one register people write in today, confidence intervals, and no number quoted anywhere until it does.
The corpus is built and reproducible (
corpus/README.md,npm run corpus:fetch), but every machine unit in it predates 2025. That single gap blocks the two live questions: whether to re-weight categories by measured lift (#71), and how to recalibrate the score (#70).What exists
Both are external, which is the point — a corpus generated by the person evaluating the detector measures their prompting as much as the detector.
What is missing
A modern instruction-tuned model writing in the registers this skill actually targets: LinkedIn posts, blog drafts, investor emails, release notes, README prose. Nothing in the corpus is that. RAID writes recipes and encyclopedia entries; HC3 answers questions.
This matters more than it sounds because the genre's whole premise is that model habits shift. Anecdotally GPT-5.x cut em-dash usage sharply; if true, several rules here are already measuring a habit that has moved.
The design problem, stated honestly
Every option has a bias:
Option 4 plus a clearly-labelled option 1 leg is probably the honest combination: attested contributions as the primary set, local generation as a labelled secondary with its bias stated in the README.
Acceptance
Whatever lands should satisfy the public-claim gate already written into
corpus/README.md: n >= 100 per claim cell, more than one register people write in today, confidence intervals, and no number quoted anywhere until it does.