Skip to content

Machine corpus has no post-2024 model — blocks re-weighting and recalibration #72

Description

@conorbronsdon

The corpus is built and reproducible (corpus/README.md, npm run corpus:fetch), but every machine unit in it predates 2025. That single gap blocks the two live questions: whether to re-weight categories by measured lift (#71), and how to recalibrate the score (#70).

What exists

Source Era Task shape Units
RAID (Dugan et al. 2024, MIT) 2024 in-domain continuation 373 machine
HC3 (Guo et al. 2023, CC-BY-SA) Dec 2022 assistant answers, paired on question 406 machine

Both are external, which is the point — a corpus generated by the person evaluating the detector measures their prompting as much as the detector.

What is missing

A modern instruction-tuned model writing in the registers this skill actually targets: LinkedIn posts, blog drafts, investor emails, release notes, README prose. Nothing in the corpus is that. RAID writes recipes and encyclopedia entries; HC3 answers questions.

This matters more than it sounds because the genre's whole premise is that model habits shift. Anecdotally GPT-5.x cut em-dash usage sharply; if true, several rules here are already measuring a habit that has moved.

The design problem, stated honestly

Every option has a bias:

  1. Generate locally. Register-matched and current, but the prompts get written by someone who knows the pattern list, so the result partly measures the prompting. Also single-family unless several providers are available.
  2. Find a current public dataset. No self-generation bias, but nothing well-licensed covers 2026 models in marketing/professional registers yet.
  3. Collect in the wild. Real LinkedIn and blog text suspected of being AI-drafted. Highest external validity, no reliable ground truth — which defeats the corpus's core property that labels come from provenance rather than judgment.
  4. Solicit contributions. Ask users to submit AI-drafted text they wrote with a model, with the model named. Real provenance, real register, slow to accumulate. The false-positive form is the template.

Option 4 plus a clearly-labelled option 1 leg is probably the honest combination: attested contributions as the primary set, local generation as a labelled secondary with its bias stated in the README.

Acceptance

Whatever lands should satisfy the public-claim gate already written into corpus/README.md: n >= 100 per claim cell, more than one register people write in today, confidence intervals, and no number quoted anywhere until it does.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions