Add us_constitution ground truth; record D-012 (heading recovery 36% vs ODL 96%)

Ground truth has 105 expected heading elements over 18 pages: a four-
level hierarchy with H1 title, H2 articles and amendments, H3 SECTION
blocks within articles, and H4 state-name signatory blocks at the end.

D-012 captures the result. ODL recovers about 96% of the heading
elements (101 / 105); PdfStruct recovers about 36% (38 / 105). The
font-rarity classifier finds the most prominent Article-level
headings and some SECTION-level headings on this Latin single-column
fixture, but misses two-thirds of the structure including the H4
state signatories. Combined with D-011 on kr_constitution (0%) and
the near-100% recovery on the simpler body content of lorem_ipsum
and minimal_document, the same classifier behaves very differently
across script and typographic-uniformity axes.

The followup direction set in D-008 and D-011 stands: heading
classification needs at least one signal beyond global font rarity.
A plos_utilizing_llm ground truth would round out the regression
matrix as the third axis before locking any design.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: diluculo

docs/odl-divergence-log.md

@@ -155,6 +155,18 @@ These are the first divergences observed after Phase 2 was committed. They were
 | judgment | **investigate**. ODL outperforms PdfStruct by a wide margin on heading recovery for this fixture, and the gap is structural: PdfStruct's font-rarity classifier finds no rarity cluster strong enough to mark anything as a heading on the 14-page document, even though the title and chapter labels carry visibly distinct weight and size. The article-as-list mistake in ODL is likely a side-effect of the same wcag-algorithms list detection that drives D-001 — articles begin with a label-like prefix `제N조` followed by space and body, which matches the list-prefix shape. With ground truth on `us_constitution` and `plos_utilizing_llm` still missing, the scope of a fix cannot be decided here yet. What this fixture establishes is that font-rarity-only heading classification does not generalise to densely-structured Korean legal layouts. |
 | followup | The heading classifier needs at least one signal beyond global font rarity for documents like this — a rarity model that operates within the document's own typographic distribution, an alternative classifier that runs as a strategy alongside the font-rarity one, or both. CLAUDE.md already calls out a `RegexHeadingClassifier` strategy slot composed via `CompositeElementClassifier`, distinct from `FontBasedElementClassifier`; future work should design how the two combine without one overriding the other on documents where they disagree. The dropped `대한민국헌법` title is the inverse of D-008's "sparse-document title miss" — the classifier appears to degenerate at both extremes (very few blocks, very many uniformly-styled blocks). The article-merged-into-first-clause failure is a separate paragraph-merger / line-grouping concern: `제N조 ①…` lays the article marker on the same line as body text, and the merger needs structural awareness to keep them apart. Re-score on `us_constitution` and a Western academic fixture before locking the design — a kr_constitution-only fix would likely regress `lorem_ipsum` or `kr_lorem_ipsum`. |
 
+### D-012 — us_constitution heading recovery: PdfStruct ~36%, ODL ~96%
+
+| field | value |
+|---|---|
+| fixture | `us_constitution.pdf` (18 pages, 105 expected heading elements) |
+| area | heading |
+| ground truth | Four-level hierarchy: H1 `Constitution of the United States`; H2 the seven `Article. <roman>.` blocks plus the amendment blocks; H3 `SECTION. N` within articles; H4 state-name signatory blocks (`NEW HAMPSHIRE`, `MASSACHUSETTS`, …). 105 heading elements total. Source: `src/PdfStruct.Tests/GroundTruth/us_constitution.md`. |
+| odl | 101 heading elements (≈96% of expected). Most articles, sections, amendments, and state blocks are recovered. ODL also extracts 2 images, surfaces 16 footers + 2 headers as first-class elements, and emits 13 lists / 24 list items (likely the enumerated powers in `Article. I. SECTION. 8` and the signature block). |
+| ours | 38 heading elements (≈36% of expected) plus 252 paragraphs. The font-rarity classifier captures prominent Article-level and some SECTION-level headings but misses two-thirds of the structure; the H4 state-name signatories are not recovered. No image elements, no header/footer separation. |
+| judgment | **investigate**. Recovery on `us_constitution` lands between PdfStruct's 0% on `kr_constitution` (D-011) and the near-100% on the simple body content of `lorem_ipsum` / `minimal_document`. Same classifier, very different results across the three legal-document fixtures — the gap to ground truth on heading detection is now the most consistent signal across the regression matrix. ODL's near-100% recovery here vs. its article-as-list mistakes on `kr_constitution` suggests its heading logic also handles Latin documents better than densely-uniform Korean ones. |
+| followup | Continues from D-008 / D-011. The 36% / 0% / ≈100% split across the three legal fixtures shows the same classifier behaves very differently on documents that differ in script and typographic uniformity. A complementary signal (document-internal rarity, structural-marker strategy via `CompositeElementClassifier`, or a hybrid) is the design space. A `plos_utilizing_llm` ground truth (two-column academic with figures, tables, references) would round out the matrix as the third axis before locking the design. |
+
 ---
 
 ## Format specification for new entries

src/PdfStruct.Tests/GroundTruth/README.md

@@ -39,7 +39,7 @@ Six fixtures cover PdfStruct's regression matrix. They are added one at a time a
 | `lorem_ipsum` | 2026-05-02 | single-column body with quoted introductory paragraphs; tests paragraph segmentation |
 | `kr_lorem_ipsum` | 2026-05-02 | Korean lorem-ipsum body; tests CJK line-joining inside a paragraph (no separator at mid-word wrap points) |
 | `kr_constitution` | 2026-05-02 | Korean legal text with `제N조` headings and `①②③` clauses; tests deep heading hierarchy (5 levels) and ordered lists inside articles |
-| `us_constitution` | — | Western legal text with Article / Amendment hierarchy |
+| `us_constitution` | 2026-05-02 | Western legal text with Article / Amendment hierarchy and a state-name signatory block |
 | `plos_utilizing_llm` | — | two-column academic with references and bullet lists |
 
 When all six exist, `compare-to-ground-truth.ps1` (not yet written) will compute element-level precision and recall for both PdfStruct and ODL against this set.