perf/memory smoke testing for summary stats: varied datasets, CI reports, checks for runtime-added stats

[paddymul]

Checks

• I have checked that this enhancement has not already been requested

Subsystem: Summary stats; Pluggable analysis framework; CI; Developer ergonomics

Problem

A summary stat that is correct but pathological — a per-row python loop, an O(n²) algorithm, a large intermediate allocation — passes every existing check. StatPipeline.unit_test (stat_pipeline.py:335) runs the pipeline against the 10-row PERVERSE_DF (utils.py:11) and only reports raised errors, so the failure surfaces later as a hung widget or an OOM on real data. The authors most likely to write such a stat have the least feedback: user- and LLM/MCP-authored stats added at runtime through add_analysis or a project stat dir.

#917 adds the core harness: perf_smoke_test(stat_funcs) wraps each StatFunc with wall-time and tracemalloc peak-memory instrumentation, runs the pipeline over a seeded synthetic frame at a small scale then full scale, and reports findings against machine-relative limits (ratios vs canonical pandas ops timed on the same frame, with absolute floors). A flagged stat is never run again, so a quadratic stat can't hang the smoke run itself.

Remaining work

1. Dataset variety. make_smoke_df covers int, float-with-nans, low-cardinality string, and all-distinct string. Stats blow up on shapes we already have bug history for: unhashable object cells (perf: DefaultSummaryStats O(n²) on object columns with unhashable values (lists/dicts/sets) #843), int64 beyond 2^53 (Xorq stats fail silently on int64 values > 2^53 (JSON-safe-integer limit) #800, Histogram computation crashes on large integers near 2^53 (float64 precision loss) #632), Decimal columns (Polars histogram stat fails silently on Decimal columns (DDD weird_types) #801), tz-aware datetimes (Failing on time zone information #277), all-null columns, categorical dtype. Either widen the frame or run the harness over a small suite of named frames so a finding says which shape triggered it.

2. CI report. Run the harness over the shipped stat lists (PD_ANALYSIS_V2 at minimum) in CI and emit a per-stat time / peak-memory table to the job summary. Assert only on the loose smoke limits so the job fails on order-of-magnitude blowups; the table is for spotting drift between runs.

3. Runtime-added stats. Call the smoke test when stats arrive at runtime — StatPipeline.add_stat (stat_pipeline.py:345), DfStats.add_analysis (analysis_management.py:316), project stat dirs loaded by load_project_stat_klasses (server/xorq_loading.py:137), an MCP add-stat tool. Warn by default rather than reject; the finding messages are already written to be actionable for an LLM author.

Context

Identified while triaging #903, which covers a different class of the same authoring problem (non-deterministic xorq stat expressions). #127 was a coarse widget-level version of this idea (2023). The harness lands in #917; this issue tracks the follow-ups. Pandas pipeline only for now — tracemalloc can't see polars/Arrow-native allocations.