-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathMETHODS.yml
More file actions
1165 lines (1088 loc) · 54.9 KB
/
Copy pathMETHODS.yml
File metadata and controls
1165 lines (1088 loc) · 54.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
version: 1
purpose: >
Seed method and skill catalog for a retail-trader quant research agent.
Methods are used via triggers, logged in USAGE_LOG.ndjson, and scored in METHOD_SCORECARD.csv.
New methods must be added with evidence refs and lifecycle state (experimental/preferred/caution/avoid).
Expanded seed quant/statistics method catalog (hedge-fund grade).
Designed to be used and updated under the Quant Research Loop spec.
Each method must be logged in USAGE_LOG.ndjson with outcomes and scored.
method_cards:
# =========================
# A) DATA VALIDITY & PIPELINE INTEGRITY (NON-NEGOTIABLE)
# =========================
- method_id: data_window_validity_gate
name: Data Window Validity Gate
tags: [data_integrity, sanity_checks, time_series, mandatory]
problem_signals:
- "Any feature/result computed over a time window (session/rolling/first N minutes)"
- "Unexpectedly many zeros, constants, or NaNs"
- "Sparse results or suspiciously clean outputs"
how_to:
- "Compute observation counts per window (mean/p1/p50/p99) and % windows with 0 obs."
- "Compute timestamp coverage: min/max timestamps within each window; validate timezone."
- "Check degeneracy: % zeros, #unique, variance/IQR per feature."
- "Hard stop if %0-obs windows exceeds tolerance; mark experiment INVALID."
assumptions:
- "Timestamp fields are accurate and comparable after timezone normalization."
- "Window definition is meaningful for the instrument/data source."
failure_modes:
- "Missing data coerced to 0.00 or forward-filled silently."
- "Wrong session boundary or timezone causes empty windows."
- "Filtering/join errors drop rows without obvious exceptions."
alternatives:
- "None (mandatory invariant)."
canonical_keywords: ["coverage check", "bar count sanity", "timestamp range validation", "degenerate feature detection"]
status: preferred
- method_id: join_integrity_check
name: Join Integrity & Row Explosion Check
tags: [data_integrity, joins, sanity_checks]
problem_signals:
- "Any join between fact table and dimension/mapping tables"
- "Metric jumps unexpectedly after adding a join"
- "Duplicates suspected (same timestamp/instrument repeated)"
how_to:
- "Before join: record rowcount and key uniqueness stats."
- "After join: re-check rowcount and uniqueness; identify multiplicative blow-ups."
- "Sample problematic keys and inspect join cardinality."
assumptions:
- "Join keys have stable semantics (not changing across time)."
failure_modes:
- "Many-to-many join duplicates rows."
- "Key normalization (trim/upper) changes matching unexpectedly."
alternatives:
- "Pre-aggregate dimensions; enforce distinct keys; use validated mapping tables."
canonical_keywords: ["row explosion", "many-to-many join", "duplicate amplification"]
status: preferred
- method_id: missingness_semantics
name: Missingness Semantics (Missing ≠ Zero)
tags: [data_integrity, preprocessing, sanity_checks]
problem_signals:
- "Features show large masses at 0"
- "Market closed windows or sparse data feeds"
- "Any imputation/defaulting behavior present"
how_to:
- "Explicitly represent missing as NaN/None, not numeric defaults."
- "Track missingness rate per feature per window."
- "Run sensitivity: treat missing as drop vs impute vs carry-forward and compare."
assumptions:
- "Downstream models can handle missingness appropriately."
failure_modes:
- "Default 0 turns 'no data' into a fake signal."
- "Imputation leaks future info (e.g., forward-fill across gaps)."
alternatives:
- "Indicator flags for missingness + model that can use it."
canonical_keywords: ["missing not zero", "imputation leakage", "missingness indicator"]
status: preferred
- method_id: timezone_and_calendar_validation
name: Timezone & Calendar Validation
tags: [data_integrity, time_series]
problem_signals:
- "Session boundaries, opens/closes, or intraday patterns matter"
- "Data source unclear about timezone"
- "Results differ across DST transitions"
how_to:
- "Normalize all timestamps to UTC for storage; convert only for reporting."
- "Plot/summary bar counts by hour-of-day; detect dead zones."
- "Check DST boundaries explicitly (before/after)."
assumptions:
- "Instrument trading schedule can vary (cash vs futures vs FX)."
failure_modes:
- "DST shift misaligns features/labels."
- "Session definitions mismatched to data."
alternatives:
- "Empirically infer active trading hours from bar density."
canonical_keywords: ["DST", "market hours validation", "UTC normalization", "session boundary inference"]
status: preferred
# =========================
# B) LABELS, LEAKAGE & EVALUATION DISCIPLINE
# =========================
- method_id: label_definition_horizon_check
name: Label Definition & Horizon Alignment
tags: [leakage, labels, time_series, evaluation]
problem_signals:
- "Any predictive modeling or signal evaluation"
- "Forward returns, event labels, barrier-based labels"
how_to:
- "Define label: horizon, entry price proxy, exit logic, and when label becomes known."
- "Ensure features are computed strictly before the label start time."
- "Check off-by-one: shift feature timestamps and verify no performance 'miracle'."
assumptions:
- "Execution proxy chosen is consistent with decision time."
failure_modes:
- "Feature uses future bar close when decision is at open."
- "Label uses information unavailable at decision time."
alternatives:
- "Use event-driven labeling (triple barrier) with strict time cutoff."
canonical_keywords: ["forward return leakage", "horizon alignment", "label availability"]
status: preferred
- method_id: leakage_trap_tests
name: Leakage Trap Tests (Shifts & Placebos)
tags: [leakage, evaluation, sanity_checks]
problem_signals:
- "Strategy performance seems too good"
- "Complex feature pipelines"
- "Any new join or aggregation step"
how_to:
- "Placebo horizon: evaluate at an intentionally wrong horizon; should not work."
- "Shift labels forward/back; performance should degrade predictably."
- "Shuffle labels (within blocks) for a permutation sanity check."
assumptions:
- "Placebo setup preserves pipeline structure without true signal."
failure_modes:
- "Pipeline manufactures signal from leakage or target encoding."
alternatives:
- "Purged CV + embargo (for ML); strict walk-forward for simpler strategies."
canonical_keywords: ["placebo test", "label shuffle", "time shift leakage test"]
status: preferred
- method_id: baseline_discipline
name: Baseline Discipline (Weak Baselines Forbidden)
tags: [evaluation, baselines, mandatory]
problem_signals:
- "Any claim of improvement"
- "Comparisons only against trivial baselines"
how_to:
- "Always include: do-nothing, random-frequency baseline, simple heuristic baseline."
- "Include strong baseline if available (best known strategy/model)."
- "Require after-cost baselines where relevant."
assumptions:
- "Baselines reflect realistic constraints (trade frequency, holding time)."
failure_modes:
- "Beating a strawman baseline creates false confidence."
alternatives:
- "Benchmark against your own last best validated strategy."
canonical_keywords: ["strong baseline", "strawman baseline", "benchmark discipline"]
status: preferred
- method_id: walk_forward_evaluation
name: Walk-Forward / Rolling Out-of-Sample Evaluation
tags: [evaluation, time_series, robustness]
problem_signals:
- "Time-series prediction or strategy backtesting"
- "Non-stationarity suspected"
how_to:
- "Define rolling train/validate/test windows."
- "Report metrics per window and aggregated."
- "Avoid random shuffles unless strictly justified."
assumptions:
- "Past resembles future enough for learning, but not identical."
failure_modes:
- "Single split hides instability."
alternatives:
- "Purged CV + embargo for ML on overlapping labels."
canonical_keywords: ["walk-forward", "rolling OOS", "time split evaluation"]
status: preferred
# =========================
# C) COSTS, EXECUTION, AND RETAIL REALISM
# =========================
- method_id: conservative_execution_model
name: Conservative Execution Model (Retail)
tags: [execution, costs, realism]
problem_signals:
- "Any strategy test intended to be deployable"
- "Stops/limits, gaps, or session opens matter"
how_to:
- "Use bid/ask (or spread proxy) not mid for fills."
- "Add slippage proxy: function of volatility and spread (pessimistic by default)."
- "Model gaps: stop fills at next available price, not stop level."
- "Include financing where applicable (CFD/spread bet)."
assumptions:
- "You can approximate spread and slippage from historical data or broker stats."
failure_modes:
- "Mid-price fill fantasy."
- "Stops assumed perfect under gaps."
alternatives:
- "Paper trade to calibrate slippage/spread distributions."
canonical_keywords: ["bid ask fill", "slippage model", "gap risk", "stop fill realism"]
status: preferred
- method_id: cost_sensitivity_curve
name: Cost Sensitivity Curve
tags: [execution, costs, robustness]
problem_signals:
- "Edge is small"
- "Trade frequency high"
- "Broker spread varies by session/volatility"
how_to:
- "Re-run metrics across a grid of cost/slippage assumptions."
- "Plot performance vs costs; identify break-even."
- "Promote only if edge survives realistic/pessimistic ranges."
assumptions:
- "Costs can be parameterized meaningfully."
failure_modes:
- "Edge disappears under small cost increase."
alternatives:
- "Reduce turnover, add filters, improve entry/exit timing."
canonical_keywords: ["break-even costs", "slippage sensitivity", "spread sensitivity"]
status: preferred
- method_id: position_sizing_risk_caps
name: Position Sizing & Risk Caps (Retail Survivability)
tags: [risk, sizing, deployment]
problem_signals:
- "Leveraged products"
- "Strategy has tail risk"
- "Live deployment planning"
how_to:
- "Define max % capital at risk per trade/day."
- "Use vol targeting or fixed-fractional sizing."
- "Hard caps: max drawdown, max daily loss, max leverage."
- "Simulate risk-of-ruin under pessimistic scenario."
assumptions:
- "Capital and leverage constraints are known."
failure_modes:
- "Backtest looks good but leverage kills it."
alternatives:
- "Reduce size; trade fewer instruments; add circuit breakers."
canonical_keywords: ["risk of ruin", "vol targeting", "max daily loss", "circuit breaker"]
status: preferred
# =========================
# D) STATISTICS & DIAGNOSTICS (RETAIL-RELEVANT)
# =========================
- method_id: distribution_and_tail_diagnostics
name: Distribution & Tail Diagnostics
tags: [statistics, tails, risk]
problem_signals:
- "PnL dominated by few trades"
- "Large drawdowns"
- "Fat-tail markets (indices, crypto)"
how_to:
- "Report max drawdown; worst 1%/5% periods/trades."
- "Compute concentration: top N trades contribution to PnL."
- "Compare median vs mean expectancy."
- "Stress test high-volatility windows."
assumptions:
- "Sufficient sample size for tail estimates."
failure_modes:
- "Apparent edge is hidden short-vol or tail exposure."
alternatives:
- "Tail hedges, volatility filters, reduced leverage, wider stops."
canonical_keywords: ["fat tails", "expectancy", "PnL concentration", "max drawdown"]
status: preferred
- method_id: regime_bucket_analysis
name: Regime / Bucket Analysis
tags: [robustness, non_stationarity, diagnostics]
problem_signals:
- "Performance unstable across time"
- "Different behavior in high vs low vol"
how_to:
- "Bucket by volatility, trend strength, hour-of-day, day-of-week."
- "Report metrics per bucket."
- "Identify where strategy fails and why."
assumptions:
- "Bucketing variables are not leaky."
failure_modes:
- "Strategy only works in one regime (overfit)."
alternatives:
- "Add regime filter; ensemble strategies; dynamic parameterization."
canonical_keywords: ["vol buckets", "regime split", "conditioned performance"]
status: preferred
- method_id: parameter_perturbation
name: Parameter Perturbation (±20% Rule)
tags: [robustness, overfitting]
problem_signals:
- "Strategy depends on threshold/indicator parameter"
- "Sharp optimum in backtest"
how_to:
- "Perturb key thresholds ±20%."
- "Re-evaluate metrics and stability."
- "Reject strategies with knife-edge tuning."
assumptions:
- "Parameters are meaningful and not scale-dependent without normalization."
failure_modes:
- "Fragile edge from curve fitting."
alternatives:
- "Normalize features; simplify rules; use robust defaults."
canonical_keywords: ["knife-edge parameter", "curve fit detection", "sensitivity test"]
status: preferred
- method_id: multiple_testing_awareness
name: Multiple Testing Awareness (P-hacking Control)
tags: [statistics, evaluation, process]
problem_signals:
- "Many features/variants tried"
- "Frequent restarts after bad results"
how_to:
- "Pre-register plan for Validation experiments."
- "Use strict OOS holdout and avoid reusing it repeatedly."
- "Track number of variants tested; downgrade confidence accordingly."
assumptions:
- "You can maintain an experiment registry."
failure_modes:
- "One of many trials looks good by chance."
alternatives:
- "Nested validation; fresh unseen period; paper trading."
canonical_keywords: ["p-hacking", "multiple comparisons", "holdout exhaustion"]
status: preferred
# =========================
# E) STRATEGY PATTERN TOOLKIT (RETAIL-FRIENDLY STARTERS)
# =========================
- method_id: volatility_targeted_stops
name: Volatility-Targeted Stops/Targets (ATR/Range Based)
tags: [strategy_pattern, execution, risk]
problem_signals:
- "Stop-outs increase during volatility spikes"
- "Fixed stops underperform across regimes"
how_to:
- "Compute ATR or session range proxy."
- "Set stop/target distances as k * volatility."
- "Validate across vol buckets; check tail risk."
assumptions:
- "Volatility measure is computed without leakage."
failure_modes:
- "Vol proxy lags; widening stops increases tail risk."
alternatives:
- "Regime-based sizing; time stops; partial exits."
canonical_keywords: ["ATR stop", "range stop", "vol scaling"]
status: experimental
- method_id: simple_trend_filter
name: Simple Trend Filter (e.g., MA slope / breakout confirmation)
tags: [strategy_pattern, regime]
problem_signals:
- "Mean-reversion fails in trending regimes"
- "Chop vs trend separation needed"
how_to:
- "Define a simple trend measure."
- "Trade only when trend conditions meet criteria."
- "Test for confounding by volatility/time-of-day."
assumptions:
- "Trend measure isn’t just volatility proxy."
failure_modes:
- "Late entry; whipsaw in transitions."
alternatives:
- "Two-regime system; breakout-only; MR-only."
canonical_keywords: ["trend regime filter", "MA slope", "breakout confirmation"]
status: experimental
- method_id: mean_reversion_bands
name: Mean Reversion Bands (z-score / Bollinger-like)
tags: [strategy_pattern, time_series]
problem_signals:
- "Price oscillates with stable range"
- "Reversion opportunities in low trend regimes"
how_to:
- "Define rolling mean and deviation (robust if heavy-tailed)."
- "Enter at extremes; exit at mean or time stop."
- "Validate across regimes; check tail and gap risk."
assumptions:
- "Reversion behavior exists in selected market/timeframe."
failure_modes:
- "Trend regime breaks bands; large drawdowns."
alternatives:
- "Trend filter + MR; volatility filter; adaptive bands."
canonical_keywords: ["z-score reversion", "bands", "bollinger style"]
status: experimental
# =========================
# F) WEB RESEARCH / CROSS-DOMAIN INGESTION (PROCESS TOOL)
# =========================
- method_id: just_in_time_literature_search
name: Just-in-Time Literature / Technique Search
tags: [meta, research, web_search]
problem_signals:
- "Unfamiliar pattern; no method card fits"
- "Promising but fragile results need robust evaluation method"
- "Need canonical terminology to proceed"
how_to:
- "Search for canonical terms + 'time series finance' + 'robust' + failure modes."
- "Extract 2–3 candidate methods; choose simplest viable."
- "Create/Update method card with source note and minimal validation experiment."
- "Treat external knowledge as hypothesis until validated."
assumptions:
- "Sources are relevant and reputable."
failure_modes:
- "Cargo-culting methods without validation."
alternatives:
- "Rely on existing preferred methods; paper trade earlier."
canonical_keywords: ["literature search", "robust backtest method", "cross-domain technique"]
status: preferred
# =========================
# G) FOUNDATIONAL STATISTICS & INFERENCE (DEPENDENCE-AWARE)
# =========================
- method_id: eda_distribution_diagnostics
name: EDA: Distribution Diagnostics & Robust Summaries
tags: [statistics, eda, robustness]
problem_signals:
- "Any new dataset/feature/label"
- "Unexpected outliers or heavy tails"
how_to:
- "Report robust location/scale: median, IQR, MAD; compare to mean/std."
- "Check skew/kurtosis; QQ-plot conceptually; tail mass via quantiles."
- "Winsorize/sanitize for exploratory plots; never hide raw distribution in validation."
- "Check stability over time: rolling medians/IQR."
assumptions:
- "Data scale is meaningful; units consistent."
failure_modes:
- "Relying on mean/std under heavy tails misleads."
- "Outliers are data errors vs true tail events (must distinguish)."
alternatives:
- "Robust transforms (log, rank, z-score with MAD)."
canonical_keywords: ["MAD", "IQR", "robust statistics", "heavy tails", "distribution shift"]
status: preferred
- method_id: autocorrelation_dependence_checks
name: Dependence Checks: ACF/PACF, Heteroskedasticity, Serial Correlation
tags: [time_series, statistics, diagnostics]
problem_signals:
- "Time-series data (returns, volatility, spreads)"
- "Model residuals look structured"
how_to:
- "Compute/check autocorrelation (ACF) and partial autocorrelation (PACF)."
- "Test/diagnose heteroskedasticity patterns (vol clustering)."
- "For regressions: use HAC/Newey-West style reasoning for dependent errors."
assumptions:
- "Sampling frequency and timestamps are correct."
failure_modes:
- "Treating dependent observations as IID inflates significance."
alternatives:
- "Block bootstrap; time-series CV; explicit volatility models."
canonical_keywords: ["ACF", "PACF", "Newey-West", "serial correlation", "volatility clustering"]
status: preferred
- method_id: block_bootstrap_inference
name: Block Bootstrap / Stationary Bootstrap for Time-Series Inference
tags: [statistics, inference, time_series]
problem_signals:
- "Need confidence intervals under autocorrelation"
- "PnL/trade outcomes dependent"
how_to:
- "Use block bootstrap over time blocks (choose block length via dependence scale)."
- "Estimate CI for mean return, Sharpe proxy, hit rate, etc."
- "Compare to naive IID CI; if they differ materially, use bootstrap results."
assumptions:
- "Dependence is local enough for blocks to capture."
failure_modes:
- "Wrong block length gives misleading CI."
- "Structural breaks invalidate stationarity assumptions."
alternatives:
- "Subsample by regimes; Bayesian models; robust stress tests."
canonical_keywords: ["block bootstrap", "stationary bootstrap", "dependent data CI"]
status: preferred
- method_id: multiple_hypothesis_control
name: Multiple Hypothesis Control (Reality Check Discipline)
tags: [statistics, evaluation, process]
problem_signals:
- "Large feature sweeps"
- "Many strategy variants tested"
how_to:
- "Track number of trials; downgrade confidence as trials grow."
- "Use strict untouched OOS period; avoid reusing holdout."
- "Use permutation/placebo tests; consider FDR intuition for feature screens."
assumptions:
- "You can maintain an experiment registry and keep a clean holdout."
failure_modes:
- "False discovery via p-hacking."
alternatives:
- "Nested CV; fresh forward period; paper trading."
canonical_keywords: ["FDR", "multiple comparisons", "false discovery", "holdout exhaustion"]
status: preferred
# =========================
# H) TIME-SERIES FORECASTING / SIGNAL RESEARCH (HEDGE FUND BASICS)
# =========================
- method_id: return_predictability_regression
name: Predictability via Regression (with Proper Controls)
tags: [time_series, regression, signals]
problem_signals:
- "Testing if a feature predicts returns"
- "Need interpretability of signal direction/strength"
how_to:
- "Set up regression of forward returns on features (+ controls like volatility, time-of-day)."
- "Use robust standard errors / HAC logic."
- "Check stability of coefficients across time/regimes."
- "Validate out-of-sample; avoid in-sample storytelling."
assumptions:
- "Linear approximation is informative; controls reduce confounding."
failure_modes:
- "Spurious regression due to non-stationarity."
- "Coefficient stability illusion due to regime shifts."
alternatives:
- "Nonlinear models; rank-based methods; conditional tests."
canonical_keywords: ["predictive regression", "HAC errors", "coefficient stability", "spurious regression"]
status: preferred
- method_id: feature_screening_ic_rank
name: Feature Screening: Information Coefficient (IC) / Rank IC
tags: [signals, feature_engineering, cross_section, time_series]
problem_signals:
- "Many candidate features"
- "Need fast triage before modeling"
how_to:
- "Compute correlation between feature and forward return (Pearson + Spearman)."
- "Do it by bucket/time/regime; report IC mean, IC t-stat (dependence-aware)."
- "Prefer rank IC for heavy tails."
assumptions:
- "Feature and label alignment correct; no leakage."
failure_modes:
- "IC inflated by microstructure noise or confounds."
- "IC not monetizable after costs."
alternatives:
- "Simple strategy backtest with costs; mutual information with caution."
canonical_keywords: ["IC", "rank IC", "factor screening", "signal decay"]
status: preferred
- method_id: signal_decay_horizon_analysis
name: Signal Decay & Horizon Analysis
tags: [signals, time_series, diagnostics]
problem_signals:
- "Signal exists but unclear holding period"
- "Edge disappears at chosen horizon"
how_to:
- "Evaluate predictive power across multiple horizons (e.g., 1, 3, 6, 12 bars)."
- "Plot decay curve: performance vs horizon."
- "Choose horizon where net-of-cost edge maximizes risk-adjusted return."
assumptions:
- "Costs and execution scale reasonably with horizon."
failure_modes:
- "Horizon mining (multiple testing) without holdout."
alternatives:
- "Event-driven exits; adaptive horizon by regime."
canonical_keywords: ["signal decay", "holding period selection", "horizon sweep"]
status: preferred
- method_id: volatility_modeling_garch_family
name: Volatility Modeling (ARCH/GARCH intuition + regimes)
tags: [time_series, volatility, risk]
problem_signals:
- "Vol clustering impacts stops/sizing"
- "Need risk forecasts"
how_to:
- "Model volatility dynamics with rolling realized vol baseline."
- "Optionally test GARCH-type forecasts; validate out-of-sample."
- "Use volatility buckets/regimes for strategy conditioning."
assumptions:
- "Volatility is forecastable to some extent; regime persistence exists."
failure_modes:
- "Overfitting GARCH parameters; ignoring structural breaks."
alternatives:
- "EWMA vol; HAR-RV style realized volatility; simple bucket regimes."
canonical_keywords: ["GARCH", "EWMA volatility", "realized volatility", "vol regime"]
status: experimental
- method_id: change_point_detection
name: Change-Point Detection / Structural Breaks
tags: [time_series, non_stationarity, regimes]
problem_signals:
- "Model performance shifts abruptly"
- "Feature distribution drift"
how_to:
- "Use rolling metrics to detect step changes."
- "Apply change-point methods (conceptually: detect shifts in mean/variance)."
- "Re-evaluate strategy per regime; consider retrain/recalibrate."
assumptions:
- "Breaks are detectable with available sample size."
failure_modes:
- "Overreacting to noise; false break detection."
alternatives:
- "Regime bucketing by vol/trend; robust rolling windows."
canonical_keywords: ["structural break", "change point", "regime shift detection"]
status: preferred
# =========================
# I) CROSS-SECTIONAL / FACTOR QUANT (HEDGE FUND CORE)
# =========================
- method_id: cross_sectional_factor_model
name: Cross-Sectional Factor Model Basics (Ranking, Neutralization)
tags: [cross_section, factors, portfolio_construction]
problem_signals:
- "Trading multiple instruments"
- "Need to avoid unintended exposures (beta, sector, size)"
how_to:
- "Build signals as ranks/z-scores across instruments."
- "Neutralize exposures to known risk factors (market beta, sector buckets where applicable)."
- "Evaluate factor returns and stability."
assumptions:
- "Cross-sectional universe consistent; survivorship handled."
failure_modes:
- "Hidden factor bets drive returns, not the signal."
alternatives:
- "Single-instrument strategies; hedged pair trades."
canonical_keywords: ["factor neutralization", "cross-sectional ranking", "beta neutral", "sector neutral"]
status: experimental
- method_id: information_ratio_and_turnover
name: Information Ratio vs Turnover Tradeoff
tags: [portfolio_construction, costs, evaluation]
problem_signals:
- "High turnover signals"
- "Net edge sensitive to costs"
how_to:
- "Compute turnover; estimate cost drag."
- "Optimize signal smoothing/holding to maximize net IR."
- "Report IR per cost assumption."
assumptions:
- "Turnover proxy maps to costs reasonably."
failure_modes:
- "Over-smoothing kills edge; under-smoothing burns costs."
alternatives:
- "Event-based trading; lower frequency horizon."
canonical_keywords: ["information ratio", "turnover", "net alpha", "trading frictions"]
status: preferred
# =========================
# J) RISK MODELS, PORTFOLIO & BET SIZING (HEDGE FUND EXPECTATIONS)
# =========================
- method_id: risk_decomposition_exposures
name: Risk Decomposition & Exposure Attribution
tags: [risk, portfolio, diagnostics]
problem_signals:
- "Multi-asset trading"
- "Unexpected drawdowns"
how_to:
- "Decompose PnL by instrument, regime, time-of-day, direction."
- "Compute exposures to market direction and volatility proxies."
- "Identify concentration and hidden bets."
assumptions:
- "Attribution dimensions are correctly defined."
failure_modes:
- "Attribution misses true driver due to wrong bucketing."
alternatives:
- "Scenario stress tests; factor model attribution."
canonical_keywords: ["PnL attribution", "risk decomposition", "hidden exposures"]
status: preferred
- method_id: kelly_and_fractional_kelly
name: Kelly Criterion (and Fractional Kelly) as Upper Bound
tags: [risk, sizing, decision_theory]
problem_signals:
- "Need sizing guidance from edge/variance estimates"
- "Strategy has measurable expectancy"
how_to:
- "Estimate edge and variance conservatively; compute Kelly fraction."
- "Use fractional Kelly (e.g., 0.1–0.25 Kelly) due to estimation error."
- "Stress test under worse-than-estimated edge."
assumptions:
- "Edge and variance estimates are stable enough; usually they aren’t."
failure_modes:
- "Overbetting due to estimation error (common)."
alternatives:
- "Fixed fractional risk; vol targeting; drawdown-based caps."
canonical_keywords: ["Kelly sizing", "fractional Kelly", "estimation error"]
status: experimental
- method_id: scenario_stress_testing
name: Scenario & Stress Testing
tags: [risk, tails, robustness]
problem_signals:
- "Tail losses possible (gaps, opens, news)"
- "Leverage used"
how_to:
- "Re-run strategy on stress windows (high vol, crisis periods, big gaps)."
- "Apply synthetic shocks: spread widening, slippage increase, gap events."
- "Check risk of ruin and max drawdown under stress."
assumptions:
- "Stress scenarios are relevant to traded instrument."
failure_modes:
- "Stress not representative; false comfort."
alternatives:
- "Paper trade during volatile periods; reduce leverage."
canonical_keywords: ["stress test", "scenario analysis", "gap stress", "spread widening"]
status: preferred
# =========================
# K) MICROSTRUCTURE AWARENESS (EVEN FOR BAR DATA)
# =========================
- method_id: microstructure_noise_awareness
name: Microstructure Noise Awareness (Even on 1–5 min bars)
tags: [microstructure, execution, diagnostics]
problem_signals:
- "Very short horizons"
- "Signal disappears with small cost increase"
- "Performance concentrated at session opens/closes"
how_to:
- "Check sensitivity to using bid/ask vs mid proxies."
- "Evaluate signal after excluding first/last X minutes."
- "Downsample frequency and see if edge persists."
assumptions:
- "Microstructure effects are non-negligible at tested horizons."
failure_modes:
- "Signal is just spread capture illusion or open auction artifact."
alternatives:
- "Longer horizons; more conservative fills; avoid open/close windows."
canonical_keywords: ["microstructure noise", "open/close effect", "bid-ask bounce"]
status: preferred
# =========================
# L) MACHINE LEARNING (ONLY THE PARTS YOU NEED, DONE PROPERLY)
# =========================
- method_id: time_series_ml_splitting
name: Time-Series ML Splitting (Purged CV / Embargo Concept)
tags: [ml, evaluation, leakage]
problem_signals:
- "Overlapping labels (e.g., forward returns with overlap)"
- "ML models on time-series"
how_to:
- "Use time-based splits; purge overlapping periods between train/test."
- "Add embargo buffer if labels overlap."
- "Validate stability across folds/windows."
assumptions:
- "Overlap structure is known."
failure_modes:
- "Leakage via overlapping labels; inflated performance."
alternatives:
- "Simpler models; event-based labeling to reduce overlap."
canonical_keywords: ["purged CV", "embargo", "overlapping labels leakage"]
status: preferred
- method_id: calibration_and_proper_scoring
name: Calibration & Proper Scoring (When Predicting Probabilities)
tags: [ml, statistics, evaluation]
problem_signals:
- "Predicting probabilities of up/down or event occurrence"
- "Threshold-based trading from probabilities"
how_to:
- "Evaluate calibration (reliability curve concept)."
- "Use proper scoring rules (log loss, Brier)."
- "Check that better score translates to better trading outcomes after costs."
assumptions:
- "Probability outputs are meaningful; often they aren’t without calibration."
failure_modes:
- "Good classifier metrics but poor trading PnL (thresholding mismatch)."
alternatives:
- "Directly optimize trading objective; rank-based decisions."
canonical_keywords: ["calibration", "Brier score", "log loss", "reliability curve"]
status: experimental
- method_id: bar_shape_tokenization_market_language
name: Bar-Shape Tokenization (“Market Language”) for Stress/Regime Proxies
tags: [ml, feature_engineering, time_series, regimes, behavioral]
problem_signals:
- "Want NLP-style framing from price/volume only (no external news)"
- "Handcrafted features feel arbitrary; need a compact 'vocabulary' of bar shapes"
- "Need interpretable stress proxy that can be controlled for volatility/time-of-day"
how_to:
- "Define a small token alphabet for each bar using past-only bins: e.g., range bin, body location, wick asymmetry, gap-like jump bin, volume shock bin."
- "Represent a rolling window as a 'document' via token counts / n-grams / entropy / burstiness."
- "Define stress score as distribution shift vs baseline window (e.g., KL divergence) or as the predicted probability of adverse outcomes."
- "Validate incremental value OOS with controls: vol proxy (ATR/range) + UTC time-of-day buckets."
- "Monetize as gate/sizer; include retention-matched random gate baseline."
assumptions:
- "Tokenization uses fixed, past-only thresholds (or fit on IS and frozen for OOS)."
- "Sufficient sample size to estimate token distributions reliably."
failure_modes:
- "Tokens collapse to volatility proxy (range bins dominate)."
- "Overfitting token thresholds/vocabulary to IS."
alternatives:
- "Handcrafted stress features + predictive regression with controls."
- "Latent-state models (HMM) over continuous features."
canonical_keywords: ["tokenization", "candlestick tokens", "K-line", "SAX", "bag of patterns", "market language"]
sources:
- "Kronos: A Foundation Model for the Language of Financial Markets (2025, arXiv:2508.02739) https://arxiv.org/abs/2508.02739"
- "Stocks-BERT: tokenization + BERT for stock trend prediction (2025, Expert Systems with Applications) https://www.sciencedirect.com/science/article/pii/S1568494624014017"
- "CPC-SAX: chart pattern classification using SAX (2024) https://www.sciencedirect.com/science/article/pii/S2405918824000175"
status: experimental
- method_id: latent_state_stress_hmm
name: Latent Stress State Modeling (HMM / Regime Switching)
tags: [time_series, regimes, ml, risk, behavioral]
problem_signals:
- "Need a single stress probability for gating/sizing"
- "Suspect discrete regimes (normal vs stressed) with persistence"
- "Want separation between volatility regime and liquidity/stress regime"
how_to:
- "Choose emissions as past-only bar features (e.g., tails, reversals, gap-like jumps, volume shocks) plus optional vol proxy."
- "Fit small-state HMM/regime-switching model on IS; freeze parameters; infer filtered state probabilities online (past-only)."
- "Define stress score as P(stress_state) and validate OOS: does it predict forward drawdown/slippage proxies after controlling for vol + ToD?"
- "Confound check: within-vol-bucket and within-ToD-bucket, does P(stress) still separate outcomes?"
assumptions:
- "Regimes are approximately stationary within each evaluation window."
- "Model complexity kept low (2–3 states) to avoid overfit."
failure_modes:
- "Model merely recovers volatility clustering (stress==high vol)."
- "State interpretation unstable across time (label switching / drift)."
alternatives:
- "Simple bucketing rules; change-point detection; volatility regimes only."
canonical_keywords: ["HMM", "regime switching", "latent state", "stress regime", "turbulent regime"]
sources:
- "Adaptive hierarchical HMMs for bull/bear/turbulent regimes (MDPI JRF, 2026) https://www.mdpi.com/1911-8074/19/1/15"
- "Markov-switching ACI for intraday volatility/liquidity shocks (Econometrics/Finance Letters, earlier; overview) https://www.sciencedirect.com/science/article/abs/pii/S0165188921000129"
status: experimental
- method_id: tail_event_burstiness_hawkes
name: Tail-Event Burstiness (Hawkes-Style Clustering) as Stress Proxy
tags: [time_series, microstructure, regimes, risk]
problem_signals:
- "Adverse outcomes cluster (drawdowns/slippage episodes), not IID"
- "Want stress proxy driven by clustering of tail bars / gap-like events"
- "Need to model time-of-day seasonality explicitly"
how_to:
- "Define discrete stress events on bar data (e.g., tail bar exceedance, gap-like jump, failed breakout)."
- "Compute rolling event intensity as a practical proxy (count/EMA); optionally fit Hawkes-like self-exciting intensity with intraday seasonality."
- "Validate OOS: intensity predicts forward drawdown/slippage proxies within vol and ToD buckets."
- "Monetize: gate/sizer based on intensity percentile, with retention-matched random baseline."
assumptions:
- "Event definitions are stable and not tuned to OOS."
- "Seasonality is handled (ToD modulation) to avoid spurious clustering."
failure_modes:
- "Event intensity is just realized volatility in disguise."
- "Event thresholding creates lookahead via adaptive thresholds not frozen to IS."
alternatives:
- "Change-point detection; HMM over continuous features; simple volatility regimes."
canonical_keywords: ["Hawkes", "self-exciting", "event clustering", "tail events", "flash events", "liquidity stress"]
sources:
- "Compound Hawkes process for limit order book dynamics incl. intraday seasonality (2024) https://www.sciencedirect.com/science/article/pii/S1544612324011863"
- "Hawkes-driven order book dynamics with liquidity migration (2025, arXiv:2511.18117) https://arxiv.org/abs/2511.18117"
status: experimental
# =========================
# M) META: RESEARCH PROCESS / KNOWLEDGE MANAGEMENT (FROM SPEC)
# =========================
- method_id: method_registry_usage_logging
name: Method Registry + Usage Logging (Evidence-Backed Learning)
tags: [meta, process, knowledge_management, mandatory]
problem_signals:
- "Any method used in an experiment"
- "Any new technique discovered"
how_to:
- "Ensure method card exists/updated."
- "Append a USAGE_LOG record with why/how/outcome."
- "Update scorecard rollup; adjust method lifecycle state if warranted."
assumptions:
- "Logs remain append-only; scorecard generated deterministically."
failure_modes:
- "Knowledge base becomes vibes; no evidence linking to outcomes."
alternatives:
- "None (core to system learning)."
canonical_keywords: ["method evidence", "usage log", "scorecard", "playbook evolution"]
status: preferred
# =========================
# N) CROSS-DOMAIN: POKER SKILLS APPLIED TO TRADING
# =========================
- method_id: ev_thinking_expected_value
name: EV Thinking (Expected Value Over Outcome)
tags: [poker, decision_theory, process, meta]
problem_signals:
- "Judging a strategy based on a small sample of wins/losses"
- "Strong emotional response to recent outcomes"
- "Changing rules after a few bad trades"
how_to:
- "Separate decision quality from outcome: evaluate trades by expected value given information at entry."
- "Use a pre-defined edge estimate (even rough) + cost model + risk to compute expected value per trade."
- "Track EV proxy per trade alongside realized PnL; review divergence as variance, not proof."
assumptions:
- "You can approximate EV inputs (hit rate, payoff, costs) even if noisy."
failure_modes:
- "Outcome bias: quitting good strategy after a losing streak."
- "Storytelling: attributing wins to skill and losses to bad luck selectively."
alternatives:
- "Forward testing with fixed rules; block bootstrap CI for expectancy."
canonical_keywords: ["outcome bias", "expected value", "decision quality", "variance vs edge"]
status: preferred
- method_id: bankroll_management_risk_of_ruin
name: Bankroll Management (Risk of Ruin, Not Max Return)
tags: [poker, risk, sizing, deployment]
problem_signals:
- "Leveraged trading where a few losses can blow the account"
- "Increasing size after wins (heater) or after losses (chasing)"
- "High-vol strategies with fat tails"
how_to:
- "Define bankroll (capital available for strategy) separate from total net worth."
- "Set max risk per trade/day/week; enforce hard stop-loss rules."
- "Simulate risk-of-ruin under pessimistic edge/variance assumptions."
- "Use fractional Kelly only as an upper bound; default to conservative fixed-fractional sizing."
assumptions:
- "Edge and variance can be approximated conservatively."
failure_modes:
- "Overbetting due to overconfidence / estimation error."
- "Hidden tail risk causes rare but fatal loss."
alternatives:
- "Vol targeting; drawdown-based de-leveraging; circuit breakers."
canonical_keywords: ["bankroll", "risk of ruin", "fractional Kelly", "table stakes"]
status: preferred
- method_id: variance_and_sample_size_discipline
name: Variance & Sample Size Discipline (Downswings Happen)
tags: [poker, statistics, evaluation]
problem_signals:
- "Interpreting short backtests as proof"
- "Frequent strategy switching"
- "Confusing noise with signal"
how_to:
- "Estimate variance of outcomes and required sample size for confidence (rough is fine)."
- "Use block bootstrap / time-based subsampling to get realistic uncertainty."
- "Set a minimum sample threshold before concluding 'dead' or 'works'."
- "Maintain a 'confidence label' and downgrade when sample is small or trials are many."
assumptions:
- "Outcomes are variable and non-IID; uncertainty must reflect this."
failure_modes:
- "Overreacting to noise (tilting strategy selection)."
- "False discovery from repeated testing."
alternatives:
- "Paper trade; strict walk-forward; strong baselines."
canonical_keywords: ["downswing", "variance", "sample size", "confidence intervals"]
status: preferred
- method_id: game_selection_and_edge_hunting
name: Game Selection (Choose Easier Games / Higher Edge Environments)
tags: [poker, regimes, strategy_selection]
problem_signals:
- "Strategy only works in certain conditions"
- "Costs/spreads widen in some sessions"
- "Low-liquidity periods cause slippage"
how_to:
- "Treat markets/regimes like poker tables: some are tougher (efficient) and some softer (inefficient)."
- "Define 'playability filters' (volatility, spread, liquidity proxy, session window)."
- "Trade only when conditions match where edge historically exists."
- "Validate filters aren’t just cherry-picking (use OOS and confounding checks)."
assumptions:
- "Edge is regime-dependent; you can identify regimes with non-leaky proxies."
failure_modes:
- "Filter overfits past conditions; disappears live."
- "Filter is proxy for lookahead or data artifacts."
alternatives:
- "Broader strategy with robust risk controls; longer horizons."
canonical_keywords: ["game selection", "table selection", "regime filter", "playability"]
status: preferred
- method_id: exploitative_vs_gto_tradeoff
name: Exploitative vs Balanced Strategy (GTO Analogy)
tags: [poker, strategy_design, robustness]
problem_signals:
- "Considering aggressive edge capture that may not generalize"
- "Strategy depends on specific market behavior"
- "Live performance differs from backtest"
how_to:
- "Balanced strategy (GTO-like): robust across opponents/regimes, lower peak edge."
- "Exploitative strategy: higher edge in a specific regime but can break when regime shifts."
- "Decide explicitly which you are building; apply stricter robustness for exploitative strategies."
- "If exploitative: define regime detector + kill-switch + monitoring thresholds."
assumptions:
- "Market changes; opponents adapt; regimes shift."
failure_modes:
- "Over-optimization to historical regime; collapses live."
alternatives:
- "Ensemble of smaller edges; diversified strategies; slower horizons."
canonical_keywords: ["GTO vs exploit", "robustness", "regime shift", "adaptation"]
status: preferred
- method_id: range_thinking_and_scenario_planning
name: Range Thinking (Scenario Sets, Not Point Forecasts)