P0: Data Integrity - Automated Stat Verification System

[kevsilk597]

Objective

Every number published in a card must be accurate and traceable to our database. We cannot have cards making claims that are wrong or unverifiable.

Current example:

"76ers are 0-3 at home when favored teams roll in hot"

Is this true? Can we prove it? If a customer asks "where did that stat come from?", we need an answer in 30 seconds.

Requirements

1. Stat Claim Extraction

Build a function that extracts all statistical claims from a card:

def extract_stat_claims(card: dict) -> list[dict]:
    """
    Extract all verifiable claims from card text.
    
    Returns:
        [
            {
                "claim": "76ers are 0-3 at home when favored teams roll in hot",
                "stat_type": "team_record",
                "entity": "PHI",
                "value": "0-3",
                "conditions": ["home", "opponent_favored", "opponent_coming_off_wins"],
                "source_field": "why_it_matters[1]"
            },
            {
                "claim": "averaging 30.9 PPG last 7 games",
                "stat_type": "player_average",
                "entity": "Bam Adebayo",
                "value": 30.9,
                "window": 7,
                "stat": "points",
                "source_field": "heroStat"
            }
        ]
    """

Use regex patterns to find:

• Record claims: "X-Y record", "0-3 at home", "won 5 straight"
• Average claims: "averaging X PPG/RPG/APG", "X points per game"
• Ranking claims: "ranks Nth in", "top 10 in"
• Comparison claims: "up X% from", "above/below average"
• Streak claims: "last N games", "since [date]"

2. Claim Verification Engine

For each extracted claim, build a verifier that queries the database:

def verify_claim(claim: dict) -> dict:
    """
    Verify a stat claim against the database.
    
    Returns:
        {
            "claim": "76ers are 0-3 at home when favored teams roll in hot",
            "verified": True|False,
            "actual_value": "0-3",  # What the DB says
            "claimed_value": "0-3", # What the card says
            "match": True,
            "query": "SELECT ... FROM games WHERE ...",
            "query_result": [...],
            "verified_at": "2026-03-24T12:00:00Z"
        }
    """

3. Verification Query Library

Build reusable query templates for common claim types:

VERIFICATION_QUERIES = {
    "team_home_record": """
        SELECT 
            SUM(CASE WHEN home_score > away_score THEN 1 ELSE 0 END) as wins,
            SUM(CASE WHEN home_score < away_score THEN 1 ELSE 0 END) as losses
        FROM games
        WHERE home_team = :team
        AND game_date BETWEEN :start_date AND :end_date
        AND home_score IS NOT NULL
    """,
    
    "player_ppg_last_n": """
        SELECT AVG(pts) as ppg
        FROM player_stats
        WHERE player_name = :player
        ORDER BY game_date DESC
        LIMIT :n
    """,
    
    "team_record_vs_condition": """
        -- Dynamic based on condition type
    """
}

4. Pre-Storage Verification

Before any card is stored, verify all its claims:

def store_card(card: dict) -> bool:
    claims = extract_stat_claims(card)
    
    verification_results = []
    for claim in claims:
        result = verify_claim(claim)
        verification_results.append(result)
        
        if not result["verified"]:
            logger.error(f"[VERIFICATION FAILED] Card {card['id']}: {claim['claim']}")
            logger.error(f"  Claimed: {result['claimed_value']}, Actual: {result['actual_value']}")
            return False  # Do not store card with false claims
    
    # Embed verification proof in card
    card["verification"]["claims"] = verification_results
    card["verification"]["all_claims_verified"] = True
    
    # Store card
    ...

5. Source Attribution

Every card must include traceable source information:

card["sources_used"] = {
    "tables_queried": ["games", "player_stats", "injuries"],
    "date_range": {"from": "2026-01-01", "to": "2026-03-24"},
    "row_counts": {"games": 45, "player_stats": 234},
    "queries": [
        {
            "purpose": "team home record",
            "query_hash": "abc123",  # For audit trail
            "executed_at": "2026-03-24T12:00:00Z"
        }
    ]
}

6. Verification Endpoint

Create an admin endpoint to verify any card on demand:

GET /api/v1/admin/verify-card?id=trap_game-1690

Response:
{
    "card_id": "trap_game-1690",
    "claims_found": 3,
    "claims_verified": 3,
    "claims_failed": 0,
    "verification_details": [
        {
            "claim": "OKC is 4-point favorite",
            "verified": true,
            "source": "odds table",
            "query": "SELECT spread FROM odds WHERE game_id = ..."
        },
        ...
    ]
}

7. Batch Verification Script

Create a script to verify all existing cards:

python -m gtd.verify_all_cards --date 2026-03-24

Output:
Verifying 6 cards...
  trap_game-1690: 3/3 claims verified ✓
  rivalry-1650: 2/2 claims verified ✓
  ...

Summary:
  Total cards: 6
  Fully verified: 6
  Partial failures: 0
  Complete failures: 0

Run this daily as part of monitoring.

Validation

1. Generate a new card
2. Run verification on it:

curl "http://cypher.178.156.223.137.nip.io/api/v1/admin/verify-card?id=NEW_CARD_ID"

3. Manually spot-check 2-3 claims against the database
4. Confirm verification results match manual check

Definition of Done

• extract_stat_claims() function implemented
• verify_claim() function implemented for top 5 claim types
• Pre-storage verification blocks cards with false claims
• Source attribution embedded in every card
• /api/v1/admin/verify-card endpoint working
• Batch verification script runs on all existing cards
• 100% of current cards pass verification
• Documentation: how to trace any stat claim to its source