P0: Enforce Quality Gate - Disable Shadow Mode

[kevsilk597]

Problem

Every card generated today has:

"verification": {
    "score": 60,
    "checks_total": 0,
    "checks_failed": 0,
    "mode": "shadow"
}
"qualityGatePassed": true

Cards with verification score 60 are passing the quality gate. The quality gate is running in "shadow" mode which means it logs failures but does not block bad cards.

This defeats the entire purpose of having a quality gate.

Root Cause

From the hardening plan review (Issue #15):

_validate_card_v3 runs in soft mode by default. When a card fails validation, the errors are logged but the card is marked as passed anyway.

Requirements

1. Disable Shadow Mode

Find where shadow/soft mode is set and change to enforcement mode:

# BEFORE (shadow mode - logs but passes)
validation_mode = "shadow"

# AFTER (enforce mode - blocks bad cards)
validation_mode = "enforce"

2. Set Minimum Verification Score

MINIMUM_VERIFICATION_SCORE = 75  # Cards below this NEVER pass

def validate_card(card: dict) -> bool:
    score = calculate_verification_score(card)
    
    if score < MINIMUM_VERIFICATION_SCORE:
        log(f"[QUALITY_GATE] BLOCKED card {card['id']}: score {score} < {MINIMUM_VERIFICATION_SCORE}")
        return False
    
    # Run other validation checks...
    checks = run_validation_checks(card)
    if checks["failed"] > 0:
        log(f"[QUALITY_GATE] BLOCKED card {card['id']}: {checks['failed']} checks failed")
        return False
    
    return True

3. Actually Run Validation Checks

Current cards show checks_total: 0. The validator is not running any checks.

Ensure these checks actually execute:

• Stat accuracy check (do the numbers match the database?)
• Text completeness check (no truncated sentences?)
• Source citation check (are sources listed?)
• Entity extraction check (are teams/players identified?)

4. Block Cards That Fail

# In card generation pipeline
for card in generated_cards:
    if not validate_card(card):
        continue  # Do not store or serve this card
    
    store_card(card)

5. Add Quality Gate Metrics to Health Endpoint

{
    "quality_gate": {
        "mode": "enforce",
        "cards_generated_today": 100,
        "cards_passed": 45,
        "cards_blocked": 55,
        "block_rate": 0.55,
        "average_score_passed": 82,
        "average_score_blocked": 58
    }
}

Validation

After deploy:

• Generate cards
• Verify cards with score <75 are NOT in the feed
• Verify cards with truncated sentences are NOT in the feed
• Verify mode is "enforce" not "shadow"
• Verify checks_total > 0 for all cards

Definition of Done

• Shadow/soft mode disabled
• Minimum verification score enforced (75)
• Validation checks actually run (checks_total > 0)
• Cards that fail validation are not stored/served
• Health endpoint shows quality gate metrics
• No card with score <75 appears in production feed