Visa Predictive Fraud Intelligence

A production-grade transaction fraud detection system that identifies fraudulent payments in real time. Built to demonstrate the full lifecycle — from raw transaction ingestion through model training, evaluation, and deployment — aligned with how Visa's fraud operations teams actually work.

---

Executive Summary

Metric	Value
Detection Accuracy (AUC-ROC)	0.998
Fraud Caught	143 of 157 test cases (91.1%)
False Positives	6 out of 9,843 legitimate transactions (0.06%)
Net Dollar Savings	$164,274 on test set
Scoring Latency	<100 ms per transaction
Features Analyzed	36 per transaction

The system catches over 91% of fraudulent transactions while incorrectly flagging fewer than 1 in 1,600 legitimate ones. On the test set alone, that translates to $164K in prevented fraud losses after accounting for false positive investigation costs.

---

How It Works

Raw Transactions (13 fields per transaction)
        |
   Feature Engineering (36 derived signals)
        |
   +----------+-----------+
   |          |           |
 XGBoost   LightGBM   LSTM / Transformer
   |          |           |
   +----------+-----------+
        |
   Stacking Ensemble (cross-validated meta-learner)
        |
   Cost-Optimal Threshold Selection
        |
   Decision: APPROVE / REVIEW / FLAG / BLOCK
        |
   Monitoring: drift detection + performance tracking

Step 1 — Feature Engineering: Each transaction is enriched with 36 features covering spending velocity (how fast is this cardholder spending?), statistical profiles (is this amount unusual for them?), behavioral flags (new merchant? cross-border?), and temporal patterns (late night? weekend?).

Step 2 — Ensemble Scoring: Two gradient-boosted tree models (XGBoost and LightGBM) each score the transaction independently. A logistic regression meta-learner combines their outputs using 5-fold cross-validated predictions to avoid overfitting.

Step 3 — Decision: The fraud probability is compared against a cost-optimal threshold that minimizes total business cost (fraud losses + investigation costs). Transactions are classified as LOW / MEDIUM / HIGH / CRITICAL risk.

Step 4 — Monitoring: Feature distributions are tracked over time using Population Stability Index (PSI). If the data shifts significantly, the system flags it for retraining.

---

Model Performance

Trained on 50,000 synthetic transactions with a 1.5% fraud rate (750 fraudulent). Time-based split to prevent data leakage.

Model	AUC-ROC	F1 Score	Precision	Recall	Fraud Caught	False Positives
XGBoost (Optuna-tuned)	0.9983	0.926	0.986	0.873	137/157	2
LightGBM	0.9985	0.940	0.979	0.904	142/157	3
Stacking Ensemble	0.9984	0.935	0.960	0.911	143/157	6
LSTM	0.800	0.380	0.294	0.536	30/56	72
Transformer	0.753	0.366	0.459	0.304	17/56	20

A note on these numbers: These metrics come from synthetic data with known fraud patterns. In production, expect AUC-ROC of 0.95–0.97 due to label noise, concept drift, and adversarial behavior. The benchmark validates the pipeline architecture and feature engineering — not production accuracy.

Why tree-based models outperform deep learning here

Gradient-boosted trees (XGBoost, LightGBM) consistently beat neural networks on structured tabular data because they handle heterogeneous features, irregular distributions, and small-to-medium datasets more naturally. The deep learning models are included to demonstrate sequence modeling architecture but require substantially more data and training time. See Grinsztajn et al. (NeurIPS 2022).

---

Business Impact Analysis

The system evaluates each model not just on statistical metrics but on dollar impact:

Model	Fraud Dollars Saved	False Positive Cost	Net Savings
XGBoost	$166,214	$2,375	$163,839
LightGBM	$164,990	$575	$164,415
Ensemble	$164,674	$400	$164,274

Cost-optimal thresholds are selected by sweeping 500 candidates and minimizing total cost (missed fraud + $25/false positive investigation cost).

---

Feature Engineering

36 features computed per transaction, organized into six categories:

Category	Features	What it captures
Velocity	8	Transaction count and spend over 1h, 6h, 24h, 7d windows
Statistical	4	Per-cardholder and per-merchant-category z-scores, amount rank
Behavioral	4	New merchant flag, amount deviation, cross-border, high-value
Temporal	4	Hour sin/cos encoding, weekend indicator, night indicator
Frequency	4	Encoded frequency of merchant category, entry mode, card type, country
Interactions	12	Amount x risk, velocity x night, entry mode x amount, and more

All features use a fit/transform pattern to prevent data leakage between train and test sets.

---

Live Demo

An interactive demo is available for scoring transactions in real time:

# Run locally
python app.py
# Opens at http://localhost:7860

The demo includes:

• Live scoring with 12 predefined scenarios (from routine groceries to suspected account takeover)
• Custom transaction builder to test any combination of inputs
• Model comparison dashboard
• Visualization gallery (9 pipeline-generated plots)
• Architecture overview and model documentation

Also deployed on HuggingFace Spaces.

---

Tech Stack

Component	Technology
Language	Python 3.10+
Data Processing	PySpark 3.5, Pandas, NumPy
ML Models	scikit-learn, XGBoost, LightGBM, Optuna
Deep Learning	PyTorch
Experiment Tracking	MLflow
Monitoring	Custom PSI drift detection
Serving	FastAPI REST API, Gradio demo, A/B testing framework
Fairness	Bias auditing, per-group threshold calibration
Graph Analysis	NetworkX cardholder-merchant transaction graphs
Feature Store	Pluggable backend abstraction (InMemory, Redis)
Testing	pytest (99 tests — unit + integration)
Infrastructure	Docker, Makefile, GitHub Actions

---

Quick Start

git clone https://github.com/Dheeraj-Bhaskaruni/visa-fraud-detection.git
cd visa-fraud-detection

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run the full pipeline (generates data, trains models, evaluates, monitors)
python scripts/run_end_to_end.py

# Run tests
pytest tests/ -v

# Launch the demo
python app.py

---

Project Structure

visa-fraud-detection/
├── app.py                    # Gradio demo application
├── configs/                  # Model and pipeline configurations
├── data/
│   ├── raw/                  # Transaction logs
│   ├── processed/            # Engineered features + fitted transformers
│   └── external/             # Watchlists, merchant codes
├── models/                   # Trained model artifacts
├── notebooks/                # EDA and modeling walkthrough
├── outputs/
│   ├── plots/                # 9 analysis plots
│   └── reports/              # Evaluation reports (JSON, CSV)
├── scripts/
│   ├── run_end_to_end.py     # Full 8-step pipeline
│   ├── train_dl_models.py    # Standalone deep learning training
│   └── generate_synthetic_data.py
├── src/
│   ├── api/                  # FastAPI REST API (score, batch, explain)
│   ├── features/             # Feature engineering (36 features), feature store, graph features
│   ├── models/               # XGBoost, LightGBM, Ensemble, LSTM, Transformer
│   ├── serving/              # Inference service, A/B champion-challenger testing
│   ├── agents/               # Automated investigation framework
│   ├── pipeline/             # Spark ETL + Airflow DAG
│   ├── monitoring/           # Drift detection, performance tracking, fairness auditing
│   └── visualization/        # Plotting utilities
├── tests/                    # Unit + integration tests (99 total)
├── MODEL_CARD.md             # Model documentation
├── Dockerfile
├── docker-compose.yml
├── Makefile
└── requirements.txt

---

Visualizations

The pipeline generates 9 plots covering the full analysis:

Plot	Description
EDA Overview	Transaction distributions, fraud rates by hour/entry mode/merchant
Feature Correlations	36-feature correlation heatmap
ROC & PR Curves	Discrimination performance across all models
DL Training Curves	LSTM and Transformer loss/AUC per epoch
Confusion Matrices	Classification outcomes at optimal thresholds
Feature Importance	Top 20 features by contribution
Score Distributions	Fraud vs. legitimate probability distributions
Monitoring Dashboard	PSI drift values, performance windows, score stability
Business Impact	Cost curves and dollar-impact comparison

---

Limitations

This is a portfolio project built on synthetic data. Key limitations for production use:

• Synthetic data: Fraud patterns are hand-crafted; real fraud is adversarial and constantly evolving
• Limited fraud types: 4 patterns vs. hundreds in production (friendly fraud, refund abuse, synthetic identity, etc.)
• No concept drift: Training and test data come from the same synthetic distribution
• No adversarial testing: Model not evaluated against evasion attacks
• Batch only: No streaming/real-time inference pipeline See MODEL_CARD.md for full model documentation including ethical considerations.

---

Enterprise Components

REST API (src/api/)

Production-ready FastAPI service for real-time scoring. Endpoints for single transaction scoring, batch scoring (up to 1,000), prediction explanations, and model info. Includes API key authentication, latency headers, health/readiness probes, and Pydantic request validation.

uvicorn src.api.app:app --reload
# POST /v1/score, POST /v1/score/batch, POST /v1/explain, GET /v1/model/info, GET /health

Feature Store (src/features/feature_store.py)

Pluggable feature store abstraction with InMemory and Redis backends. Supports online serving (low-latency latest features), offline training (point-in-time lookups to prevent leakage), feature group registry, freshness tracking, and feature statistics. Wraps the existing FeatureEngineer for materialize-once, serve-many workflows.

Graph/Network Features (src/features/graph_features.py)

Extracts 10 graph-derived features from the cardholder-merchant bipartite transaction network: shared merchants, merchant fraud rate (Bayesian-smoothed), card fan-out, merchant fan-in, shared card score, cross-merchant velocity, entry mode diversity, country diversity, PageRank, and community fraud rate. Same fit/transform pattern as the core feature engineer.

Fairness & Bias Auditing (src/monitoring/fairness.py)

Audits model predictions for demographic bias across protected attributes. Computes demographic parity, equalized odds, equal opportunity, false positive rate ratios, and predictive parity. Checks compliance against the 4/5ths rule. Generates reports with severity-graded violations and actionable recommendations. Includes per-group threshold calibration to equalize false positive rates.

A/B Champion-Challenger (src/serving/ab_testing.py)

Framework for comparing production models side-by-side. Deterministic hash-based traffic routing ensures the same transaction always goes to the same variant. Tracks outcomes, computes per-variant metrics (AUC-ROC, precision, recall, F1), and runs bootstrap significance testing with configurable guardrail thresholds. Automated promotion decisions with guardrail violation checks.