eCFR Metrics App is a local-first full-stack project that ingests public regulatory data, normalizes it into SQLite, and turns it into agency-level burden and change metrics. It highlights backend/data engineering judgment as much as UI work by pairing the app with audits, repair scripts, reproducible local data, and reliability hardening.
This project pulls agency mappings and dated XML snapshots from eCFR, normalizes them into a small relational model, and computes transparent text-based metrics for agency-level analysis. The backend handles flattening a nested source directory, deduplicating CFR references, planning whole-title vs part-level fetches, parsing XML into scoped text blocks, and persisting both current and historical metric layers. The frontend consumes those metrics through a small FastAPI API and exposes them as a ranked overview, per-agency drilldown, and methodology page. The repository also includes smoke checks, anomaly audits, and a repair path for rebuilding current metrics from stored history. The strongest technical signal is the combination of public-data ingestion, normalization, auditability, and fail-closed behavior around unreliable historical fetches.
- Built a local-first ingestion pipeline for public eCFR JSON/XML with retry logic, raw payload caching, and bounded historical snapshots.
- Normalized a nested agency directory into a relational SQLite model with explicit reference deduplication and separate current/history tables.
- Parsed XML into scoped text blocks and matched them against normalized CFR references to compute agency-level metrics.
- Added transparent heuristic metrics for restrictive language, process burden, recent change activity, and content fingerprints.
- Hardened the system with smoke checks, dataset audits, drift-repair scripts, and tests that prove fail-closed behavior on partial history.
- Prevented duplicate CFR mappings from distorting fetch planning and downstream metrics.
- Refused to publish partial historical metrics when required dated XML fetches failed.
- Made the seeded local experience reproducible even when live upstream history endpoints were unreliable.
Local-first eCFR analytics dashboard for agency-level burden, change volatility, and audit-ready text metrics.
Built a full-stack regulatory analytics app on top of public eCFR data using Python, FastAPI, React, and SQLite. The project covers ingestion, normalization, agency mapping, text metrics, audit tooling, and reliability hardening, with a local-first workflow that makes the dataset and scoring logic easy to inspect.