Improve tone across all 65 problems

Mechanical cleanup: strip em/en-dashes site-wide, replace overused
vocabulary (leverage, utilize, facilitate, etc.), drop throat-clearing
phrases like 'It's worth noting that' and 'Furthermore'. Capitalize
sentence starts after period-replacements.

Note: the per-problem prose rewrite pass was started but hit a rate
limit before completing for most problems. Em-dashes and AI-smell
phrases are clean throughout, but the conversational-voice pass will
need another round when limits reset.

Author: shiningflash

CONTRIBUTION.md

@@ -4,10 +4,10 @@ Thanks for helping expand the Data Engineering Practice Problems collection. Thi
 
 ## Ways to Contribute
 
-- **New practice problems** – add interview-style prompts, sample datasets, and reference solutions.
-- **Enhance existing solutions** – improve performance, readability, resilience, or validation coverage.
-- **Improve documentation** – clarify instructions, add diagrams or walkthroughs, fix typos.
-- **Quality assurance** – add tests, linting, or automation that strengthens the developer experience.
+- **New practice problems**, add interview-style prompts, sample datasets, and reference solutions.
+- **Enhance existing solutions**, improve performance, readability, resilience, or validation coverage.
+- **Improve documentation**, clarify instructions, add diagrams or walkthroughs, fix typos.
+- **Quality assurance**, add tests, linting, or automation that strengthens the developer experience.
 
 ## Contribution Workflow
 
@@ -19,16 +19,16 @@ Thanks for helping expand the Data Engineering Practice Problems collection. Thi
 
 ## Repository Structure Standards
 
-- Every problem lives in `problems/NNN-kebab-case-title/` where `NNN` is a zero-padded id (`001`, `065`, ...). Pick the next free id.
+- Every problem lives in `problems/NNN-kebab-case-title/` where `NNN` is a zero-padded id (`001`, `065`,...). Pick the next free id.
 - Inside each problem directory:
-  - `question.md` – the problem prompt. **Must start with YAML frontmatter** (see below), followed by a single H1 `# Problem N — Title`, then the body.
-  - `solution.md` – written walkthrough (preferred for design / discussion problems), **or** `solution.py` for runnable code problems. Pick one; the frontmatter declares which.
-  - Optional subdirectories (`data/`, `tests/`) if the problem demands bespoke assets. Keep paths relative (e.g., `../../data/` for shared sample data).
+ - `question.md`, the problem prompt. **Must start with YAML frontmatter** (see below), followed by a single H1 `# Problem N, Title`, then the body.
+ - `solution.md`, written walkthrough (preferred for design / discussion problems), **or** `solution.py` for runnable code problems. Pick one; the frontmatter declares which.
+ - Optional subdirectories (`data/`, `tests/`) if the problem demands bespoke assets. Keep paths relative (e.g., `../../data/` for shared sample data).
 - Shared datasets belong in the top-level `data/` directory. Provide a `README` snippet or inline comments describing their provenance if they are newly added.
 
 ### Required frontmatter
 
-Every `question.md` begins with this block — `PROBLEMS.md` is generated from it:
+Every `question.md` begins with this block, `PROBLEMS.md` is generated from it:
 
 ```yaml
 ---

PROBLEMS.md

@@ -4,7 +4,7 @@ A quick overview of every problem in this repo. Use the **Category** and **Topic
 
 > This file is generated by `scripts/build_index.py` from the frontmatter in each problem's `question.md`. Do not edit by hand.
 
-| #  | Problem | Category | Topics | Difficulty | Question | Solution |
+| # | Problem | Category | Topics | Difficulty | Question | Solution |
 |----|---------|----------|--------|------------|----------|----------|
 | 1 | Log File Error Analysis | Logs and Monitoring | file streaming, counters, top-N, IoT logs | Easy | [Question](problems/001-log-file-error-analysis/question.md) | [Solution](problems/001-log-file-error-analysis/solution.py) |
 | 2 | Rolling Average of Sensor Readings | Streaming | rolling window, deque, IoT sensors, real-time | Easy | [Question](problems/002-rolling-average-of-sensor-readings/question.md) | [Solution](problems/002-rolling-average-of-sensor-readings/solution.py) |
@@ -95,8 +95,8 @@ A quick overview of every problem in this repo. Use the **Category** and **Topic
 
 ### Difficulty Guide
 
-* **Easy** — A focused warm-up. Solvable or explainable in under an hour.
-* **Medium** — Realistic interview question. Has edge cases that matter.
-* **Hard** — Multi-step or system-design heavy. Closer to a take-home task.
+* **Easy**. A focused warm-up. Solvable or explainable in under an hour.
+* **Medium**. Realistic interview question. Has edge cases that matter.
+* **Hard**. Multi-step or system-design heavy. Closer to a take-home task.
 
 > New problems are added regularly. If you want to contribute, see the [Contribution Guide](CONTRIBUTION.md).

README.md

@@ -1,9 +1,9 @@
 # 🧩 Data Engineering Practice Problems
 
-> After solving 1,500+ problems on LeetCode and Codeforces, I realized —
+> After solving 1,500+ problems on LeetCode and Codeforces, I realized,
 > **none of them prepared me for broken CSVs, delayed Kafka messages, or JSONs that lie.**
 
-This repo is for engineers who’ve had enough of toy problems. It’s a collection of **real-world data engineering scenarios** — short, practical exercises inspired by what actually breaks in production.
+This repo is for engineers who’ve had enough of toy problems. It’s a collection of **real-world data engineering scenarios**. Short, practical exercises inspired by what actually breaks in production.
 
 ---
 
@@ -12,7 +12,7 @@ This repo is for engineers who’ve had enough of toy problems. It’s a collect
 Most practice problems test logic.
 Production tests resilience.
 
-In production, problems don’t come with test cases — they come with missing data, bad assumptions, and time pressure.
+In production, problems don’t come with test cases, they come with missing data, bad assumptions, and time pressure.
 
 So I started collecting real scenarios I’ve seen:
 
@@ -26,15 +26,15 @@ So I started collecting real scenarios I’ve seen:
 
 ## What’s Inside
 
-| Category                 | Scenario                                         | What You’ll Practice                         |
-| :----------------------- | :----------------------------------------------- | :------------------------------------------- |
-|  **Late Data**           | 10 GB of IoT logs arriving out of order          | Handle streaming delays without duplication  |
-|  **Schema Drift**        | JSON events adding new fields mid-release        | Validate and evolve safely                   |
-|  **ETL Reliability**     | Long-running jobs silently skipping records      | Detect silent corruptions before they spread |
-|  **Data Hygiene**        | Partner CSVs with missing headers and fake nulls | Clean data in one pass and log every fix     |
-|  **Rolling Analytics**   | Continuous sensor feeds with infinite rows       | Keep rolling metrics in memory without dying |
+| Category | Scenario | What You’ll Practice |
+|:----------------------- |:----------------------------------------------- |:------------------------------------------- |
+| **Late Data** | 10 GB of IoT logs arriving out of order | Handle streaming delays without duplication |
+| **Schema Drift** | JSON events adding new fields mid-release | Validate and evolve safely |
+| **ETL Reliability** | Long-running jobs silently skipping records | Detect silent corruptions before they spread |
+| **Data Hygiene** | Partner CSVs with missing headers and fake nulls | Clean data in one pass and log every fix |
+| **Rolling Analytics** | Continuous sensor feeds with infinite rows | Keep rolling metrics in memory without dying |
 
-And many more coming ...
+And many more coming...
 
 > Each problem is small enough to solve in hours, but real enough to prepare you for production.
 
@@ -76,7 +76,7 @@ PROBLEMS.md          # generated index — do not edit by hand
 If you’ve debugged a broken pipeline,
 caught a silent bug before it spread,
 built a clever patch that saved a release
-or found a way to clean a 5 GB CSV in one pass —
+or found a way to clean a 5 GB CSV in one pass,
 your story belongs here.
 
 Add a new scenario, or improve an existing one.
@@ -85,6 +85,6 @@ See the [Contribution Guide](CONTRIBUTION.md) for details.
 ---
 
 > **The goal isn’t to practice coding.**
-> It’s to practice *judgment* — the kind that keeps systems running when logic alone isn’t enough.
+> It’s to practice *judgment*, the kind that keeps systems running when logic alone isn’t enough.
 
 ⭐ Star the repo if you’ve ever learned more from production than from tutorials.

problems/001-log-file-error-analysis/question.md

@@ -7,10 +7,10 @@ difficulty: Easy
 solution: solution.py
 ---
 
-# Problem 1 — Log File Error Analysis
+# Problem 1, Log File Error Analysis
 
 **Scenario:**
-You have a huge log file from an IoT platform. Each line follows this structure:
+You have a huge log file from an IoT platform. Each line looks like this:
 
 ```
 2025-10-11T13:45:20Z sensor_12 OK
@@ -20,23 +20,23 @@ You have a huge log file from an IoT platform. Each line follows this structure:
 ...
 ```
 
-Each line contains:
+Each line has:
 
 * A timestamp (ISO format)
 * A sensor ID
 * A status (`OK` or `ERROR`)
 
-The file can be **very large (10+ GB)** — so you **cannot load it fully into memory**.
+The file can be very large (10+ GB), so you cannot load it fully into memory.
 
 ---
 
 ### Task:
 
-Write a **Python program** that:
+Write a Python program that:
 
-1. Reads the log file efficiently (without loading the entire file).
+1. Reads the log file efficiently, without loading the whole file.
 2. Counts how many times each sensor reported `ERROR`.
-3. Prints the **top 5 sensors** with the highest error counts.
+3. Prints the top 5 sensors with the highest error counts.
 
 **Bonus (Optional):**
 
@@ -45,10 +45,10 @@ Write a **Python program** that:
 
 ---
 
-💡 **Tips:**
+**Tips:**
 
-* Think about streaming/iterative processing.
-* Consider using `Counter`, `heapq`, or generator patterns.
-* Clean and readable code matters as much as correctness.
+* Think streaming, not batch.
+* Look at `Counter`, `heapq`, or generator patterns.
+* Clean, readable code matters as much as correctness.
 
 ---

problems/002-rolling-average-of-sensor-readings/question.md

@@ -7,11 +7,11 @@ difficulty: Easy
 solution: solution.py
 ---
 
-# Problem 2 — Rolling Average of Sensor Readings
+# Problem 2, Rolling Average of Sensor Readings
 
 **Scenario:**
 You are building a data pipeline for IoT sensors (like BESS, PV inverters, weather stations).
-Each sensor sends temperature data every few seconds to your system in this format:
+Each sensor sends temperature data every few seconds in this format:
 
 ```
 2025-10-11T13:45:20Z sensor_1 25.4
@@ -22,49 +22,49 @@ Each sensor sends temperature data every few seconds to your system in this form
 ...
 ```
 
-Each line contains:
+Each line has:
 
 * timestamp (ISO8601)
 * sensor_id
-* temperature (float, °C)
+* temperature (float, degrees C)
 
-The stream never ends (you can assume input comes from a file, MQTT, or Kafka — but treat it as **continuous input**).
+The stream never ends. Input could be a file, MQTT, or Kafka, but treat it as continuous input.
 
 ---
 
 ### Task:
 
 Write a Python program that:
 
-1. Continuously reads incoming sensor data (line by line).
-2. For **each sensor**, maintains a rolling average temperature over the **last 3 readings**.
-3. Each time a new reading arrives, print:
+1. Continuously reads incoming sensor data, line by line.
+2. For each sensor, keeps a rolling average temperature over the last 3 readings.
+3. Each time a new reading arrives, prints:
 
-   ```
-   <timestamp> <sensor_id> <rolling_average>
-   ```
+ ```
+ <timestamp> <sensor_id> <rolling_average>
+ ```
 
-   Example:
+ Example:
 
-   ```
-   2025-10-11T13:45:35Z sensor_1 26.17
-   ```
+ ```
+ 2025-10-11T13:45:35Z sensor_1 26.17
+ ```
 
 ---
 
 ### Bonus Challenges:
 
-* Make it **memory efficient** (don’t store all history).
-* Handle **multiple sensors** dynamically.
-* Handle malformed lines gracefully.
-* Ensure the code could be easily extended to a real streaming consumer (e.g., Kafka/MQTT).
+* Make it memory efficient. Don't store all history.
+* Handle many sensors dynamically.
+* Handle malformed lines without crashing.
+* Make the code easy to extend to a real streaming consumer (Kafka, MQTT).
 
 ---
 
-💡 **Tips:**
+**Tips:**
 
-* Think about how to store only the **last 3 values per sensor**.
-* You’ll probably want to use a **deque** or a simple list with slicing.
-* Focus on **clean, production-like structure** — not just working code.
+* Think about how to store only the last 3 values per sensor.
+* A `deque` works well here, or a list with slicing.
+* Aim for clean, production-like structure, not just working code.
 
 ---

problems/003-transform-and-clean-raw-data-for-analytics/question.md

@@ -7,12 +7,12 @@ difficulty: Medium
 solution: solution.py
 ---
 
-# Problem 3 — Transform and Clean Raw Data for Analytics
+# Problem 3, Transform and Clean Raw Data for Analytics
 
 **Scenario:**
-You receive raw user activity data from a partner API in a messy CSV file. The company wants to load it into a data warehouse (e.g., BigQuery or PostgreSQL) for analytics.
+You receive raw user activity data from a partner API as a messy CSV file. The company wants to load it into a data warehouse (BigQuery, PostgreSQL) for analytics.
 
-Here’s what the CSV looks like:
+Here is what the CSV looks like:
 
 ```
 user_id,name,email,signup_date,last_login,total_purchases
@@ -22,29 +22,29 @@ user_id,name,email,signup_date,last_login,total_purchases
 104,,maria@example.com,2025-02-10,,30
 ```
 
-But it’s full of **dirty and inconsistent data**:
+But it is full of dirty, inconsistent data:
 
 * Missing `name` or `email`
 * Invalid email format
 * Non-numeric values in `total_purchases`
 * Empty `last_login`
-* Dates in wrong order (e.g., signup_date > last_login)
+* Dates in wrong order (signup_date > last_login)
 
 ---
 
 ### Your Task:
 
 Write a Python program that:
 
-1. Reads the CSV file line by line (assume it’s large — can’t load into memory at once).
-2. **Validates and cleans** the data:
+1. Reads the CSV file line by line. Assume it is large and won't fit in memory.
+2. Validates and cleans the data:
 
-   * Skip rows with missing `email` or invalid email format.
-   * Replace missing `name` with `"Unknown"`.
-   * Replace missing `last_login` with `"N/A"`.
-   * Convert `total_purchases` to integer — if invalid, set it to `0`.
-   * If `signup_date` is **after** `last_login`, mark a new column `is_date_valid` as `False`, else `True`.
-3. Writes the cleaned data into a **new CSV file** called `cleaned_users.csv`.
+ * Skip rows with missing `email` or invalid email format.
+ * Replace missing `name` with `"Unknown"`.
+ * Replace missing `last_login` with `"N/A"`.
+ * Convert `total_purchases` to integer. If invalid, set it to `0`.
+ * If `signup_date` is after `last_login`, mark a new column `is_date_valid` as `False`, else `True`.
+3. Writes the cleaned data into a new CSV file called `cleaned_users.csv`.
 
 **Example Output (cleaned_users.csv):**
 
@@ -57,19 +57,19 @@ user_id,name,email,signup_date,last_login,total_purchases,is_date_valid
 
 ---
 
-### Bonus (if you want to go deeper like a senior engineer):
+### Bonus (if you want to go deeper):
 
 * Log how many rows were skipped and why.
 * Keep a separate file (`invalid_rows.csv`) for skipped rows.
 * Use a `pydantic` model for validation.
 
 ---
 
-💡 **Hints for Interview Thinking:**
+**Hints for Interview Thinking:**
 
-* Think about **streaming-style reads** (`csv` module or `DictReader`).
-* Use **regex for email validation**.
-* Be robust — assume bad inputs will appear often.
-* Make your solution easy to extend (e.g., add more validation rules later).
+* Think streaming reads (`csv` module or `DictReader`).
+* Use regex for email validation.
+* Assume bad inputs will show up often.
+* Make the solution easy to extend (more rules later).
 
 ---