Skip to content

Update measurement handling for Cloudflare protocol#321

Open
Victor1Ja wants to merge 7 commits into
developfrom
feat/cloudflare-measurements-v2
Open

Update measurement handling for Cloudflare protocol#321
Victor1Ja wants to merge 7 commits into
developfrom
feat/cloudflare-measurements-v2

Conversation

@Victor1Ja

@Victor1Ja Victor1Ja commented May 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Update measurement handling for Cloudflare protocol

Summary

This PR enables dual measurement providers (M-Lab + Cloudflare) on the backend: Cloudflare uploads via a dedicated route, persisted protocol discrimination with derived quality metrics, and a new protocol-config API for country/school provider settings. It also removes the legacy country_config table in favor of country_protocol_config / school_protocol_config, and adds repository documentation (PR history, ADR, agent rules).

Commits included (last 4):

Commit Description
7e41360 Cloudflare measurements, protocol-config module, Prisma migrations, tests
82cf5cf Drop CountryConfig model and MeasurementProvider enum; migration remove_old_country_config
c7528e0 Docs: docs/, prs/, .agent/, ADR 001, CONTRIBUTING/README updates
a3195bd PR template and validation checklist refinements

Context

Giga Meter must ingest connectivity data from more than one provider without breaking existing M-Lab clients. The Daily Check app and admin flows need:

  • A native Cloudflare upload path (camelCase payload).
  • Filterable protocol and quality fields on measurements without parsing JSON on every read.
  • Runtime configuration (which provider, delay between tests) resolved per school/country.

See ADR 001 and /prs/321-cloudflare-measurements-protocol.md for full technical decisions.

What changed

API

Method Path Notes
POST /api/v1/measurements/:protocol Phase 1: cloudflare only; mlab reserved → 400
POST /api/v1/measurements Unchanged legacy M-Lab path
GET /api/v1/measurements, /v2, … Optional protocol=mlab|cloudflare
GET /api/v1/protocol-config/resolve School → country → default
PUT / DELETE /api/v1/protocol-config/country/:countryCode Admin upsert/delete
PUT / DELETE /api/v1/protocol-config/school/:gigaIdSchool Admin upsert/delete

Code modules

  • src/measurement/ — Cloudflare DTOs, mapper, quality extraction (measurement-quality-metrics.ts), query validation, POST :protocol, list filters.
  • src/protocol-config/ — New module (controller, service, DTOs, tests).
  • src/public/ — Protocol-related fields on public measurement responses.
  • src/app.module.ts — Registers protocol-config.

Database

New / extended:

  • country_protocol_config, school_protocol_config
  • measurements.protocol (default mlab), quality columns + index on protocol

Removed:

  • country_config table
  • MeasurementProvider enum (replaced by string provider on protocol-config tables)

Migrations:

  1. 20260506140000_protocol_config_tables
  2. 20260512160000_add_measurement_protocol_quality_metrics
  3. 20260518164253_remove_old_country_config

Local seed (optional): src/prisma/seed.ts, src/prisma/scripts/local-dev-seed.sql

Documentation (repo)

  • .agent/CONTEXT_RULES.md — agent operational rules
  • docs/ — CURRENT_STATE.md, CHANGELOG.md, ADR 001
  • prs/ — PR history template + record for this change
  • CONTRIBUTING.md / README.md — links and PR documentation process

Technical highlights

  1. Cloudflare → shared persistence — mapCloudflareMeasurementToAddDto() then existing createMeasurement().
  2. Explicit columns + raw results — index-friendly queries; full JSON kept for audit.
  3. Best-effort quality extraction — malformed/partial results do not fail inserts.
  4. Protocol config precedence — school (only if override fields set) → country → default (mlab, 0s delay).
  5. Legacy cleanup — country_config dropped after protocol-config tables are in place.

Out of scope

  • POST /api/v1/measurements/mlab on the protocol route (reserved, not implemented).
  • Backfill of quality columns from historical results JSON.
  • Client-side Cloudflare offline retry (Daily Check app — separate PR).

Test plan

  • npm run test -- --testPathPattern="measurement|protocol-config|cloudflare"
  • Manual: POST /api/v1/measurements/cloudflare with fixture payload
  • Manual: GET /api/v1/protocol-config/resolve?gigaIdSchool=...

Related documentation

@Victor1Ja
Victor1Ja requested a review from vipulbhavsar94 May 18, 2026 16:11
Victor1Ja and others added 7 commits May 28, 2026 14:55
@Victor1Ja @cursoragent
Update measurement handling for Cloudflare protocol and enhance quali…
6d40f77 
…ty metrics extraction

- Updated `package.json` and `package-lock.json` to use `@nestjs/schedule` version 6.1.3 and `cron` version 4.4.0.
- Added new DTOs for Cloudflare measurements, including detailed metrics for data usage, access information, server information, and measurement results.
- Implemented mapping functions to convert Cloudflare measurement data into the application's internal format.
- Enhanced the `MeasurementController` to support creating measurements by protocol, specifically for Cloudflare.
- Added validation for measurement list queries and protocols.
- Introduced quality metrics extraction functions to derive metrics from Cloudflare results.
- Updated service methods to handle Cloudflare measurements and persist relevant quality metrics.
- Added tests to ensure correct functionality for Cloudflare measurement handling and quality metrics extraction.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Victor1Ja
Remove CountryConfig model and MeasurementProvider enum from Prisma s…
186a78e 
…chema, and update migration to drop related database constraints and tables.
@Victor1Ja
Add contribution guidelines for documenting merged PRs and update REA…
719f490 
…DME with links to new documentation files

- Introduced a section in `CONTRIBUTING.md` outlining the process for documenting merged pull requests, including file naming conventions and required content.
- Updated `README.md` to include links to `CONTRIBUTING.md`, `CHANGELOG.md`, and new documentation directories for per-PR history and AI agent rules.
- Added `CONTEXT_RULES.md` to define operational directives for AI agents working within the repository.
- Created `CHANGELOG.md` for maintaining an append-only log of merged PRs.
- Established `CURRENT_STATE.md` to serve as the single source of truth for the current system state.
- Added architecture decision records (ADR) to document significant architectural choices.

This commit enhances the clarity and structure of the contribution process and documentation for future contributors.
@Victor1Ja
Update PR template and validation steps for Cloudflare measurements
cd1d172 
- Removed the migration check from the PR template to streamline the validation process.
- Updated the validation section in the Cloudflare measurements protocol documentation to reflect that manual testing is now marked as complete, removing the need for a separate migration check.

These changes enhance clarity in the PR process and ensure accurate documentation of validation steps.
@Victor1Ja
Refactor protocol configuration by removing legacy country-config API
770eba5 
- Removed the `country-config` module, including its controller, service, and DTOs, as well as the associated database table and enum.
- Updated the `CHANGELOG.md` to reflect the removal of the legacy `country-config` API and the introduction of the `protocol-config` module for country/school configuration management.
- Enhanced documentation in `CURRENT_STATE.md` and ADR 001 to clarify the new structure and the rationale behind the removal of legacy components.
- Adjusted the application module to unregister the `CountryConfigModule` and ensure all references to the legacy API are removed.

These changes streamline the configuration management process and eliminate outdated components, promoting a more cohesive architecture.
@Victor1Ja
Refactor MeasurementController tests to override guards for improved …
e29934b 
…test isolation

- Removed the direct provision of `AuthGuard` and `ThrottlerGuard` in the test setup.
- Implemented overrides for these guards to return a resolved promise for `canActivate`, enhancing test reliability and isolation.
- Updated `ProtocolConfigService` tests to include new properties in the country protocol configuration, ensuring alignment with the latest schema changes.

These adjustments streamline the testing process and ensure that tests accurately reflect the current application logic.
@Victor1Ja
Remove deprecated countryConfigs field from the country model and del…
2771778 
…ete obsolete migration files related to country configuration and measurement protocol quality metrics. Introduce a new migration for the updated protocol configuration structure, including new tables and foreign key constraints.
@Victor1Ja
Victor1Ja force-pushed the feat/cloudflare-measurements-v2 branch from b4d8f48 to 2771778 Compare May 28, 2026 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vipulbhavsar94 vipulbhavsar94 Awaiting requested review from vipulbhavsar94

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Victor1Ja