Summary
Add cursor-based pagination and a hard result cap to the two miner endpoints. They currently return unbounded result sets.
Motivation
miners.service.ts:11-122 and miners.service.ts:125-225 return every row matching the author + since window with no LIMIT. A prolific contributor whose since is unset (defaults to 35 days, but can be passed in by the caller) can produce responses with hundreds of PRs, each carrying nested linked_issues, labels, review_summary JSON. The default 35-day window mitigates but does not bound — and the caller can pass any since.
The payload also embeds nested aggregations for every row (miners.service.ts:62-99), so memory and serialization cost scale super-linearly with result count.
Proposed Solution
API
Add to both endpoints:
GET /api/v1/miners/:githubId/pulls?since=...&cursor=...&limit=50
GET /api/v1/miners/:githubId/issues?since=...&cursor=...&limit=50
limit defaults to 50, capped at 200.
cursor is an opaque base64-encoded (created_at, repo_full_name, pr_number) tuple. Cursor decoding is server-side only — clients treat it as opaque.
- Response shape adds
next_cursor (null when no more rows).
Query change
Replace the ORDER BY p.created_at DESC clauses with:
ORDER BY p.created_at DESC, p.repo_full_name DESC, p.pr_number DESC
LIMIT $N
When cursor is present, add a keyset predicate using the decoded tuple. Keyset ordering is stable and uses idx_pull_requests_author plus the PK ordering.
Controller
Update miners.controller.ts:26-60 to accept cursor and limit query params via @ApiQuery annotations.
Backward compatibility
- Default response is bounded at 50 rows. Existing callers that don't paginate get a smaller page but in the same shape (plus
next_cursor).
- This is technically a breaking change for callers relying on getting all rows. To soften: ship behind a config flag
MINERS_PAGINATION_ENABLED and announce in CHANGELOG before flipping defaults.
Summary
Add cursor-based pagination and a hard result cap to the two miner endpoints. They currently return unbounded result sets.
Motivation
miners.service.ts:11-122 and miners.service.ts:125-225 return every row matching the author +
sincewindow with noLIMIT. A prolific contributor whosesinceis unset (defaults to 35 days, but can be passed in by the caller) can produce responses with hundreds of PRs, each carrying nestedlinked_issues,labels,review_summaryJSON. The default 35-day window mitigates but does not bound — and the caller can pass anysince.The payload also embeds nested aggregations for every row (miners.service.ts:62-99), so memory and serialization cost scale super-linearly with result count.
Proposed Solution
API
Add to both endpoints:
limitdefaults to 50, capped at 200.cursoris an opaque base64-encoded(created_at, repo_full_name, pr_number)tuple. Cursor decoding is server-side only — clients treat it as opaque.next_cursor(null when no more rows).Query change
Replace the
ORDER BY p.created_at DESCclauses with:When
cursoris present, add a keyset predicate using the decoded tuple. Keyset ordering is stable and uses idx_pull_requests_author plus the PK ordering.Controller
Update miners.controller.ts:26-60 to accept
cursorandlimitquery params via@ApiQueryannotations.Backward compatibility
next_cursor).MINERS_PAGINATION_ENABLEDand announce in CHANGELOG before flipping defaults.