Skip to content

Fixes after the move to proxy-scraper.de - #361

Merged
maximilianfeix merged 1 commit into
mainfrom
fix/code-review-2026-10-25
Oct 25, 2026
Merged

maximilianfeix merged 1 commit into
mainfrom
fix/code-review-2026-10-25

Conversation

@maximilianfeix

Copy link
Copy Markdown
Owner

Found with a high-effort code review of #360:

  • Report feed had two entries for 2026-W42 live (ids contained the old github.io URL). Entries are now matched by week; one per week is kept. The live duplicate disappears with next Monday's entry.
  • LiveSource (MCP + Python API) read only the website. The website upload is separate and may fail, so a stale website copy now triggers a look at the proxy-list branch; the newer list wins. Tests with a fixed clock.
  • Website upload: rsync -rl --checksum instead of -rlt, so unchanged files keep mtime/ETag (304s work); file count excludes .git.
  • Bot, CITATION.cff, pyproject: links → https://proxy-scraper.de. ⚠️ Merging deploys the bot (bot/** changed).
  • mirror/publish wording; publish.py uses SITE_URL.

Tests: 1404 passed, 2 skipped; bot 38 passed; ruff clean.

…wer list wins, stable timestamps, bot URL

- report feed: match entries by week, not by id (ids carried the old URL, so W42 was added twice)
- LiveSource: when the website copy is stale, also ask the proxy-list branch and keep the newer list
- website upload: rsync without -t so unchanged files keep their mtime/ETag; file count without .git
- bot, CITATION.cff, pyproject Documentation: https://proxy-scraper.de
- mirror/publish: say "website", publish.py uses SITE_URL
@maximilianfeix
maximilianfeix merged commit b1ade5b into main Oct 25, 2026
25 checks passed
@maximilianfeix
maximilianfeix deleted the fix/code-review-2026-10-25 branch October 25, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant