Skip to content

Update fide player rating mongodb scripts#19682

Open
allanjoseph98 wants to merge 3 commits intolichess-org:masterfrom
allanjoseph98:fide-player-rating-mongo
Open

Update fide player rating mongodb scripts#19682
allanjoseph98 wants to merge 3 commits intolichess-org:masterfrom
allanjoseph98:fide-player-rating-mongo

Conversation

@allanjoseph98
Copy link
Member

I wanted to reproduce fide rating history data locally and ran into the following issues:

  1. The python script failed to fetch rating lists because the regex failed to account for https links. I don't know if it's because FIDE has changed their site in the meanwhile but it was https for me.
  2. It took 10 minutes to convert the collection using the encoding script. Iterating over 700,000 elements and updateOne on each of them is slow. I asked copilot to migrate the script using updateMany and an aggregation to be more performant and it now runs in 30s. Does it matter for 99% of devs? No. But I guess there will be at least one other loon like me without mongo access to prod that will need to go down the same path that I did and I'd like to save them those 10 mins.

AI use disclaimer:

The encoding script rewrite was entirely vibe coded but verified to work on my dev setup using Copilot and GPT-5.3-Codex
Prompt:

  1. transform this migration to use updateMany and an aggregation function instead so that it is more performant.
  2. I get the following error when running the script on an already migrated collection: Plan executor error during update :: caused by :: $arrayElemAt's first argument must be an array, but is int. The script should try to detect whether the entry has been migrated or not and only run on non-migrated entries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant