Skip to content

Verify that piece-indexer is adding PieceCID->PayloadCID mappings for Curio SPs #126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bajtos opened this issue Feb 19, 2025 · 3 comments
Closed
Assignees

Comments

@bajtos
Copy link
Member

bajtos commented Feb 19, 2025

Improve piece-indexer to check if the advertisement's ContextID is formatted in the new way described in filecoin-project/FIPs#1089. If it is, extract PieceCID from there.

@bajtos bajtos moved this to 📥 next in Space Meridian Feb 19, 2025
@bajtos bajtos changed the title Verify that piece-indexer is adding PieceCID->PayloadCID mappings for Curio SPs. Verify that piece-indexer is adding PieceCID->PayloadCID mappings for Curio SPs Feb 19, 2025
@bajtos bajtos moved this from 📥 next to 📋 planned in Space Meridian Mar 3, 2025
@bajtos
Copy link
Member Author

bajtos commented Mar 11, 2025

Nice to do after we verify the new solution works as intended:

Write a one-time script to iterate over all walker-state:* entries. For each entry, parse the value as JSON and inspect the value of addsMissingPieceCID. If it's non-zero, then delete the entry to trigger a full re-scan of all advertisements from this provider.

This will populate the Piece->Payload index for older deals that were created in the past, before we deployed the patch.

@NikolasHaimerl
Copy link
Contributor

Piece indexer successfully walked the advertisement chain for the id 12D3KooWJ91c6xQshrNe7QAXPFAaeRrHWq2UrgXGPf8UmMZMwyZ5:

27.0.0.1:16379> GET walker-state:12D3KooWJ91c6xQshrNe7QAXPFAaeRrHWq2UrgXGPf8UmMZMwyZ5
"{\"head\":\"baguqeerabqqerhrmj3cknac7rmecl3eoirchplx7k44dc3jeayqbnwl5co7a\",\"tail\":\"baguqeera3vs4sxnjvdglzbmhia5hjfisanmwy7bbimlv5gxupkbpfuwp2kjq\",\"status\":\"Walking the advertisements from baguqeerabqqerhrmj3cknac7rmecl3eoirchplx7k44dc3jeayqbnwl5co7a, next step: baguqeera3vs4sxnjvdglzbmhia5hjfisanmwy7bbimlv5gxupkbpfuwp2kjq\"}"

@NikolasHaimerl
Copy link
Contributor

Nice to do after we verify the new solution works as intended:

Write a one-time script to iterate over all walker-state:* entries. For each entry, parse the value as JSON and inspect the value of addsMissingPieceCID. If it's non-zero, then delete the entry to trigger a full re-scan of all advertisements from this provider.

This will populate the Piece->Payload index for older deals that were created in the past, before we deployed the patch.

Done!

@github-project-automation github-project-automation bot moved this from 📋 planned to ✅ done in Space Meridian Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

3 participants