Skip to content

Remove redundant datas from sync-store #2236

@valamidev

Description

@valamidev

Problem / use case

I did run a minor benchmark, reference-erc20 example:

 startBlock: 13142655,
endBlock: 13151917,

Current ponder_sync table sizes after sync:

"ponder_sync"."transactions"    "15 MB"
"ponder_sync"."logs"	             "13 MB"
"ponder_sync"."blocks"	              "7320 kB"

Sum: ~35MB

After removed every data which is either duplicated, not used anywhere in the Ponder.

"ponder_sync"."logs"	               "10192 kB"
"ponder_sync"."blocks"	               "4992 kB"
"ponder_sync"."transactions"      "4400 kB"

Sum: ~20MB

Almost 50% storage reduction without any schema change or using compression simply set to default value in the sync-store/encode.ts those fields which are not in use, running Ponder at scale it is significant our Ponder infra eat ~500GB / month with 90/10 ratio between ponder_sync size / actually indexed data size.

Proposed solution

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions