Skip to content

Historic: Perform Reorg Check Only After Streaming Last Block Range #227

@0xNeshi

Description

@0xNeshi

Assumptions:

  • reorgs are super rare
  • reorgs during historic are even rarer
  • caller is willing to pay performance cost in case of reorgs, if in return they get much better performance in majority, non-reorg cases

Changes:

  • BlockRangeScanner (BRS) kept mostly as-is, i.e. as a sequential block range "calculator"
    • EventScanner (ES) receives block ranges as fast as BRS can provide them, handles all the parallel logic (separate subject)
    • BRS in historic mode no longer handles reorgs before/after each stream; instead, it checks if a reorg occurred only after it streams the last block range
  • In case a reorg did occur, it re-streams all relevant non-finalized blocks
    • No fancy common ancestor logic
    • Performs the last 2 steps as many times as necessary
  • ES handles corrected ranges as long as BRS is streaming them

Results:

  • Clean separation of concerns
    • Block range and reorg logic still separately handled by BRS
    • Parallelization and result ordering a concern of ES
  • Increased performance in historic/sync modes
    • no batch_end block fetching RPC calls
    • no intermediate reorg check RPC calls
    • fetching 1-2 blocks "costs" almost the same as fetching 64 blocks

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: enhancementImprovement to existing functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions