Skip to content

Optimize sender-filter performance for getAnnouncementsForUser and watch/scan flows #107

Description

@marcomariscal

includeList / excludeList filtering currently requires a getTransaction(hash) lookup per relevant announcement to recover the sender address.

That is correct, but it can become the dominant cost in:

  • large historical scans/pages
  • live watch bursts
  • any flow that combines user-key filtering with sender filtering

The default path without sender filters is fine. This issue is specifically about reducing RPC fanout when sender filters are used.

Current behavior

  • getAnnouncementsForUser(...) first checks whether each announcement is for the user
  • if includeList or excludeList is provided, it then fetches the transaction by hash to get from
  • this happens per announcement

Why this matters

  • per-log tx lookups are much more expensive than the local stealth-address check
  • history pages can trigger many RPC calls
  • live watch bursts can backlog behind sender lookups
  • this cost is paid even though sender filtering is conceptually just a secondary filter

Goals

  • preserve current include/exclude semantics
  • reduce RPC fanout and latency when sender filters are used
  • avoid changing the default no-filter path

Possible approaches

  • add per-session txHash -> from caching for scan/watch flows
  • dedupe repeated transaction lookups within a batch
  • evaluate indexed sender data in the subgraph/Ponder path
  • document sender filtering as a more expensive option until optimized

Non-goals

  • changing stealth-address matching behavior
  • redesigning the watch queue/backpressure model in this issue
  • adding a new high-level follow/fused API

Acceptance criteria

  • sender-filtered scans/watches do fewer redundant RPC calls
  • behavior remains identical for include/exclude filtering
  • no regression to the no-filter path
  • tests cover cache/dedupe behavior and unchanged filtering semantics

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions