Use timer coalescing instead of globalThis.setTimeout for GC to avoid performance issues

[justjake]

I'm investigating adopting tanstack/query in a large production application called Notion. We need to support tens of thousands of queries on screen at once. I'm concerned that the current implementation of query GC uses individual setTimeout per query. This means after a navigation, tanstack/query-core would call setTimeout tens of thousands of times, which can cause a lot of event loop delay.

Our solution to this issue with our current internal system is to use a very simple timer coalescing with a scheduler, so instead of 30k setTimeout calls, we have either 0 or 1 calls to setInterval. It would be great to update Removable#scheduleGc to do its scheduling via something like TimeoutGroup, similar to how the system uses notifyManager. Pseudocode below:

type PendingTimeout = {
  id: number
  deadline: number
  callback: () => void
}

class TimeoutGroup {
  constructor(private wakeupIntervalMs: number) {}
  private interval: typeof ReturnType<setInterval> | undefined
  private timeouts = new Map<number, PendingTimeout>
  private nextId = 1

  setTimeout(callback: () => void, ms: number) {
    const id = this.nextId++
    this.timeouts.set(id, { id, callback, deadline: ms + Date.now() })
    this.schedule()
    return id
  }

  clearTimeout(id: number) {
    try {
      return this.timeouts.delete(id)
    } finally {
      this.unschedule()
    }
  }

  flushPendingTimeouts() {
    const now = Date.now()
    for (const timer of this.timeouts.values()) {
      if (timer.deadline <= now) { this.flushTimeout(timer) }
    }
    this.unschedule()
  }

  private schedule() {
    if (!this.timeouts.size) { return }
    this.interval ??= setInterval(() => this.flushPendingTimeouts(), this.wakeupIntervalMs)
  }

  private unschedule() {
    if (this.timeouts.size || !this.interval) { return }
    clearInterval(this.interval)
    this.interval = undefined
  }

  private flushTimeout(timer: PendingTimeout) {
    this.timeouts.delete(timer.id)
    try {
      timer.callback()
    } catch (error) {
      Promise.resolve().then(() => { throw error })
    }
  }
}

to avoid huge event loop delay after navigation unmounts a bunch of QueryObservers.

[justjake]

I looked around for other open-source libraries and found timer-wheel on npm which implements an expiry data structure that appears high quality and fulfills our needs. We could implement a TimoutManager (similar to NotifyManager and OnlineManager) using different strategies based on the accuracy needs of the caller:

• Very small timeouts (<5s) continue using native setTimeout
• Small timeouts (< 60s) use a simple strategy and fire with 10s granularity
• Long timeouts (>= 60s) use TimerWheel and fire with 30s granularity
• We could increase or decrease the resolution of the long-timeout setInterval depending on the resolution of the remaining timeouts in the bucket. If they're all >60 minutes away, can switch to a 30 minute interval.