Add new queue wait time metrics to dashboard

[ndr-ds]

Motivation

Proposal

Test Plan

Release Plan

• Nothing to do / These changes follow the usual release cycle.
• These changes should be backported to the latest devnet branch, then
  • be released in a new SDK,
  • be released in a validator hotfix.
• These changes should be backported to the latest testnet branch, then
  • be released in a new SDK,
  • be released in a validator hotfix.
• Backporting is not possible but we may want to deploy a new devnet and release a new
  SDK soon.

Links

• reviewer checklist

[ndr-ds]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Make batch sizes grow with network size #5135
• Add cross chain confirmation batch size option #5134
• Add new queue wait time metrics to dashboard #5133 👈 (View in Graphite)
• [will be closed] Enable fallback mode only based on epochs #5132
• [will be closed] Handling cross chain messages in batches #5131
• Move download_sender_certificates_for_receiver to proxy #5128 : 1 other dependent PR (#5158 )
• Send notifications from the proxy in batches #5129
• Cache deserialized immutable data on storage #5141
• Use quick_cache on ValueCache and introduce ParkingCache #5130
• RocksDB config tuning #5122
• Adding per block metrics to dashboard #5137
• Add metrics to track operations/bundles/transactions per block #5136
• Several dashboard changes #5124
• Send notifications in batches #5054
• Several metrics changes #5123
• Make notification and cross chain queue sizes grow with network size #5053
• Add node local DNS cache #5050
• Add Otel collector to our tracing exports to Tempo #5049
• Limit max pending message bundles in benchmarks #5121
• Properly set proxy and shard resources #5120
• Make tempo an actual chart dependency #5051 : 1 other dependent PR (#5052 )
• Combine building binaries into 1 cargo build command #5125
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.