Skip to content

feat(f3): blockstore read cache for F3.GetPowerTable #5674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat(f3): blockstore read cache for F3.GetPowerTable #5674

wants to merge 3 commits into from

Conversation

hanabi1224
Copy link
Contributor

@hanabi1224 hanabi1224 commented May 29, 2025 •
edited
Loading

Summary of changes

This PR addresses #5664 with a different but simpler approach

Changes introduced in this pull request:

  • Add a general block store read cache that can be used to improve the performance of any blockstore-read-bounded functions
  • Use block store read cache to improve the performance of F3.GetPowerTable (from ~1.7s to ~0.7s)

Log on mainnet

2025-05-29T09:53:54.625216Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006846 hit=6237 miss=41729 cache_len=41729 cache_size=13.7 MiB
2025-05-29T09:53:54.625254Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 12s 613ms 37us 652ns epoch=5006846 tsk=[bafy2bzacecjejeuezjtididp7wkx4qxa4yoibafm4enjphunrxp5f425rff6w, bafy2bzacea3ie2bbihfnrv5kiuma5rmdevzmkmwgionnlqyzotxuap36m76qe, bafy2bzacebgaethurdgrt4rvrctfgaiah3gxn2q7umudiox37jgqcs3s5ns76, bafy2bzaceacfhuc3sjrur2jev4ebyes33kulf4qmexerhosflqrjbebjaagx4, bafy2bzacedszklmz4hg44hklsxwtrlomci3bdsd25yb4vogcxd2jem6omxyu4]
2025-05-29T09:53:56.538644Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006857 hit=47946 miss=20 cache_len=41749 cache_size=13.7 MiB
2025-05-29T09:53:56.538681Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 914ms 649us 726ns epoch=5006857 tsk=[bafy2bzacecmlwmn2z2neebcib7fy7r3ya6a5ikx4cfua5mke3hy4amadeqtw6, bafy2bzacecgk3ix3tihik3zm4tmei5hwoyzq7fvzfu4yrvahop6ui5allbndg, bafy2bzacedivixav6si74slnoxodna4blqhx7od6zbi6nqpdicsn5blz4ba3o]
2025-05-29T09:53:57.295073Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006858 hit=47966 miss=0 cache_len=41749 cache_size=13.7 MiB
2025-05-29T09:53:57.295120Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 719ms 285us 416ns epoch=5006858 tsk=[bafy2bzacebzfy2westrqiej62iqxaa6j6klwf3g6qzhk7qbtuxogco2o44vsi, bafy2bzaced753vxtvtayho2mjkcbgqfz6ymnvj3dinudj5xaurys4eozwznwy, bafy2bzaceb6uf5sp2mfsbjjs27e52uenyhllmscou5zoilvwtb42n6ohxcb2k, bafy2bzacec76bzyide65bcrvufywlx5v6fcopm7ewwfz6ivngpctnvrpz54pa, bafy2bzaceb2aq27fdl5bwndfffe6ktng2dvnez7k2mxdqc3xfjpwgws2kpncm]
2025-05-29T09:53:58.026818Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006859 hit=47959 miss=7 cache_len=41756 cache_size=13.7 MiB
2025-05-29T09:53:58.026857Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 720ms 880us 902ns epoch=5006859 tsk=[bafy2bzaceas3flp6l3w6a3bb5xoouciqiqw4zkffdqs2pznzhuvh5gvu3fl3m, bafy2bzacebe5x5vqpgq6swi72xhjet6aziy4p6ccijrycwfu3ree2f6g5546c, bafy2bzacec7gekci3xgw2zsu4lp2o2nwujz3c2rv6uvh43otgordvdn77iyag, bafy2bzacebgqwiqs3mzcdfnadmijncfwxacqvlgzn2ldewn4v5gxbpjtzlqq4, bafy2bzaceatinmxecrhpsrw7qxg6cuycedtou6xjuhn6f4zrq62ixhxr5csem]
2025-05-29T09:53:58.655489Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006860 hit=47966 miss=0 cache_len=41756 cache_size=13.7 MiB
2025-05-29T09:53:58.655527Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 617ms 677us 181ns epoch=5006860 tsk=[bafy2bzacebw2vqas4ibsbkgwgkrqanhld67l5wwsac4ziyr7hrqbhsyrfbld6, bafy2bzacebvawkjfbxphrdwwyqierbhqmthp37vwi6oga6ggrcee7ctxravdm, bafy2bzaceandlnljkbml7vbbjgfm254xs2vvzag3gfktcgdwi6pbycysbf4re, bafy2bzacedvyzosjied6twndxugv7pp42zvkgok35o3u5dmdilw6evtg4kiog, bafy2bzacedsrrbxdkv7hmjeliu53ctvpszhm6dhn6kg4zrigzlzcvexqgcwna]
2025-05-29T09:53:59.343209Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable blockstore read cache epoch=5006861 hit=47958 miss=8 cache_len=41764 cache_size=13.7 MiB
2025-05-29T09:53:59.343251Z DEBUG forest::rpc::methods::f3: F3.GetPowerTable, took 677ms 113us 719ns epoch=5006861 tsk=[bafy2bzaceaulhwwwqn42er457jlm6rz3bxe2g25p2mmy533hlc3n3mpcy2aro, bafy2bzacecohigvva6qgkpjfvnpuv7vlyu7rqh72uxcg6ciikjswms5dgbas6, bafy2bzaceddp3gggbzp6rg5depcgwg2jhyhjarro5mskqovcnhu3tbt6zi4by, bafy2bzacedixm3xkfnvwuo7wzb6jx5quyfholkmhikflpvhbnfc3szrppierk, bafy2bzaceaunrsh5twwbxl7wn4b25tcll74yb6frp54q2ghccer65ulhfeumy, bafy2bzaceblymqfahq4glxgkmsbxlak2gvauumrx5rz4l25sjsos436atpqcc]

Reference issue to close (if applicable)

Closes

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

@hanabi1224 hanabi1224 marked this pull request as ready for review May 29, 2025 08:03
@hanabi1224 hanabi1224 requested a review from a team as a code owner May 29, 2025 08:03
@hanabi1224 hanabi1224 requested review from LesnyRumcajs and sudo-shashank and removed request for a team May 29, 2025 08:04
@LesnyRumcajs
Copy link
Member

What's the memory overhead? Should the cache size be configurable?

@hanabi1224
Copy link
Contributor Author

hanabi1224 commented May 29, 2025 •
edited
Loading

What's the memory overhead?

13.7MiB on mainnet according to the log

Should the cache size be configurable?

The current value has been fine-tuned for mainnet and I don't think it needs to be made configurable

@hanabi1224 hanabi1224 force-pushed the hm/blockstore-read-cache branch from f361696 to 7f684d9 Compare May 29, 2025 09:54
LesnyRumcajs
LesnyRumcajs reviewed May 29, 2025
View reviewed changes
src/rpc/methods/f3.rs
@@ -157,24 +161,33 @@ impl GetPowerTable {
ctx: &Ctx<impl Blockstore + Send + Sync + 'static>,
ts: &Arc<Tipset>,
) -> anyhow::Result<Vec<F3PowerEntry>> {
const BLOCKSTORE_CACHE_CAP: usize = 65536;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's note the memory overhead as a comment here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

LesnyRumcajs
LesnyRumcajs approved these changes May 29, 2025
View reviewed changes
Copy link
Member

@LesnyRumcajs LesnyRumcajs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LesnyRumcajs LesnyRumcajs LesnyRumcajs approved these changes

@sudo-shashank sudo-shashank Awaiting requested review from sudo-shashank sudo-shashank is a code owner automatically assigned from ChainSafe/forest

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hanabi1224 @LesnyRumcajs