You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently when loading a compressed solution file, it takes very long because zstd doesn't support multi-threaded decompression. Only compression.
One way around this would be to instead serialize into multiple files instead of one large file. Particularly the Vec[u8] data from storage1 and node_arena which can then be deserialized in parallel into a single vector in memory.
Currently when loading a compressed solution file, it takes very long because zstd doesn't support multi-threaded decompression. Only compression.
One way around this would be to instead serialize into multiple files instead of one large file. Particularly the
Vec[u8]
data fromstorage1
andnode_arena
which can then be deserialized in parallel into a single vector in memory.It looks like pzstd uses a similar strategy.
I can submit a PR if the library author is interested. I have something working now which looks promising:
The text was updated successfully, but these errors were encountered: