You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have added an option to group multiple bit planes into a chunk (default is 4). To change it, provide, say --bit_planes_per_chunk 8, when encoding. With this change, the chunks are about 150KB on average.
The chunk size can also be changed with two options when encoding:
--bricks_per_chunk (default is 512). For example, --bricks_per_chunk 4096 should create chunks of size 1 MB ish.
--brick_size (default is 32 32 32). For example, --brick_size 64 64 64 should also create chunks of size 1 MB ish.
The first option (increasing the bricks-per-chunk) will result in more metadata to be stored, but is better in that the bricks are kept small so fewer samples need to be decoded when we ask for a small region of interest (e.g., a slice).
TODO
Can we plot the impact of the chunk size on data retrieval time?
Can we allow fetching parts of an object? (so that each chunk does not have to be stored as a file)
The text was updated successfully, but these errors were encountered:
hoang-dt
changed the title
[OpenViSUS and idx2 integration] idx2 chunk being too small for cloud
[OpenViSUS and idx2 integration] idx2 chunks being too small for cloud
Nov 28, 2022
I have added an option to group multiple bit planes into a chunk (default is 4). To change it, provide, say
--bit_planes_per_chunk 8
, when encoding. With this change, the chunks are about 150KB on average.The chunk size can also be changed with two options when encoding:
--bricks_per_chunk
(default is 512). For example,--bricks_per_chunk 4096
should create chunks of size 1 MB ish.--brick_size
(default is 32 32 32). For example,--brick_size 64 64 64
should also create chunks of size 1 MB ish.The first option (increasing the bricks-per-chunk) will result in more metadata to be stored, but is better in that the bricks are kept small so fewer samples need to be decoded when we ask for a small region of interest (e.g., a slice).
TODO
The text was updated successfully, but these errors were encountered: