[18343808662] MemBlock refactor #2792

IvoDD · 2025-12-03T10:41:48Z

Reference Issues/PRs

Monday ref: 18343808662

What does this implement or fix?

This is only a refactor to prepare for parallel bool unpacking and should not change existing behavior.

It splits up the MemBlock into two separate classes inheriting a common interface IMemBlock:

DynamicMemBlock is responsible for the AllocationType::DYNAMIC and has a capacity which can gradually be filled up with resizes. This is the type of memory blocks we use for in memory operations with ChunkedBuffers e.g. when tick streaming or when performing appends etc.
ExternalMemBlock is responsible for the AllocationType::DETACHABLE and for external pointers to views. It just holds a pointer to external memory which can be owning or not. This is what we use when either when returning Segments to python or when receiving views from python

This change also adds the scaffolding needed for the packed bits buffer via ExternalPackedMemBlock. It will be used in a follow up PR to parallelise bool unpacking.

Any other comments?

Also introduced a few micro benchmarks from which the results are:

Benchmark	Time before	Time after
BM_chunked_buffer_random_access/100000/203/1/0	16.7 ns	15.5 ns
BM_chunked_buffer_random_access/100000/203/0/0	40.5 ns	34.9 ns
BM_chunked_buffer_random_access/100000/203/0/2	112 ns	117 ns
BM_chunked_buffer_allocate_with_ensure/100000/203/1/0	7.16 ms	7.84 ms
BM_chunked_buffer_allocate_with_ensure/100000/203/0/0	7.61 ms	8.13 ms
BM_chunked_buffer_allocate_with_ensure/100000/203/0/2	24.0 ms	17.9 ms

The benchmarks are run with:

./benchmarks --benchmark_filter=BM_chunked_buffer_allocate.* --benchmark_time_unit=ms --benchmark_min_time=3s --benchmark_repetitions=5
./benchmarks --benchmark_filter=BM_chunked_buffer_random_access.* --benchmark_time_unit=ns --benchmark_min_time=1s --benchmark_repetitions=5

I've not included the larger chunk size because that is identical for before and after. The only statistically significant difference is the allocation on a DETACHABLE segment improvement. That is because for ExternalMemBlocks we no longer need to allocate the intrinsic data that was previously always included in the MemBlock.

Checklist

Checklist for code changes...

Have you updated the relevant docstrings, documentation and copyright notice?
Is this contribution tested against all ArcticDB's features?
Do all exceptions introduced raise appropriate error messages?
Are API changes highlighted in the PR description?
Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

IvoDD · 2025-12-04T11:20:20Z

cpp/arcticdb/column_store/chunked_buffer.hpp

            pos_ += type_size_;

-            if (pos_ >= block_->bytes()) {
+            if (pos_ >= block_->logical_bytes()) {


Highlighting a tiny behavior change. The Column::Iterator was previously broken for blocks with extra_bytes>0

cpp/arcticdb/column_store/chunked_buffer.cpp

cpp/arcticdb/column_store/block.cpp

alexowens90 · 2025-12-08T11:34:58Z

cpp/arcticdb/column_store/block.hpp

+    virtual void resize(size_t bytes) = 0;
+    virtual void check_magic() const = 0;
+    // External block specific methods
+    [[nodiscard]] virtual uint8_t* release() = 0;
+    virtual void abandon() = 0;


It's a bit of a code smell to have interface methods for specific subclasses. The alternative is to have callers of these methods check the return value of get_type and do a reinterpret_cast as appropriate though, which also feels clunky

yeah only 90% of the interface is common and the choices either to have them as methods on the interface or to do dynamic_cast after checking get_type. The other option would be to not use inheritance but rather std::variant which would be clunkier.

Main reason I chose having them in the interface was that presumably it will be faster to just read the vtable once instead of doing get_type which also reads vtable and dynamic_cast which needs to read the RTTI.
Compiler might optimize away some of this though.

Will add a comment explaining why I think using this instead of dynamic_cast is presumably faster. Happy to be overruled though.

Note reinterpret_cast would probably be undefined behavior

My two cents:

Both are ugly

We have something that is similar to the current proposal e.g. not all clauses support all clause methods and they throw exceptions. Thus I think this is ok

Reinterpret cast won't be UB https://en.cppreference.com/w/cpp/language/reinterpret_cast.html 5) handles this case.

Checking the vtable is not that slow especially if it's not in a hot loop

Adding final to the derived classes would help the compiler optimize vtables if possible

Ah thanks for clarifying. Will add final to all methods that it makes sense for.

folly::Poly doesn't have a concept of a base class implementation, which is why we have to explicitly throw from the derived classes for clauses. We could make the base class methods non-virtual and throw an exception, and only override in derived classes where the methods make sense.

cpp/arcticdb/column_store/chunked_buffer.hpp

alexowens90 · 2025-12-08T11:53:50Z

Worth running all the Arrow tests and a smokescreen of non-Arrow tests with valgrind to make sure we haven't introduced any leaks.

vasil-pashov · 2025-12-10T11:38:46Z

cpp/arcticdb/arrow/test/test_arrow_write.cpp

+            ASSERT_EQ(buffer.blocks()[idx]->physical_bytes(), rows_per_batch[idx] * sizeof(TypeParam));
+            ASSERT_EQ(buffer.blocks()[idx]->logical_size(), rows_per_batch[idx] * sizeof(TypeParam));


nit: These two names look odd side by side. Does it make sense to either rename physical_bytes to physical_size or logical_size to logical_bytes

I have explicitly named logical_size to highlight that it does not refer to number of bytes. For the ExternalPackedBuffer buffer it refers to number of bits.

On the other hand physical_bytes does refer to the actual size of memory occupied in bytes. Thus I think the distinction is useful.

vasil-pashov · 2025-12-10T12:19:44Z

cpp/arcticdb/column_store/block.hpp

+    virtual void resize(size_t bytes) = 0;
+    virtual void check_magic() const = 0;
+    // External block specific methods
+    [[nodiscard]] virtual uint8_t* release() = 0;
+    virtual void abandon() = 0;


My two cents:

Both are ugly

We have something that is similar to the current proposal e.g. not all clauses support all clause methods and they throw exceptions. Thus I think this is ok

Reinterpret cast won't be UB https://en.cppreference.com/w/cpp/language/reinterpret_cast.html 5) handles this case.

Checking the vtable is not that slow especially if it's not in a hot loop

Adding final to the derived classes would help the compiler optimize vtables if possible

vasil-pashov · 2025-12-10T12:32:20Z

cpp/arcticdb/column_store/block.hpp

+    [[nodiscard]] virtual uint8_t& operator[](size_t pos);
+    [[nodiscard]] const uint8_t* ptr(size_t pos) const;
+    [[nodiscard]] uint8_t* ptr(size_t pos);
+    [[nodiscard]] uint8_t* end() const;


ExternalPackedMemBlock's override of operator[] throws. On the other hand you can call ptr() on it as it's part of the base class. But they are mostly doing the same thing. I'd remove the virtual operator[].

Ah thanks, the idea was ptr to use the operator[] but I missed that. Will remove operator[] as it's not used anywhere meaningfully.

vasil-pashov · 2025-12-10T12:56:30Z

cpp/arcticdb/column_store/block.cpp

+ExternalMemBlock::ExternalMemBlock(
+        uint8_t* data, size_t size, size_t offset, entity::timestamp ts, bool owning, size_t extra_bytes
+) :
+    bytes_(size),
+    offset_(offset),
+    timestamp_(ts),
+    external_data_(data),
+    owns_external_data_(owning),
+    extra_bytes_(extra_bytes) {}


We can remove this. The other constructor's doing the same thing.

vasil-pashov · 2025-12-10T12:58:49Z

cpp/arcticdb/column_store/block.cpp

+}
+
+void ExternalMemBlock::abandon() {
+    free_detachable_memory(external_data_, physical_bytes());


Shouldn't we check if this is owning the external data?

yes, good catch. Previously we were only doing an assert here, but now we use it in abandon_block, so using an if.

vasil-pashov · 2025-12-10T13:07:50Z

cpp/arcticdb/column_store/block.cpp

+
+void ExternalMemBlock::resize(size_t bytes) { util::raise_rte("Can't resize a non dynamic block. Bytes: {}", bytes); }
+
+void ExternalMemBlock::check_magic() const {}


I know it's not releated to your changes but I think we should either get rid of the magic or start using it. AFAIK it was added for the V2 encoding which used native structures instead of protobuf. The magic was placed at the end of the buffer containing some object. This way you could

Inspect the raw byte buffer and get a sense of what's in it

Verify that something was loaded correctly

I don't think the mem blocks need it

agreed, removing entirely

vasil-pashov · 2025-12-10T13:08:36Z

cpp/arcticdb/column_store/block.hpp

+
+  private:
+    size_t logical_size_ = 0UL;
+    size_t shift_ = 0UL;


Discussed offline, but a comment about what shift is will be helpful

vasil-pashov · 2025-12-10T13:15:55Z

cpp/arcticdb/column_store/chunked_buffer.hpp


-    static_assert(sizeof(BlockType) == BlockType::Align + BlockType::MinSize);
-    static_assert(DefaultBlockSize >= BlockType::MinSize);
+    static_assert(sizeof(DynamicMemBlock) == DynamicMemBlock::Align + DynamicMemBlock::MinSize);


It makes more sense for this to be near the DynamicMemBlock implementation rather than here

This is only a refactor to prepare for parallel bool unpacking and should not change existing behavior. It splits up the `MemBlock` into two separate classes inheriting a common interface `IMemBlock`: - `DynamicMemBlock` is responsible for the `AllocationType::DYNAMIC` and has a capacity which can gradually be filled up with resizes. This is the type of memory blocks we use for in memory operations with `ChunkedBuffer`s e.g. when tick streaming or when performing appends etc. - `ExternalMemBlock` is responsible for the `AllocationType::DETACHABLE` and for external pointers to views. It just holds a pointer to external memory which can be owning or not. This is what we use when either when returning Segments to python or when receiving views from python This change also adds the scaffolding needed for the packed bits buffer via `ExternalPackedMemBlock`. It will be used in a follow up PR to parallelise bool unpacking.

The checks in particular getting the `physical_bytes` introduced a slowdown in `BM_iteration_via_scalar_at` because: - Previously `bytes()` call could be inlined and it can no longer be inlined because of the runtime polymorphism - The non inlined function calls are taking a big part of the time. Removing the check in `operator[]` removes two function calls in release builds. This brings performance roughly inline to before the refactor.

IvoDD added patch Small change, should increase patch version no-release-notes This PR shouldn't be added to release notes. labels Dec 3, 2025

IvoDD changed the title ~~MemBlock refactor~~ [18343808662] MemBlock refactor Dec 3, 2025

IvoDD force-pushed the mem-block-refactor branch 6 times, most recently from 3c5044d to de733b8 Compare December 4, 2025 11:13

IvoDD commented Dec 4, 2025

View reviewed changes

IvoDD force-pushed the mem-block-refactor branch 2 times, most recently from fd7c6a3 to 7822686 Compare December 4, 2025 13:15

IvoDD marked this pull request as ready for review December 4, 2025 13:16

IvoDD requested review from alexowens90 and poodlewars as code owners December 4, 2025 13:16

IvoDD force-pushed the mem-block-refactor branch 2 times, most recently from 21acd7d to 1753b0d Compare December 8, 2025 08:08

alexowens90 requested changes Dec 8, 2025

View reviewed changes

IvoDD force-pushed the mem-block-refactor branch 2 times, most recently from 9d9b0bc to 0662547 Compare December 9, 2025 13:07

vasil-pashov reviewed Dec 10, 2025

View reviewed changes

IvoDD force-pushed the mem-block-refactor branch 3 times, most recently from a739aec to d3efb32 Compare December 12, 2025 08:03

vasil-pashov approved these changes Dec 16, 2025

View reviewed changes

IvoDD added 4 commits December 23, 2025 10:12

Benchmarks for MemBlock

1ed03e1

Address Alex's comments

5b478f0

Address Vasil's comments

d8998e6

IvoDD force-pushed the mem-block-refactor branch from d3efb32 to 1aa9c4f Compare December 23, 2025 11:12

IvoDD force-pushed the mem-block-refactor branch from 1aa9c4f to 5ebe971 Compare December 23, 2025 12:49

		ASSERT_EQ(buffer.blocks()[idx]->physical_bytes(), rows_per_batch[idx] * sizeof(TypeParam));
		ASSERT_EQ(buffer.blocks()[idx]->logical_size(), rows_per_batch[idx] * sizeof(TypeParam));


		void ExternalMemBlock::resize(size_t bytes) { util::raise_rte("Can't resize a non dynamic block. Bytes: {}", bytes); }

		void ExternalMemBlock::check_magic() const {}

[18343808662] MemBlock refactor #2792

Are you sure you want to change the base?

[18343808662] MemBlock refactor #2792

Uh oh!

Conversation

IvoDD commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement or fix?

Any other comments?

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexowens90 Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alexowens90 commented Dec 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vasil-pashov Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

IvoDD commented Dec 3, 2025 •

edited

Loading

alexowens90 Dec 15, 2025 •

edited

Loading

vasil-pashov Dec 10, 2025 •

edited

Loading