Skip to content

Conversation

@danielsn
Copy link
Contributor

@danielsn danielsn commented Nov 12, 2025

Description

Converts the memory profiler from C to C++.
This enables us to use things like RAII to manage memory, and collections like std::vector instead of hand-rolled array lists.
There should be no functional differences.

Testing

Refactor change that is covered by existing tests.

Risks

Additional Notes

@danielsn danielsn requested review from a team as code owners November 12, 2025 21:31
@danielsn danielsn requested a review from nsrip-dd November 12, 2025 21:31
@danielsn danielsn added the changelog/no-changelog A changelog entry is not required for this PR. label Nov 12, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 12, 2025

CODEOWNERS have been resolved as:

ddtrace/profiling/collector/_memalloc_heap_map.hpp                      @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc.cpp                               @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_debug.h                           @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_heap.cpp                          @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_heap.h                            @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_heap_map.cpp                      @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_reentrant.h                       @DataDog/profiling-python
ddtrace/profiling/collector/_memalloc_tb.h                              @DataDog/profiling-python
ddtrace/profiling/collector/_pymacro.h                                  @DataDog/profiling-python
setup.py                                                                @DataDog/python-guild
ddtrace/profiling/collector/_memalloc_heap_map.h                        @DataDog/profiling-python
ddtrace/profiling/collector/_utils.h                                    @DataDog/profiling-python

void
memalloc_heap_tracker_init(uint32_t sample_size)
{
// TODO(dsn): what should we do it this was already initialized?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we take any risks by reusing the existing heap tracker?

Comment on lines 393 to 394
global_heap_tracker->deinit();
delete global_heap_tracker;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't deinit be (or be part of) the destructor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like deinit is a bad name. The issue is that currently, deallocating the global_heap_tracker requires calls into cpython, which could release the GIL. So we need a way to stop recording those operations as we're deallocating. I'm not sure this is the right way tho. I will give that some thought.


// memalloc_heap_map implementation
memalloc_heap_map::memalloc_heap_map()
: impl(new Impl())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to have a unique_ptr<Impl> instead?

};

memalloc_heap_map::iterator::iterator()
: impl(new Impl())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here with unique_ptr vs. raw pointer

} heap_tracker_t;
};

static heap_tracker_t global_heap_tracker;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To confirm, have you made sure we have everything needed to handle forks and correctly allocating a new global_heap_tracker on fork?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to add unit tests for that container?

@danielsn danielsn added the Profiling Continous Profling label Nov 13, 2025
@danielsn danielsn requested a review from a team as a code owner November 13, 2025 13:59
@danielsn danielsn requested a review from RamyElkest November 13, 2025 13:59
@github-actions
Copy link
Contributor

github-actions bot commented Nov 13, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 235 ± 3 ms.

The average import time from base is: 241 ± 2 ms.

The import time difference between this PR and base is: -5.3 ± 0.1 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 2.360 ms (1.00%)
ddtrace.bootstrap.sitecustomize 1.623 ms (0.69%)
ddtrace.bootstrap.preload 1.623 ms (0.69%)
ddtrace.internal.remoteconfig.client 0.703 ms (0.30%)
ddtrace 0.737 ms (0.31%)
ddtrace.internal._unpatched 0.072 ms (0.03%)
json 0.072 ms (0.03%)
json.decoder 0.072 ms (0.03%)
re 0.072 ms (0.03%)
enum 0.072 ms (0.03%)
types 0.036 ms (0.02%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR. Profiling Continous Profling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants