Skip to content

Conversation

@andralex
Copy link
Member

While tracking dmd's memory usage I noticed there are very many calls to OutBuffer::enlarge, which in turn makes a ton of calls to realloc:

1999520 calls to allocation functions with 1.07GB peak consumption from
Outbuffer::enlarge(unsigned long)

Turns out the vast majority of these calls are with buf is null and many others deal with small sizes, unnecessarily making realloc roundtrips.

This improves the strategy to special-case frequent cases and to double the allocated size for small buffers. The result for the same build:

1247003 calls to allocation functions with 773.87MB peak consumption from
Outbuffer::enlarge(unsigned long)

I also took the liberty to make a bunch of functions private. I have no idea why that works because some are actually used from client code. Why is that the case?

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @andralex!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#12519"

Copy link
Member

@ibuclaw ibuclaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/dmd/backend/cv8.d(141): Error: no property `writen` for type `dmd.backend.outbuf.Outbuffer*`
src/dmd/backend/cv8.d(142): Error: no property `writeByte` for type `dmd.backend.outbuf.Outbuffer*`

Not sure why it compiled for you locally, but private should not have been added to any of the member functions.

@John-Colvin
Copy link
Contributor

Great find! 👍

@andralex
Copy link
Member Author

src/dmd/backend/cv8.d(141): Error: no property `writen` for type `dmd.backend.outbuf.Outbuffer*`
src/dmd/backend/cv8.d(142): Error: no property `writeByte` for type `dmd.backend.outbuf.Outbuffer*`

Not sure why it compiled for you locally, but private should not have been added to any of the member functions.

Heh, that explains a lot. I was building using the makefile and it seems dependencies are not tracked properly, Now I touch **/*.d before building... will get back.

@andralex
Copy link
Member Author

I took the liberty to remove some cruft as well. private is such an awesome tool.

Copy link
Member

@ibuclaw ibuclaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andralex andralex merged commit 074939d into dlang:master May 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants