perf: swap Signature.bind for internal dataclass within Annotable #11691
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
This PR makes some performance improvements to the
Annotable
class which is a base used for many ibis internals, particularlyNode
, and therefore provides a performance improvement to building all expressionsWhile doing further investigation around points made in #11641 I found that
Signature.bind
is a major bottleneck in instantiation of Annotable objects.Signature.bind
is a python library function that implements, for a given python function/class, an "interpreter" for mapping passed args/kwargs to actual named args/kwargs. However because it is implemented in python it is orders of magnitude slower (~10-20x) than the cpython code that implements the same process.In this PR I've therefore replaced calling Signature.bind with following:
Annotable
class is created, also create a "proxy dataclass" using standard python dataclasses, with exactly the same signature__dict__
from that dataclass instanceChanges are not that intrusive but a little bit "funky". Performance is improved by 10-50% across all expression building benchmarks (larger expressions benefit more)
This is a POC - a few tests that check Annotable raises when incorrect args/kwargs are passed fail because exceptions raised have slightly different text than before. I think these are all solvable but I wanted to check that the approach is acceptable before continuing
I've attached some profiles. (ipython) code to generate these is below
profile-after.txt
profile-before.txt
@kszucs and @cpcloud if you could take a look as time allows and let me know thoughts that'd be much appreciated. Thanks!