-
Notifications
You must be signed in to change notification settings - Fork 5
Add caching of sharding function #85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the patch! See my comments inline.
Also please add a 'Closes #82' to commit message and a number of issue to PR description.
a5089d1
to
9e0f3f8
Compare
d8c016d
to
a1799da
Compare
26c1da3
to
62da6fa
Compare
62da6fa
to
213337a
Compare
213337a
to
3862d21
Compare
81fcff6
to
42b0295
Compare
878ee46
to
b3cd5ab
Compare
Since we aim to resolve a performance problem here I would like to see some performance measurements. For example, the following:
It would be good to commit the microbenchmark right into the repository with a short instruction. If the good case is not so good, we can look a bit deeper and, say, check, whether implementing a fast path for a function that is placed directly in |
b3cd5ab
to
79ea5db
Compare
79ea5db
to
f0539c2
Compare
adefb00
to
9b0fc46
Compare
9b0fc46
to
989e61f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your fixes!
In general everything is fine, however I left several non-critical comments. Please consider them. I will give an LGTM to don't block merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Please, address stylistic suggestions from Sergey, squash (or rework-and-squash) the proposed commit into your one and let me know that everything is done.
The ddl.bucket_id() function needs to know a sharding function. It is costly to obtain the function declaration / definition stored in the _ddl_sharding_func space. Cache contains: raw_tuple - raw sharding metadata, used for get_schema() parsed_func_name - parsed dot notation (like {'foo', 'bar'}) callable - function ready to call, this offloads using of loadstring() error - string with an error: not nil only if setting callable fails Cache will be rebuilded if: * _ddl_sharding_func space changed: cache sets _ddl_sharding_func:on_replace trigger * schema changed: cache checks box.internal.schema_version changes This patch does not serve hot reload techniques. This entails an on_replace trigger duplication if hot reload occurs. Hot reload support will be done in separate task: #87 Benchmarks - 10000000 bucket_id() calls (file test/bench_cache.lua): Baseline (no DDL): 3.38s user 0.01s system 99% cpu 3.389 total After this patch: DDL with function body: 3.81s user 0.01s system 99% cpu 3.818 total DDL with function name: 5.49s user 0.00s system 99% cpu 5.495 total Before patch: DDL with function body: 55.95s user 0.40s system 99% cpu 56.354 total DDL with function name: 13.68s user 0.13s system 99% cpu 13.807 total Closes #82
42c01d5
to
725b241
Compare
The ddl.bucket_id() function needs to know a sharding function.
It is costly to obtain the function declaration /
definition stored in the _ddl_sharding_func space.
Cache contains:
raw_tuple - raw sharding metadata, used for get_schema()
parsed_func_name - parsed dot notation (like {'foo', 'bar'})
callable - function ready to call, this offloads using of loadstring()
error - string with an error: not nil only if setting callable fails
Cache will be rebuilded if:
trigger
This patch does not serve hot reload techniques.
This entails an on_replace trigger duplication if hot reload occurs.
Hot reload support will be done in separate task:
#87
Benchmarks - 10000000 bucket_id() calls (file test/bench_cache.lua):
Baseline (no DDL): 3.38s user 0.01s system 99% cpu 3.389 total
After this patch:
DDL with function body: 3.81s user 0.01s system 99% cpu 3.818 total
DDL with function name: 5.49s user 0.00s system 99% cpu 5.495 total
Before patch:
DDL with function body: 55.95s user 0.40s system 99% cpu 56.354 total
DDL with function name: 13.68s user 0.13s system 99% cpu 13.807 total
Closes #82