-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comparison with Kryo IO #1
Comments
I haven't run any benchmarks against Kryo and hadn't looked at it before, but looking through ByteBufferInput now, I see several patterns that I avoided for performance reasons. Someone should benchmark it :-) I expect it to be much faster than DataInputStream/BufferedInputStream but still significantly slower than perfio.BufferedInput (within a factor of 2). I did run some tests with unsafe array access for perfIO but the results were not compelling. This is certainly worth re-examining for various use cases. With tricks in the range checks to elide unnecessary double checks in many cases, and the VarHandle based array accessor, the results so far didn't justify additional unsafe code paths. |
I've added Kryo to BufferedInputNumBenchmark and BufferedOutputNumBenchmark. Even the unsafe version turned out to be more than a factor of 2 slower than perfIO.
I also ran some more tests with Unsafe for both input and output, but it's slower than the current safe VarHandle-based access. |
Oops, I didn't realize that the regular
This matches my original expectation: Kryo is significantly slower, but within a factor of 2. |
Huh, the growing output is actually a bit faster with Kryo's unsafe implementation. (The numbers above aren't great; it takes too long for HotSpot optimization to stabilize, but more warmup leads to the same results.) This is unexpected since the flushing implementation is not overridden in UnsafeOutput and the preallocated UnsafeOutput version is actually slower than the safe one. And the difference is even larger when flushing to a FileOutputStream. I suppose I need to run some more experiments with Unsafe specifically for these use cases. The previous tests were all writing to a preallocated buffer (because that produces the most consistent results). |
I'm reopening this ticket since there's some opportunity for performance improvement based on comparison with Kryo's I've identified several factors that result in lower performance compared to Kryo's
|
I benchmarked KryoUnsafe on Java 8 several years ago and found it to be quite fast. It's great to see that it remains competitive, and even better to know there's still room for improvement. perfio is already the fastest and will become even faster with these changes. |
My assessment last night may have been too optimistic. I got a stripped-down version of BufferedOutput down to the same speed as Kryo's UnsafeOutput in this specific benchmark (where it was slower than Kryo before), but I'm not having much luck doing it with the full-featured version so far. Using Unsafe does result in a higher speed on this micro-benchmark, but the other changes are not helping. They probably rely on inlining of the outer buffering loop that we're not getting with the full version. In particular, the clever range check is still much better overall than the direct one, even with unsafe array access. Only a minimized micro-benchmark showed a better performance with the direct version. |
How does the performance of this library compare to the IO classes of Kryo?
Like
https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/io/ByteBufferInput.java
or the older fast Inputs:
https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/unsafe/UnsafeByteBufferInput.java
The text was updated successfully, but these errors were encountered: