forked from apache/arrow-rs
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit e25cd78
ARROW-5921: [C++] Fix multiple nullptr related crashes in IPC
Fixes the following known fuzzer crashes:
- 09f72ba2a52b80366ab676364abec850fc668168
- 607e9caa76863a97f2694a769a1ae2fb83c55e02
- cb8cedb6ff8a6f164210c497d91069812ef5d6f8
- f37e71777ad0324b55b99224f2c7ffb0107bdfa2
- fd237566879dc60fff4d956d5fe3533d74a367f3
Issue: ARROW-5921
Benchmark before:
```
Running ./release/arrow-ipc-read-write-benchmark
Run on (4 X 2700 MHz CPU s)
CPU Caches:
L1 Data 32K (x2)
L1 Instruction 32K (x2)
L2 Unified 256K (x2)
L3 Unified 3072K (x1)
-----------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------
WriteRecordBatch/1/real_time 87688 ns 87539 ns 7918 11.1368GB/s
WriteRecordBatch/4/real_time 85367 ns 85253 ns 8100 11.4396GB/s
WriteRecordBatch/16/real_time 91921 ns 91804 ns 7548 10.6239GB/s
WriteRecordBatch/64/real_time 104752 ns 104574 ns 6982 9.3226GB/s
WriteRecordBatch/256/real_time 162785 ns 162496 ns 4222 5.99908GB/s
WriteRecordBatch/1024/real_time 465386 ns 463993 ns 1485 2.09839GB/s
WriteRecordBatch/4096/real_time 1699539 ns 1693003 ns 356 588.395MB/s
WriteRecordBatch/8192/real_time 4211065 ns 4197466 ns 166 237.47MB/s
ReadRecordBatch/1/real_time 885 ns 884 ns 793175 1103.52GB/s
ReadRecordBatch/4/real_time 1658 ns 1657 ns 415874 588.825GB/s
ReadRecordBatch/16/real_time 5023 ns 5020 ns 139792 194.416GB/s
ReadRecordBatch/64/real_time 18926 ns 18911 ns 37075 51.5991GB/s
ReadRecordBatch/256/real_time 97449 ns 97351 ns 7180 10.0213GB/s
ReadRecordBatch/1024/real_time 393441 ns 393068 ns 1761 2.48211GB/s
ReadRecordBatch/4096/real_time 2271884 ns 2265855 ns 309 440.163MB/s
ReadRecordBatch/8192/real_time 4900659 ns 4886188 ns 143 204.054MB/s
```
Benchmark after:
```
Running ./release/arrow-ipc-read-write-benchmark
Run on (4 X 2700 MHz CPU s)
CPU Caches:
L1 Data 32K (x2)
L1 Instruction 32K (x2)
L2 Unified 256K (x2)
L3 Unified 3072K (x1)
-----------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------
WriteRecordBatch/1/real_time 87828 ns 87674 ns 7990 11.1191GB/s
WriteRecordBatch/4/real_time 84026 ns 83867 ns 8467 11.6222GB/s
WriteRecordBatch/16/real_time 88073 ns 87904 ns 7940 11.0881GB/s
WriteRecordBatch/64/real_time 99008 ns 98867 ns 7026 9.8635GB/s
WriteRecordBatch/256/real_time 161226 ns 160809 ns 4357 6.05712GB/s
WriteRecordBatch/1024/real_time 453623 ns 452135 ns 1385 2.15281GB/s
WriteRecordBatch/4096/real_time 1710791 ns 1704345 ns 413 584.525MB/s
WriteRecordBatch/8192/real_time 4220409 ns 4207769 ns 166 236.944MB/s
ReadRecordBatch/1/real_time 861 ns 860 ns 809986 1.10794TB/s
ReadRecordBatch/4/real_time 1664 ns 1663 ns 423113 586.955GB/s
ReadRecordBatch/16/real_time 5122 ns 5118 ns 138673 190.66GB/s
ReadRecordBatch/64/real_time 19298 ns 19286 ns 36599 50.6033GB/s
ReadRecordBatch/256/real_time 97265 ns 97182 ns 7228 10.0402GB/s
ReadRecordBatch/1024/real_time 397791 ns 397433 ns 1756 2.45497GB/s
ReadRecordBatch/4096/real_time 1677272 ns 1673565 ns 409 596.206MB/s
ReadRecordBatch/8192/real_time 4891401 ns 4876719 ns 144 204.44MB/s
```
The benchmarks are really flaky and I cannot see a large difference, so I think this is OK.
Author: Marco Neumann <[email protected]>
Closes apache#4862 from crepererum/ARROW-5921 and squashes the following commits:
127bbaf9c <Marco Neumann> update arrow-testing submodule
32070ee0b <Marco Neumann> Fix multiple nullptr related crashes in IPC1 parent eeb6f3d commit e25cd78Copy full SHA for e25cd78
1 file changed
+1
-1
lines changedtesting
Copy file name to clipboard- data/arrow-ipc/crash-09f72ba2a52b80366ab676364abec850fc668168
- data/arrow-ipc/crash-1fb75de2edd2815ad7a653684c449d814f39290e
- data/arrow-ipc/crash-607e9caa76863a97f2694a769a1ae2fb83c55e02
- data/arrow-ipc/crash-74aec871d14bb6b07c72ea8f0e8c9f72cbe6b73c
- data/arrow-ipc/crash-cb8cedb6ff8a6f164210c497d91069812ef5d6f8
- data/arrow-ipc/crash-f37e71777ad0324b55b99224f2c7ffb0107bdfa2
- data/arrow-ipc/crash-fd237566879dc60fff4d956d5fe3533d74a367f3
0 commit comments