[DO NOT MERGE] Remove unnecessary synchronization (miss-sync) during Parquet reading

[JigaoLuo]

Description

For the issue #18967, this is a PR Draft aimed at removing all unnecessary synchronization points (termed "miss-sync") in the Parquet reader. Please hold off on merging this PR draft. The plan is to split it into smaller PRs for the actual merge.

TL;DR: This is the performance gain in scalability 🚀 once the future small PRs from this draft are merged:

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[JigaoLuo]

Hi @vuule @mhaseeb123 @wence- @GregoryKimball,
Thanks for the review and discussion! I think there is still a need for discussion. After that, I’ll address the feedback in separate PRs after we finalize the details. I’ll definitely make time to merge this in my free time.

I’ll also try to explore performance optimizations in benchmarks. [Update: you can find the benchmark in the following message]

[JigaoLuo]

Hi @vuule @mhaseeb123 @wence- @GregoryKimball,
I’ve run preliminary benchmarks, and as expected, the PR draft shows significant 🚀 gains in scenarios with frequent synchronization stalls (“miss-sync”). This is particularly evident when each thread reads multiple small-sized, concurrent file segments: a use case, for example, each thread reading different row groups from a large Parquet file. If I can convince you with this use case, then we are one step closer to a SpeedOfLight Parquet reader.

I plotted my benchmark as the style from Greg's comment #15620 (comment) . The PR draft is fully compilable, so feel free to test it when you have time.

You can also find my simple command and details here:

How many miss-sync are saved?

You will see where this speedup comes from and how nasty the miss-sync can be:

Upstream

$ $ nsys export --output report_upstream_8threads.sqlite --type sqlite report_upstream_8threads.nsys-rep 
$  nsys analyze -r cuda_memcpy_async:rows=-1 report_upstream_8threads.sqlite | wc -l
166861

This PR

$ nsys analyze -r cuda_memcpy_async:rows=-1 report_pr_8threads.sqlite | wc -l
301

And all those miss-sync are coming from the Parquet writer. No pageable memcpy exists in the Parquet reader with this patch.

Command

With num_iterations=10, I let each thread read 10 times of size 128MB to mimic the use case and also to create more miss-sync.

for t in 1 2 4 8 16 32 64 128; 
do 
  ./PARQUET_MULTITHREAD_READER_NVBENCH -d 0 -b 0 --axis num_cols=32 --axis run_length=2 --axis total_data_size=$((1024 * 1024 * 128 * t)) --axis num_threads=$t --axis num_iterations=10 --csv <PATH>;
done

And I use this same command on this PR draft as well as on the upstream branch 25-08 to generate two sets of CSV files.

Hardware setup

RMM memory resource = pool
CUIO host memory resource = pinned_pool
# Devices

## [0] `NVIDIA A100-SXM4-40GB`
* SM Version: 800 (PTX Version: 800)
* Number of SMs: 108
* SM Default Clock Rate: 1410 MHz
* Global Memory: 19704 MiB Free / 40339 MiB Total
* Global Memory Bus Peak: 1555 GB/sec (5120-bit DDR @1215MHz)
* Max Shared Memory: 164 KiB/SM, 48 KiB/Block
* L2 Cache Size: 40960 KiB
* Maximum Active Blocks: 32/SM
* Maximum Active Threads: 2048/SM, 1024/Block
* Available Registers: 65536/SM, 65536/Block
* ECC Enabled: Yes

[JigaoLuo]

Host to Host copy (with make_pinned_vector_async)

[JigaoLuo]

Host to Host copy

[JigaoLuo]

Note: There may be a bug in this draft or somewhere in the codebase. I ran millions of read operations over a 10-hour period—purely reading—and encountered a single instance of incorrect results, with an estimated trigger rate of just 0.0001%. I can not be sure where the bug is: is it on my draft or in the cudf code (I am using branch-25.08 still).

I’m leaving this note here as a reminder for myself, and we can consider adding more checks during the merging for the rest of this draft.

[Selfnote Update] I’ve observed the same bug even without the patch. The reproduction now seems tied to my metadata caching PR. I’ll allocate time to reproduce it more reliably.