IN LIST: add branchless filter for small primitive lists

[geoffreyclaude]

Which issue does this PR close?

• Part of Further improve performance of IN list evaluation #19241.
• Stacked on IN LIST: reinterpret small-width types for bitmap filters #23013.
• Next in stack: IN LIST: add direct-probe hash filter for large primitive lists #23015.
• Extracted from Optimize IN performance with specialized implementations #19390.

Rationale for this change

For very small IN lists, building or probing a hash table can be more work than just comparing the input value with each constant.

For example, for x IN (10, 20, 30), the fast path can behave like:

x == 10 OR x == 20 OR x == 30

Because the list is tiny, those comparisons are cheap. The implementation stores the constants in a fixed-size array and checks them with a compact comparison chain.

“Branchless” here means the comparisons are combined without stopping at the first match. That can be faster for these small fixed-width lists because the CPU gets a predictable sequence of simple operations instead of hash-table setup and probe logic.

Larger primitive lists are left for #23015, where a purpose-built hash lookup becomes the better tradeoff.

For 1-byte and 2-byte primitives, this PR uses branchless only for the short lists where a few direct comparisons can avoid the setup cost of building a bitmap. The cutoffs are intentionally modest: up to 16 values for 1-byte types, and up to 8 values for 2-byte types. Beyond that, the comparison sequence grows quickly and the bitmap path remains the better default.

What changes are included in this PR?

• Adds a const-generic BranchlessFilter for small primitive IN lists.
• Adds thresholds for when this path is used:
  • up to 16 values for 1-byte types
  • up to 8 values for 2-byte types
  • up to 32 values for 4-byte types
  • up to 16 values for 8-byte types
  • up to 4 values for 16-byte types
• Reuses the zero-copy reinterpretation approach from IN LIST: reinterpret small-width types for bitmap filters #23013 for compatible same-width primitive types.
• Keeps the same IN / NOT IN null behavior as the rest of the stack.
• Adds focused coverage for sliced reinterpreted arrays.

Are these changes tested?

Yes.

• cargo fmt --all --check
• cargo test -p datafusion-physical-expr reinterpreted_ --lib
• cargo test -p datafusion-physical-expr in_list_int_types --lib
• cargo test -p datafusion-physical-expr in_list_float64 --lib
• cargo test -p datafusion-physical-expr in_list_decimal --lib
• cargo test -p datafusion-physical-expr test_in_list_from_array_type_combinations --lib
• cargo clippy -p datafusion-physical-expr --all-targets --all-features -- -D warnings

Are there any user-facing changes?

No. This is an internal performance optimization only.

Local benchmark snapshot

Benchmark command:

cargo bench -p datafusion-physical-expr --profile release-nonlto --bench in_list_strategy -- --save-baseline <name>

Method: compare adjacent saved baselines using raw Criterion sample minima (min(time / iters)). Lower is better; changes within +/-5% are treated as noise.

Compared baselines: #23013 -> #23014

Relevant scope: small primitive-list rows.

Summary: 27 relevant rows, 24 faster, 0 slower, 3 within +/-5%.

Largest relevant deltas:

Benchmark	Before	After	Change
timestamp_ns/small_list/list=4/match=50%	47.55 us	3.25 us	-93.2% (14.63x faster)
f32/small_list/list=4/match=50%	33.79 us	3.04 us	-91.0% (11.12x faster)
primitive/i32/small_list/list=4/match=50%	31.84 us	3.04 us	-90.5% (10.48x faster)
primitive/i64/small_list/list=4/match=50%	28.43 us	3.20 us	-88.7% (8.87x faster)
f32/small_list/list=4/match=0%	18.97 us	3.00 us	-84.2% (6.32x faster)
timestamp_ns/small_list/list=4/match=0%	19.48 us	3.27 us	-83.2% (5.96x faster)
primitive/i32/small_list/list=4/match=0%	17.19 us	3.03 us	-82.4% (5.67x faster)
primitive/i64/small_list/list=4/match=0%	16.63 us	3.21 us	-80.7% (5.18x faster)
primitive/i32/small_list/list=16/match=50%/NOT_IN	33.19 us	7.29 us	-78.0% (4.56x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%	30.33 us	7.19 us	-76.3% (4.22x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%/NOT_IN	30.02 us	7.30 us	-75.7% (4.11x faster)
timestamp_ns/small_list/list=16/match=50%	42.51 us	11.81 us	-72.2% (3.60x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=50%	21.36 us	7.19 us	-66.4% (2.97x faster)
f32/small_list/list=32/match=50%	38.89 us	13.20 us	-66.1% (2.95x faster)
primitive/i64/small_list/list=16/match=50%	30.68 us	12.12 us	-60.5% (2.53x faster)

Full relevant table (27 rows)

Benchmark	Before	After	Change
f32/small_list/list=32/match=0%	21.56 us	13.33 us	-38.2% (1.62x faster)
f32/small_list/list=32/match=50%	38.89 us	13.20 us	-66.1% (2.95x faster)
f32/small_list/list=4/match=0%	18.97 us	3.00 us	-84.2% (6.32x faster)
f32/small_list/list=4/match=50%	33.79 us	3.04 us	-91.0% (11.12x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%	30.33 us	7.19 us	-76.3% (4.22x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=20%/NOT_IN	30.02 us	7.30 us	-75.7% (4.11x faster)
nulls/primitive/i32/small_list/list=16/match=50%/nulls=50%	21.36 us	7.19 us	-66.4% (2.97x faster)
primitive/i32/small_list/list=16/match=50%/NOT_IN	33.19 us	7.29 us	-78.0% (4.56x faster)
primitive/i32/small_list/list=32/match=0%	17.11 us	13.33 us	-22.1% (1.28x faster)
primitive/i32/small_list/list=32/match=50%	29.63 us	13.34 us	-55.0% (2.22x faster)
primitive/i32/small_list/list=4/match=0%	17.19 us	3.03 us	-82.4% (5.67x faster)
primitive/i32/small_list/list=4/match=50%	31.84 us	3.04 us	-90.5% (10.48x faster)
primitive/i64/small_list/list=16/match=0%	17.16 us	11.91 us	-30.6% (1.44x faster)
primitive/i64/small_list/list=16/match=50%	30.68 us	12.12 us	-60.5% (2.53x faster)
primitive/i64/small_list/list=4/match=0%	16.63 us	3.21 us	-80.7% (5.18x faster)
primitive/i64/small_list/list=4/match=50%	28.43 us	3.20 us	-88.7% (8.87x faster)
timestamp_ns/small_list/list=16/match=0%	19.59 us	11.79 us	-39.8% (1.66x faster)
timestamp_ns/small_list/list=16/match=50%	42.51 us	11.81 us	-72.2% (3.60x faster)
timestamp_ns/small_list/list=4/match=0%	19.48 us	3.27 us	-83.2% (5.96x faster)
timestamp_ns/small_list/list=4/match=50%	47.55 us	3.25 us	-93.2% (14.63x faster)
narrow_integer/u8/list=4/match=0%	3.88 us	2.75 us	-29.0% (1.41x faster)
narrow_integer/u8/list=4/match=50%	3.84 us	2.76 us	-28.2% (1.39x faster)
narrow_integer/u8/list=16/match=0%	3.93 us	3.86 us	-1.7% (within noise)
narrow_integer/u8/list=16/match=50%	3.85 us	3.88 us	+0.8% (within noise)
narrow_integer/i16/list=4/match=0%	3.86 us	3.16 us	-18.1% (1.22x faster)
narrow_integer/i16/list=4/match=50%	3.90 us	3.15 us	-19.2% (1.24x faster)
nulls/narrow_integer/u8/list=16/match=50%/nulls=20%	3.85 us	3.94 us	+2.3% (within noise)