perf(tui): replace filter result Vec<usize> with bitmap

[higebu]

Summary

Replace the TUI filter result storage (App::filtered_indices, a Vec of packet indices) with a packed bitmap (FilterBitmap, 1 bit per packet) plus a per-block cumulative popcount index. No new dependencies.

• select(n) (n-th filtered row → packet index): O(log blocks + 8 words), used for windowed rendering via one select + cheap iteration
• rank(idx) (packet index → display row): O(1)-ish; jump-to-packet was O(n) before
• count_ones() (match count): O(1)
• Sequential scan accumulator (FilterProgress::results) and parallel scan final concatenation are converted too, since they had the same allocation problem
• Incremental update paths preserved: bg_indexer identity growth, live capture ingest (universe grows without setting bits when a filter is active), atomic swap on scan completion

Memory (100M packets, high match rate)

	before	after
storage	8 B/match ≈ 800 MB	1 bit/packet = 12.5 MB
rank/select index	—	~1.6 MB

Tests

• New unit tests in filter_bitmap.rs: equivalence against a sorted-Vec reference (densities 0–100%, block boundaries, append growth, nearest tie-breaking)
• Sequential/parallel scan result equivalence tests
• All existing tests pass; cargo test --all-targets --all-features, fmt, clippy -D warnings, doc, taplo clean

[github-actions]

Bencher Report

Branch	tui-filter-bitmap
Testbed	ubuntu-latest

Click to view all benchmark results

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
target/release/dsct read bench_input.pcap	📈 view plot 🚷 view threshold	6.22 ms (+3.16%) Baseline: 6.03 ms	6.94 ms (89.64%)
target/release/dsct stats bench_input.pcap	📈 view plot 🚷 view threshold	71.06 ms (+2.92%) Baseline: 69.05 ms	85.64 ms (82.98%)

🐰 View full continuous benchmarking report in Bencher

[codecov]

Codecov Report

❌ Patch coverage is 94.92274% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.22%. Comparing base (c912482) to head (1f19ae0).

Files with missing lines	Patch %	Lines
src/tui/filter_bitmap.rs	97.29%	10 Missing ⚠️
src/tui/app.rs	65.38%	9 Missing ⚠️
src/tui/keys.rs	71.42%	2 Missing ⚠️
src/tui/filter_apply.rs	95.00%	1 Missing ⚠️
src/tui/ui.rs	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #34      +/-   ##
==========================================
+ Coverage   91.06%   91.22%   +0.15%     
==========================================
  Files          79       80       +1     
  Lines       17678    18067     +389     
==========================================
+ Hits        16099    16481     +382     
- Misses       1579     1586       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[greptile-apps]

Greptile Summary

Replaces App::filtered_indices: Vec<usize> with FilterBitmap, a packed 1-bit-per-packet structure plus a per-block cumulative popcount index, dropping filter-result memory from 8 B/match to 1 bit/packet (≈800 MB → 12.5 MB at 100 M packets with high match rate). All call sites across app.rs, filter_apply.rs, keys.rs, stats_collect.rs, parallel_scan.rs, and the two widget files are updated to use count_ones(), select(), rank(), iter_from(), and nearest().

• FilterBitmap provides O(1) count_ones, O(log blocks + WORDS_PER_BLOCK) select/rank, and an append-only push/push_set_range/extend_universe interface; the live-capture path correctly calls extend_universe when a filter is active so the universe stays consistent.
• ParallelFilterScan assembles the final bitmap via from_sorted_indices; the sequential scan pushes strictly-increasing indices via push() and calls extend_universe(total) at finalization to cover trailing non-matches.
• The all_set(n) / push_set_range implementation sets bits one-by-one in a per-bit loop rather than filling entire u64 words, which is ~64× slower for contiguous ranges and affects file load and filter-clear latency at scale.

Confidence Score: 4/5

Safe to merge; rank/select/iter logic is algorithmically correct and well-tested, with the only notable gap being a missed word-level fill optimisation in push_set_range.

The bitmap data structure is correct throughout — block-index invariants hold, rank/select agree with the reference Vec tests, the sequential and parallel scan paths both call extend_universe(total) before finalization, and the live-capture universe-growth path handles the filter-active vs filter-inactive branches correctly. The one issue is that push_set_range (and therefore all_set) iterates bit-by-bit over contiguous ranges rather than filling entire u64 words, which means clearing a filter or loading a large file without a filter incurs O(n) bit-set operations instead of O(n/64) word writes.

src/tui/filter_bitmap.rs — specifically the push_set_range implementation for large contiguous ranges

Important Files Changed

Filename	Overview
src/tui/filter_bitmap.rs	New 603-line packed-bitmap implementation; rank/select/iter logic is correct, but push_set_range uses a per-bit loop that is suboptimal for the contiguous all-set case
src/tui/filter_apply.rs	Sequential scan pushes monotonically-increasing indices via FilterBitmap::push; finalization correctly calls extend_universe(total) to cover trailing non-matches
src/tui/parallel_scan.rs	Parallel scan concatenation builds FilterBitmap::from_sorted_indices from ordered chunk results; ScanPoll::Complete now carries a FilterBitmap; universe covers self.total
src/tui/keys.rs	Jump-to-packet now uses O(1) rank() instead of O(n) linear scan; nearest-displayed-packet uses FilterBitmap::nearest with equivalent tie-breaking semantics
src/tui/app.rs	All filtered_indices usages correctly migrated; live-capture path properly calls extend_universe when a filter is active to keep universe consistent
src/tui/stats_collect.rs	Chunked stats iteration replaced with iter_from(cursor).take(chunk_len), which is O(log blocks) to position and O(chunk) to iterate
src/tui/widgets/packet_list.rs	Prefetch and visible-packet collection migrate cleanly to iter_from; intermediate Vec retained to avoid borrow conflicts with cache

Sequence Diagram

sequenceDiagram
    participant UI
    participant App
    participant FilterBitmap
    participant ParallelScan
    participant SeqScan

    UI->>App: apply_filter()
    alt empty filter
        App->>FilterBitmap: all_set(indices.len())
        FilterBitmap-->>App: filtered (every bit set)
    else parallel-eligible filter
        App->>ParallelScan: ParallelFilterScan::new()
        loop drain()
            ParallelScan-->>App: ScanPoll::Running
        end
        ParallelScan-->>App: ScanPoll::Complete(FilterBitmap)
        App->>FilterBitmap: finalize_filter(bitmap)
    else sequential filter
        App->>SeqScan: "FilterProgress { results: FilterBitmap::new() }"
        loop filter_tick() chunks
            SeqScan->>FilterBitmap: push(matching_idx)
        end
        SeqScan->>FilterBitmap: extend_universe(total)
        App->>FilterBitmap: finalize_filter(bitmap)
    end

    UI->>App: render packet list
    App->>FilterBitmap: iter_from(offset).take(visible_rows)
    FilterBitmap-->>App: packet indices

    UI->>App: jump to packet N
    App->>FilterBitmap: contains(N-1) → rank(N-1)
    FilterBitmap-->>App: display position (O(1))

Loading

_{Reviews (1): Last reviewed commit: "perf(tui): replace filter result Vec<usi..." | Re-trigger Greptile}