Skip to content

A RuntimeError (Size mismatch between request values and groupby keys) occurs when using the GPU engine #25575

@asddfl

Description

@asddfl

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

ctx = pl.SQLContext()

t1 = pl.LazyFrame(
    {
        "c0": [True, False],
    }
)
ctx.register("t1", t1)

result = t1.group_by("c0").agg(pl.lit(1).is_not_null())
print("CPU Result:")
print(result.collect())

print("GPU Result:")
print(result.collect(engine=pl.GPUEngine(raise_on_fail=True)))

Log output

CPU Result:
shape: (2, 2)
┌───────┬─────────┐
│ c0    ┆ literal │
│ ---   ┆ ---     │
│ bool  ┆ bool    │
╞═══════╪═════════╡
│ falsetrue    │
│ truetrue    │
└───────┴─────────┘
GPU Result:
Traceback (most recent call last):
  File "/xxx/bug.py", line 19, in <module>
    print(result.collect(engine=pl.GPUEngine(raise_on_fail=True)))
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/polars/_utils/deprecation.py", line 97, in wrapper
    return function(*args, **kwargs)
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/polars/lazyframe/opt_flags.py", line 328, in wrapper
    return function(*args, **kwargs)
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/polars/lazyframe/frame.py", line 2415, in collect
    return wrap_df(ldf.collect(engine, callback))
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/polars/_utils/scan.py", line 27, in _execute_from_rust
    return function(with_columns, *args)
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/cudf_polars/callback.py", line 263, in _callback
    return evaluate_streaming(ir, config_options)
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/cudf_polars/experimental/parallel.py", line 287, in evaluate_streaming
    return get_scheduler(config_options)(graph, key).to_polars()
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/cudf_polars/experimental/scheduler.py", line 149, in synchronous_scheduler
    cache[k] = _execute_task(graph[k], cache)
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/cudf_polars/experimental/scheduler.py", line 51, in _execute_task
    return arg[0](*(_execute_task(a, cache) for a in arg[1:]))
  File "/xxx/anaconda3/envs/polars/lib/python3.10/site-packages/cudf_polars/dsl/ir.py", line 1814, in do_evaluate
    group_keys, raw_tables = grouper.aggregate(requests, stream=df.stream)
  File "pylibcudf/groupby.pyx", line 163, in pylibcudf.groupby.GroupBy.aggregate
  File "pylibcudf/groupby.pyx", line 198, in pylibcudf.groupby.GroupBy.aggregate
RuntimeError: CUDF failure at: /tmp/conda-bld-output/bld/rattler-build_libcudf/work/cpp/src/groupby/groupby.cu:228: Size mismatch between request values and groupby keys.

Issue description

A RuntimeError will occur when combining .agg() with .is_not_null() and using the GPU engine.

Expected behavior

No error will occur when using the GPU engine.
The output is consistent when using the CPU and GPU engines.

Installed versions

--------Version info---------
Polars:              1.34.0
Index type:          UInt32
Platform:            Linux-6.14.0-35-generic-x86_64-with-glibc2.39
Python:              3.10.19 (main, Oct 21 2025, 16:43:05) [GCC 11.2.0]
Runtime:             rt32

----Optional dependencies----
Azure CLI            <not installed>
adbc_driver_manager  <not installed>
altair               <not installed>
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               2025.10.0
gevent               <not installed>
google.auth          <not installed>
great_tables         <not installed>
matplotlib           <not installed>
numpy                1.26.4
openpyxl             <not installed>
pandas               2.3.3
polars_cloud         <not installed>
pyarrow              <not installed>
pydantic             <not installed>
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>
cuda-bindings     12.9.4
cuda-pathfinder   1.3.2
cuda-python       12.9.4
cudf-polars       26.2.0a104
nvidia-ml-py      13.580.82
nvtx              0.2.13

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-gpuArea: gpu enginebugSomething isn't workingneeds triageAwaiting prioritization by a maintainerpythonRelated to Python Polars

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions