Releases: ibis-project/ibis
Releases Β· ibis-project/ibis
11.0.0
11.0.0 (2025-10-15)
β BREAKING CHANGES
- clickhouse: adjust array indexing code for later versions of clickhouse
- trino: remove deprecated password parameter (#11538)
- datatypes:
DataType.name
is removed. UseDataType.__class__.__name__
instead. - memtables:
memtable
s can no longer be named explicitly, please usecreate_table
orcreate_view
to create a named object instead. - api:
Struct.destructure
is removed in favor ofTable.unpack
. - api:
String.to_date
is removed in favor ofString.as_date
. - api:
String.to_timestamp
method is removed in favor ofString.as_timestamp
. - api:
IntegerValue.to_interval
is removed. UseIntegerValue.as_interval
instead. - api:
IntegerValue.to_timestamp
is removed. UseIntegerValue.as_timestamp
instead. - api:
ibis.case()
is removed. Useibis.cases()
instead. - pyspark: Type annotations using pd.Series/pd.DataFrame are now required per the non-deprecated PySpark Pandas UDF API.
Features
- databricks: allow reads of streaming tables (#11565) (75a944b)
- databricks: use temporary view for memtables to avoid polluting the catalog (#11562) (346504f)
- deps: support duckdb 1.4.0 (#11622) (aacf072)
Bug Fixes
- backends: ensure that memtable finalizers do not hold onto references to self (096efa3)
- clickhouse: adjust array indexing code for later versions of clickhouse (a16f13f)
- clickhouse: ensure that sqlglot does not capitalize the
translate
function (6fcabf5) - clickhouse: reraise non-unknown-table exceptions when running
get_schema
queries (#11520) (c95436d) - duckdb: allow creating tables with columns containing sql keywords (#11532) (db1a727)
- duckdb: support duckdb 1.4.1 (f9f2363)
- exasol: ensure back compat for exasol (93e847c)
- flink: update key_hack SQL for array handling (71b50a2)
- impala: compile to NOW instead of hive-default current_timestamp (563cb8e)
- lazily import packaging so we don't depend on it in a base install (bad3ca3)
- packaging: remove unnecessary base install requirement for
packaging
and cleanup packaging requirement for other backends (05397d7) - pyspark: only register json unwraps if they are being used (#11503) (2fc3303)
- snowflake: allow expressions in window bounds (a82631b)
- typing: harmonize Table.join and Table.*_join type hints (c60431e)
Documentation
- contribute: add missing "Install just" in uv set-up guide (68e8c55)
- disable duckdb credential chain creation (393da16)
- improve Value.identical_to() docstring and example (7c8fcf7)
Refactors
- api: remove deprecated
IntegerValue.to_interval
method (378f7ee) - api: remove deprecated
IntegerValue.to_timestamp
method (12b92a5) - api: remove deprecated
String.to_date
method (0d22310) - api: remove deprecated
String.to_timestamp
method (db4c83b) - api: remove deprecated
Struct.destructure
method (95caf6e) - api: remove deprecated top-level
case
function (ce927d7) - binding: make DerefMap computation lazy and support multiple value inputs (#11540) (5757caa)
- datatypes: fully remove unused DataType.name (#11102) (8a7534c)
- memtables: remove support for problematic memtable properties (5d35ad5)
- split temp table support along finalizer lines (4e4a165)
- trino: remove deprecated password parameter (#11538) (a178459)
Performance
10.8.0
10.8.0 (2025-07-28)
Features
- datatypes: add
Schema.from_sqlglot
method to produce an Ibis schema from SQLGlot (#11351) (f6641e7) - datatypes: allow dtype(Union[T, None]) (#11506) (c83f842)
- polars: support is_in queries from uncorrelated tables (476353e)
Bug Fixes
- clickhouse: return the correct type from
InSubquery
operations (69be180)
Documentation
10.7.0
10.7.0 (2025-07-27)
Features
- api: add
Expr.to_sql()
method (#11357) (493d6e5) - api: always prefer overwriting existing fields when unpacking with overlapping column names (#11470) (8027d51)
- backends: kurtosis (50c3132)
- ux: workaround for spurious display()s in jupyter (58dea60)
Bug Fixes
- api: do not assume that dir produces attributes that exist (#11497) (1a5b1a2)
- athena/trino/bigquery: make temporal diff robust to operand order (857758a)
- datafusion: ensure approx quantile works (195b4f6)
- distinct: allow deleting rows when
on
is unspecified (#11383) (4902ec6) - duckdb/snowflake: decouple Pandas and PyArrow backend converters (#11431) (0f7f7c1)
- duckdb: disallow duckdb 1.3.1 due to hangs when loading the geospatial extension (#11409) (a67a244)
- exasol: ensure that rounding is not subverted by sqlglot cast (3aa3d34)
- polars: ensure that memtables can be compiled into LazyFrames by running pre-execute hooks (#11392) (7be448e)
- postgres: make pyarrow round trip of nans and nulls higher fidelity (#11396) (60a5ba9)
- psycopg: avoid specifying the psycopg dependency implementation (#11424) (038c339)
- sql: ensure that CTEs as part of
.sql
calls are compiled in topological order (#11439) (8e227ac) - trino/athena: ensure that
DateDiff
operands are in the correct order (#11434) (e29881c) - trino/athena: ensure that regex extract compiler dispatch method is spelled correctly (#11436) (944fa99)
Documentation
- improve docs for set ops (#11394) (6521a53)
- replace: clarify types and add link to
Value.substitute
(#11413) (1c05d2f) - substitute: add to
See Also
section to fix rendered formatting (#11412) (d381536) - to_sql: fix definition package (0b5daea)
- to_sql: fix see also reference from
Value.to_sql
toibis.to_sql
(7e0042c) - udfs: add examples of udfs with struct types as inputs (#11385) (0e8a932)
Refactors
- rename
to_sqlglot
toto_sqlglot_columns_definition
(#11455) (0e4cf43) - sqlite: move sqlite3 imports local to avoid pyodide import issues with non-sqlite backends (#11500) (b8110d3)
Performance
10.6.0
10.6.0 (2025-06-16)
Features
- add Deferred to signatures of many Table methods (d73a040)
- api: allow creating specific uuid values and add to docs (#11304) (476fa9f)
- athena: support partitioning in
create_table
(6ea5cd4) - bigquery: add job_id_prefix to functions that produce bigquery jobs (#11265) (4006d68)
- bigquery: add QueryJobConfig properties to bigquery backend specified at query time (#11255) (6d7defc)
- bigquery: add the ability to specify a bigquery job id in a query (3a383d8)
- bigquery: add the ability to specify a bigquery job id in a query (e0ad6ea)
- datafusion: add BitwiseNot, Clip and IntervalFromInteger (1666e33)
- datafusion: add Greatest and Least (10aa7d7)
- datafusion: add support for anys and alls (#11162) (0f23dc5)
- improve typing and docstring for to_pyarrow (2600c65)
- make Backend._from_url() consistently use kwargs as overrides (14d103a)
- risingwave: add support for
includes
increate_source
(f88c0dc) - types: add
raw_type
to dt.Unknown (#11337) (9296107) - typing: improve type hints of .isin and .notin (#11156) (c10b627)
- typing: return Self from .fill_null() and similar (4218644)
- ux: improve rich rendering (#11326) (1b8038c)
Bug Fixes
- api: deal with null literals in
Struct.__getitem__
(#11299) (ccd9359) - bigquery: maintain custom client project id when provided (12bc4c0)
- bigquery: raise error if bigquery project id cannot be identified (a93b7d9)
- bigquery: undo removal of
job_id_prefix
functionality fromraw_sql
(#11349) (cf404c2) - bigquery: use job_id_prefix instead of job_id (364e33a)
- builds: ensure that python 3.13 builds (601ea3e)
- datafusion: override struct syntax to use bracket (dae8e7a)
- datatype: convert from sqlglot VARCHAR(MAX) correctly (#11202) (dc37796)
- duckdb: allow creating tables using data with a different column order (cf83480)
- duckdb: support materializing enum types to pyarrow (#11214) (d2f8839)
- flink: unit is ignored when str cast to timestamp (#11167) (cfa5768)
- impala: make create_database follow the CanCreateDatabase API (#11149) (15ff868)
- oracle: cast string to int for the precision and scale of decimal (969453e)
- polars: make sure sum is NULL to match other backends (#11321) (f778f03), closes #11318
- postgres: move more configuration to _post_connect() so that .from_connection() also uses it (15df443)
- postgres: set search path more compliantly (1c284a1)
- pyarrow: properly support round tripping of fixed size list types (#11330) (e2937a4)
- snowflake: allow pyarrow 20 to work (#11173) (0ed7862)
- snowflake: use newer sqlglot syntax for unnest (0b57361)
- sql: ensure that when distinct with order by is split that all fields are considered for rewrites (#11334) (e4df7a8)
- trino: fix nested row casting inside of lambda (#11146) (755e2a3)
Documentation
- actually include expression-misc.qmd in the navbar (#11306) (4a935dc)
- add concepts doc on casting (cea8ed8)
- add examples of .isin() (#11154) (b4e6272)
- databricks: add a backend documentation page (2665f53)
- improve docstrings on methods using the
database
param (#11112) (458f06d) - postgres: show passing schema to Postgres.from_url() (46cce76)
- support-matrix: do not escape the HTML links (895eb58)
Refactors
- bigquery: remove unused
connect
function inibis/__init__.py
(#11305) (77a6dff) - rich: consolidate rich logic into a more central place (#11283) (44d5e6b)
- test: centralize marks for no array, struct, etc support (#11315) (c581afe)
Performance
- duckdb: optimize ArrayIndex sql when index is a Literal (3f32794)
10.5.0
10.5.0 (2025-04-18)
Features
- api: expose DataType publicly at ibis.DataType (#11101) (4164265)
- backends: fixup drop_database() when passing a specific catalog (#11110) (57d79b7)
- databricks: add support for json via variant (a1f2bab)
- datafusion: basic map operations (16b0c4c)
- postgres: allow disablement of hstore extension loading (#11126) (35fc38b), closes #11103
- postgres: implement maps in terms of JSONB instead of HSTORE (ff6100a)
- pyflink: fix type mapping from BINARY to VARBINARY (bbcd5b9)
- risingwave: support map operations (e7d8c3a)
- temporal: implement casting floating values to timestamps (0356caf)
Bug Fixes
- athena: only replace commas instead of all special characters in column names (#11138) (4c74ddd)
- databricks: use
AS JSON
for programmatic output of schema information (d55a5ee) - duckdb: always create temp tables in
temp.main
(#11092) (20bec13) - ir: make ibis.to_sql() parse join limits and projections; fixes #11105 (a416755)
- oracle: return a
sqlglot.DataType
, notstr
, when compiling string dtype (#11124) (99be73b) - postgres: clean up possible transaction hangs (6f9d141)
- scalar-subquery: ensure that scalar subqueries
value
is resolvable with no underlying projection (#11125) (a92c3cb) - snowflake/bigquery/athena: ensure that timestamp casting works (#11135) (4bb0b78)
- typing: remove unnecessary generics in
ibis.struct()
typing (#11070) (9d3aece)
Documentation
10.4.0
10.4.0 (2025-03-27)
Features
- api: make topk() and value_counts() more flexible (#10928) (329ad7c)
- datatypes: add string length (#11045) (61dd7ea)
- flink: support
StringSplit
(#11049) (b83a88e) - postgres: add support for reading enum types as strings (#11028) (dfb818a)
- postgres: implement support for
asof_join
API via a lateral join (#11024) (8eb9d33) - ux: include type of bad value in SignatureValidationError (#10988) (a706afa)
Bug Fixes
- backends: array string join on empty array now consistently returns
None
across all supported backends (#10913) (a02a392) - polars: use engine="streaming" instead of streaming=True (e1bbe55)
- postgres: clean up leaky cursor coming from
raw_sql
(#11001) (c3097a7)
Documentation
- add introduction to pandas users for adding columns with single text values (literal) (#11010) (6032e71)
- fix to_* methods docstrings typos (#11013) (523e4c3)
- ir: remove extra backticks to fix formatting (7c3e5fc)
Refactors
Performance
10.3.1
10.3.0
10.3.0 (2025-03-11)
Features
- duckdb: add
read_xlsx
implementation (705aa16) - duckdb: add
to_xlsx
implementation (1800abd) - python: support python 3.9 again (#10949) (786d8f0)
Bug Fixes
- bigquery: adjust codegen for newer version of sqlglot (8105a09)
- memtable: ensure that constructing an empty memtable from a dataframe works (#10945) (29c796a), closes #10940
- mysql: add regular blob to type mapping (365c157)
- mysql: ensure that unsigned integers are mapped to the ibis type system (#10947) (3eb76f8), closes #10946
- postgres: create server-side cursors in the iterator to ensure cursor is not cleaned up before it is used (f5f798e)
- postgres: ensure that cursor lifetime is at least as long as the generator (f42ab45)
Documentation
- mention theseus in the readme and docs landing page diagram (360c25b)
- setup: add
just
recipe and setup entry foruv
(#10959) (dd33f47) - tutorials: reorganize the "Getting started" tab (f78a8fb)
Refactors
- oracle: remove unused
owner
column fromselect
metadata query (#10935) (56dcc42) - postgres: split out
hstore
extension registration into separate method (0bdb5a0)
Performance
10.2.0
10.2.0 (2025-03-03)
Features
Bug Fixes
- bigquery: ensure that count distinct can be used in window functions without a specified window (#10911) (c9120e1)
- build: ensure that legacy tools can recognize the validity of the Ibis license (#10905) (c29d83a)
- datatypes: document "!" as meaning non-nullable, enable
nullable
arg for type hints (#10893) (601aabe) - examples: ensure that examples work on mssql and avoid using overwrite (#10909) (8336483)
- pyarrow-conversion: ensure that non-nullability is preserved (#10897) (b2c5a0c)
- snowflake: use schema when constructing pyarrow table (#10903) (8b9b143)
Documentation
- add an example with
literal
in "Getting started" (#10918) (b69061b) - build: disable misbehaving duckdb spatial optimizer (#10898) (c37116f)
- selectors: render the module instead of hand-picking the documented components (#10892) (d841481)
Refactors
Performance
10.1.0
10.1.0 (2025-02-22)
Features
- pyspark: add partitionBy argument to create_table (c99cc23), closes #8900
- python: allow python 3.9 installations (#10859) (fbe8c8b)
Bug Fixes
- bigquery: allow sane use of
params
withraw_sql
(#10874) (0a684c3) - deps: update dependency datafusion to v44 (979cf59)
- deps: update dependency sqlglot to >=23.4,<26.5 (#10807) (f09e8e2)
- deps: update dependency sqlglot to >=23.4,<26.7 (15111f8)
- dev-tools: ensure that bump is minimal so that later release sort properly (#10878) (39729c7)
- duckdb: use the
delta
extension for reading deltalake data (#10833) (beeaa29), closes #10829 - join: error in more places on colliding column names (#10778) (ec06e1e)
- mssql: ensure that dots in
database
parameter tolist_tables
are used as path delineators (#10863) (cdbbcb9) - mssql: ensure that we only escape passwords if the password is not
None
(e589344) - mysql: explicitly handle the zero integer -> timestamp case (f5e8c4f)
- pyspark: avoid potentially different field names produced by SQL by using python-native APIs (#10877) (9538d51)
- snowflake: use
get
instead ofget_path
;get_path
does not support columns with spaces (#10836) (50c978b), closes #10835 - sqlglot: ensure that
sge.Median
is only accessed when it exists (dc6b7e0) - sqlite: avoid generating double-quoted string literals (#10873) (76b0114)
Documentation
- add blogpost for Athena backend (#10796) (f2f09eb)
- add information about reading from cloud buckets (32e82c7)
- add udf rewriting blog post (c6ecf6b)
- blog: add post on SQL understanding and Ibis (#10762) (94425ec)
- blog: convert case to cases in blog posts (#10560) (bbf98de)
- blog: use more reliable URL for geospatial data (72b7673)
- fix reference to incorrect value (1945237)
- move
__getitem__
docs so that quarto publishes them (#10870) (269cdfe) - release-notes: fixup release notes (fb0798e)
- remove incorrect parameters (#10876) (a707778)
- update post date (ea0cc95)
Refactors
- duckdb: remove the pyarrow
read_parquet
fallback (5fa0103)