Skip to content

Releases: ibis-project/ibis

11.0.0

15 Oct 13:11

Choose a tag to compare

11.0.0 (2025-10-15)

⚠ BREAKING CHANGES

  • clickhouse: adjust array indexing code for later versions of clickhouse
  • trino: remove deprecated password parameter (#11538)
  • datatypes: DataType.name is removed. Use DataType.__class__.__name__ instead.
  • memtables: memtables can no longer be named explicitly, please use create_table or create_view to create a named object instead.
  • api: Struct.destructure is removed in favor of Table.unpack.
  • api: String.to_date is removed in favor of String.as_date.
  • api: String.to_timestamp method is removed in favor of String.as_timestamp.
  • api: IntegerValue.to_interval is removed. Use IntegerValue.as_interval instead.
  • api: IntegerValue.to_timestamp is removed. Use IntegerValue.as_timestamp instead.
  • api: ibis.case() is removed. Use ibis.cases() instead.
  • pyspark: Type annotations using pd.Series/pd.DataFrame are now required per the non-deprecated PySpark Pandas UDF API.

Features

  • databricks: allow reads of streaming tables (#11565) (75a944b)
  • databricks: use temporary view for memtables to avoid polluting the catalog (#11562) (346504f)
  • deps: support duckdb 1.4.0 (#11622) (aacf072)

Bug Fixes

  • backends: ensure that memtable finalizers do not hold onto references to self (096efa3)
  • clickhouse: adjust array indexing code for later versions of clickhouse (a16f13f)
  • clickhouse: ensure that sqlglot does not capitalize the translate function (6fcabf5)
  • clickhouse: reraise non-unknown-table exceptions when running get_schema queries (#11520) (c95436d)
  • duckdb: allow creating tables with columns containing sql keywords (#11532) (db1a727)
  • duckdb: support duckdb 1.4.1 (f9f2363)
  • exasol: ensure back compat for exasol (93e847c)
  • flink: update key_hack SQL for array handling (71b50a2)
  • impala: compile to NOW instead of hive-default current_timestamp (563cb8e)
  • lazily import packaging so we don't depend on it in a base install (bad3ca3)
  • packaging: remove unnecessary base install requirement for packaging and cleanup packaging requirement for other backends (05397d7)
  • pyspark: only register json unwraps if they are being used (#11503) (2fc3303)
  • snowflake: allow expressions in window bounds (a82631b)
  • typing: harmonize Table.join and Table.*_join type hints (c60431e)

Documentation

  • contribute: add missing "Install just" in uv set-up guide (68e8c55)
  • disable duckdb credential chain creation (393da16)
  • improve Value.identical_to() docstring and example (7c8fcf7)

Refactors

  • api: remove deprecated IntegerValue.to_interval method (378f7ee)
  • api: remove deprecated IntegerValue.to_timestamp method (12b92a5)
  • api: remove deprecated String.to_date method (0d22310)
  • api: remove deprecated String.to_timestamp method (db4c83b)
  • api: remove deprecated Struct.destructure method (95caf6e)
  • api: remove deprecated top-level case function (ce927d7)
  • binding: make DerefMap computation lazy and support multiple value inputs (#11540) (5757caa)
  • datatypes: fully remove unused DataType.name (#11102) (8a7534c)
  • memtables: remove support for problematic memtable properties (5d35ad5)
  • split temp table support along finalizer lines (4e4a165)
  • trino: remove deprecated password parameter (#11538) (a178459)

Performance

  • athena: replace list_table_metadata with get_table_metadata in _get_schema_with_query (#11573) (40b73e7)
  • athena: use get_table_metadata in get_schema (#11574) (e4e582a)
  • binding: only create a dereference map in Table.bind when needed (#11531) (90e790c)

10.8.0

28 Jul 21:28

Choose a tag to compare

10.8.0 (2025-07-28)

Features

  • datatypes: add Schema.from_sqlglot method to produce an Ibis schema from SQLGlot (#11351) (f6641e7)
  • datatypes: allow dtype(Union[T, None]) (#11506) (c83f842)
  • polars: support is_in queries from uncorrelated tables (476353e)

Bug Fixes

  • clickhouse: return the correct type from InSubquery operations (69be180)

Documentation

  • add join method docstring templates for various join types (#11507) (4798af4)
  • improve docstrings for temporal Values (#11505) (8641087)

10.7.0

27 Jul 11:46

Choose a tag to compare

10.7.0 (2025-07-27)

Features

  • api: add Expr.to_sql() method (#11357) (493d6e5)
  • api: always prefer overwriting existing fields when unpacking with overlapping column names (#11470) (8027d51)
  • backends: kurtosis (50c3132)
  • ux: workaround for spurious display()s in jupyter (58dea60)

Bug Fixes

  • api: do not assume that dir produces attributes that exist (#11497) (1a5b1a2)
  • athena/trino/bigquery: make temporal diff robust to operand order (857758a)
  • datafusion: ensure approx quantile works (195b4f6)
  • distinct: allow deleting rows when on is unspecified (#11383) (4902ec6)
  • duckdb/snowflake: decouple Pandas and PyArrow backend converters (#11431) (0f7f7c1)
  • duckdb: disallow duckdb 1.3.1 due to hangs when loading the geospatial extension (#11409) (a67a244)
  • exasol: ensure that rounding is not subverted by sqlglot cast (3aa3d34)
  • polars: ensure that memtables can be compiled into LazyFrames by running pre-execute hooks (#11392) (7be448e)
  • postgres: make pyarrow round trip of nans and nulls higher fidelity (#11396) (60a5ba9)
  • psycopg: avoid specifying the psycopg dependency implementation (#11424) (038c339)
  • sql: ensure that CTEs as part of .sql calls are compiled in topological order (#11439) (8e227ac)
  • trino/athena: ensure that DateDiff operands are in the correct order (#11434) (e29881c)
  • trino/athena: ensure that regex extract compiler dispatch method is spelled correctly (#11436) (944fa99)

Documentation

  • improve docs for set ops (#11394) (6521a53)
  • replace: clarify types and add link to Value.substitute (#11413) (1c05d2f)
  • substitute: add to See Also section to fix rendered formatting (#11412) (d381536)
  • to_sql: fix definition package (0b5daea)
  • to_sql: fix see also reference from Value.to_sql to ibis.to_sql (7e0042c)
  • udfs: add examples of udfs with struct types as inputs (#11385) (0e8a932)

Refactors

  • rename to_sqlglot to to_sqlglot_columns_definition (#11455) (0e4cf43)
  • sqlite: move sqlite3 imports local to avoid pyodide import issues with non-sqlite backends (#11500) (b8110d3)

Performance

  • mutate: skip dereferencing cols preserved in .mutate (#11458) (70de1a1)

10.6.0

16 Jun 14:40

Choose a tag to compare

10.6.0 (2025-06-16)

Features

  • add Deferred to signatures of many Table methods (d73a040)
  • api: allow creating specific uuid values and add to docs (#11304) (476fa9f)
  • athena: support partitioning in create_table (6ea5cd4)
  • bigquery: add job_id_prefix to functions that produce bigquery jobs (#11265) (4006d68)
  • bigquery: add QueryJobConfig properties to bigquery backend specified at query time (#11255) (6d7defc)
  • bigquery: add the ability to specify a bigquery job id in a query (3a383d8)
  • bigquery: add the ability to specify a bigquery job id in a query (e0ad6ea)
  • datafusion: add BitwiseNot, Clip and IntervalFromInteger (1666e33)
  • datafusion: add Greatest and Least (10aa7d7)
  • datafusion: add support for anys and alls (#11162) (0f23dc5)
  • improve typing and docstring for to_pyarrow (2600c65)
  • make Backend._from_url() consistently use kwargs as overrides (14d103a)
  • risingwave: add support for includes in create_source (f88c0dc)
  • types: add raw_type to dt.Unknown (#11337) (9296107)
  • typing: improve type hints of .isin and .notin (#11156) (c10b627)
  • typing: return Self from .fill_null() and similar (4218644)
  • ux: improve rich rendering (#11326) (1b8038c)

Bug Fixes

  • api: deal with null literals in Struct.__getitem__ (#11299) (ccd9359)
  • bigquery: maintain custom client project id when provided (12bc4c0)
  • bigquery: raise error if bigquery project id cannot be identified (a93b7d9)
  • bigquery: undo removal of job_id_prefix functionality from raw_sql (#11349) (cf404c2)
  • bigquery: use job_id_prefix instead of job_id (364e33a)
  • builds: ensure that python 3.13 builds (601ea3e)
  • datafusion: override struct syntax to use bracket (dae8e7a)
  • datatype: convert from sqlglot VARCHAR(MAX) correctly (#11202) (dc37796)
  • duckdb: allow creating tables using data with a different column order (cf83480)
  • duckdb: support materializing enum types to pyarrow (#11214) (d2f8839)
  • flink: unit is ignored when str cast to timestamp (#11167) (cfa5768)
  • impala: make create_database follow the CanCreateDatabase API (#11149) (15ff868)
  • oracle: cast string to int for the precision and scale of decimal (969453e)
  • polars: make sure sum is NULL to match other backends (#11321) (f778f03), closes #11318
  • postgres: move more configuration to _post_connect() so that .from_connection() also uses it (15df443)
  • postgres: set search path more compliantly (1c284a1)
  • pyarrow: properly support round tripping of fixed size list types (#11330) (e2937a4)
  • snowflake: allow pyarrow 20 to work (#11173) (0ed7862)
  • snowflake: use newer sqlglot syntax for unnest (0b57361)
  • sql: ensure that when distinct with order by is split that all fields are considered for rewrites (#11334) (e4df7a8)
  • trino: fix nested row casting inside of lambda (#11146) (755e2a3)

Documentation

  • actually include expression-misc.qmd in the navbar (#11306) (4a935dc)
  • add concepts doc on casting (cea8ed8)
  • add examples of .isin() (#11154) (b4e6272)
  • databricks: add a backend documentation page (2665f53)
  • improve docstrings on methods using the database param (#11112) (458f06d)
  • postgres: show passing schema to Postgres.from_url() (46cce76)
  • support-matrix: do not escape the HTML links (895eb58)

Refactors

  • bigquery: remove unused connect function in ibis/__init__.py (#11305) (77a6dff)
  • rich: consolidate rich logic into a more central place (#11283) (44d5e6b)
  • test: centralize marks for no array, struct, etc support (#11315) (c581afe)

Performance

  • duckdb: optimize ArrayIndex sql when index is a Literal (3f32794)

10.5.0

18 Apr 20:10

Choose a tag to compare

10.5.0 (2025-04-18)

Features

  • api: expose DataType publicly at ibis.DataType (#11101) (4164265)
  • backends: fixup drop_database() when passing a specific catalog (#11110) (57d79b7)
  • databricks: add support for json via variant (a1f2bab)
  • datafusion: basic map operations (16b0c4c)
  • postgres: allow disablement of hstore extension loading (#11126) (35fc38b), closes #11103
  • postgres: implement maps in terms of JSONB instead of HSTORE (ff6100a)
  • pyflink: fix type mapping from BINARY to VARBINARY (bbcd5b9)
  • risingwave: support map operations (e7d8c3a)
  • temporal: implement casting floating values to timestamps (0356caf)

Bug Fixes

  • athena: only replace commas instead of all special characters in column names (#11138) (4c74ddd)
  • databricks: use AS JSON for programmatic output of schema information (d55a5ee)
  • duckdb: always create temp tables in temp.main (#11092) (20bec13)
  • ir: make ibis.to_sql() parse join limits and projections; fixes #11105 (a416755)
  • oracle: return a sqlglot.DataType, not str, when compiling string dtype (#11124) (99be73b)
  • postgres: clean up possible transaction hangs (6f9d141)
  • scalar-subquery: ensure that scalar subqueries value is resolvable with no underlying projection (#11125) (a92c3cb)
  • snowflake/bigquery/athena: ensure that timestamp casting works (#11135) (4bb0b78)
  • typing: remove unnecessary generics in ibis.struct() typing (#11070) (9d3aece)

Documentation

  • clarify NULL/NaN/inf distinction in docstring and examples (#11077) (3d4e741)
  • fix SECURITY.md formatting (63aeff4)

10.4.0

27 Mar 17:31

Choose a tag to compare

10.4.0 (2025-03-27)

Features

  • api: make topk() and value_counts() more flexible (#10928) (329ad7c)
  • datatypes: add string length (#11045) (61dd7ea)
  • flink: support StringSplit (#11049) (b83a88e)
  • postgres: add support for reading enum types as strings (#11028) (dfb818a)
  • postgres: implement support for asof_join API via a lateral join (#11024) (8eb9d33)
  • ux: include type of bad value in SignatureValidationError (#10988) (a706afa)

Bug Fixes

  • backends: array string join on empty array now consistently returns None across all supported backends (#10913) (a02a392)
  • polars: use engine="streaming" instead of streaming=True (e1bbe55)
  • postgres: clean up leaky cursor coming from raw_sql (#11001) (c3097a7)

Documentation

  • add introduction to pandas users for adding columns with single text values (literal) (#11010) (6032e71)
  • fix to_* methods docstrings typos (#11013) (523e4c3)
  • ir: remove extra backticks to fix formatting (7c3e5fc)

Refactors

  • postgres: use raw sql for get_schema invocation to simplify code (#11037) (9b25ab1)

Performance

  • duckdb: optimize generated SQL for ArrayConcat (#10999) (d38db7e)
  • struct: drop unused fields on Struct.getitem (#10996) (e5fe34c)

10.3.1

14 Mar 12:13

Choose a tag to compare

10.3.1 (2025-03-14)

Bug Fixes

  • datatypes: ensure that newer sqlglot BLOB type is not accessed if it does not exist (6452a5e)
  • deps: drop pytz from dependencies (#10976) (3ecf731)
  • postgres: use .transaction method instead of managing our own (0edbab9)

Documentation

  • link to our config file for conventional commits warning (#10986) (a945c96)

10.3.0

11 Mar 17:15

Choose a tag to compare

10.3.0 (2025-03-11)

Features

  • duckdb: add read_xlsx implementation (705aa16)
  • duckdb: add to_xlsx implementation (1800abd)
  • python: support python 3.9 again (#10949) (786d8f0)

Bug Fixes

  • bigquery: adjust codegen for newer version of sqlglot (8105a09)
  • memtable: ensure that constructing an empty memtable from a dataframe works (#10945) (29c796a), closes #10940
  • mysql: add regular blob to type mapping (365c157)
  • mysql: ensure that unsigned integers are mapped to the ibis type system (#10947) (3eb76f8), closes #10946
  • postgres: create server-side cursors in the iterator to ensure cursor is not cleaned up before it is used (f5f798e)
  • postgres: ensure that cursor lifetime is at least as long as the generator (f42ab45)

Documentation

  • mention theseus in the readme and docs landing page diagram (360c25b)
  • setup: add just recipe and setup entry for uv (#10959) (dd33f47)
  • tutorials: reorganize the "Getting started" tab (f78a8fb)

Refactors

  • oracle: remove unused owner column from select metadata query (#10935) (56dcc42)
  • postgres: split out hstore extension registration into separate method (0bdb5a0)

Performance

  • postgres: improve to_pyarrow_batches by using server-side cursors (#10954) (cb17b8b), closes #10938

10.2.0

03 Mar 18:27

Choose a tag to compare

10.2.0 (2025-03-03)

Features

  • examples: expand examples functionality to more backends (#10916) (90a174b)

Bug Fixes

  • bigquery: ensure that count distinct can be used in window functions without a specified window (#10911) (c9120e1)
  • build: ensure that legacy tools can recognize the validity of the Ibis license (#10905) (c29d83a)
  • datatypes: document "!" as meaning non-nullable, enable nullable arg for type hints (#10893) (601aabe)
  • examples: ensure that examples work on mssql and avoid using overwrite (#10909) (8336483)
  • pyarrow-conversion: ensure that non-nullability is preserved (#10897) (b2c5a0c)
  • snowflake: use schema when constructing pyarrow table (#10903) (8b9b143)

Documentation

  • add an example with literal in "Getting started" (#10918) (b69061b)
  • build: disable misbehaving duckdb spatial optimizer (#10898) (c37116f)
  • selectors: render the module instead of hand-picking the documented components (#10892) (d841481)

Refactors

  • duckdb: simplify loading and installation of extensions (#10900) (421972a)

Performance

10.1.0

22 Feb 14:51

Choose a tag to compare

10.1.0 (2025-02-22)

Features

  • pyspark: add partitionBy argument to create_table (c99cc23), closes #8900
  • python: allow python 3.9 installations (#10859) (fbe8c8b)

Bug Fixes

  • bigquery: allow sane use of params with raw_sql (#10874) (0a684c3)
  • deps: update dependency datafusion to v44 (979cf59)
  • deps: update dependency sqlglot to >=23.4,<26.5 (#10807) (f09e8e2)
  • deps: update dependency sqlglot to >=23.4,<26.7 (15111f8)
  • dev-tools: ensure that bump is minimal so that later release sort properly (#10878) (39729c7)
  • duckdb: use the delta extension for reading deltalake data (#10833) (beeaa29), closes #10829
  • join: error in more places on colliding column names (#10778) (ec06e1e)
  • mssql: ensure that dots in database parameter to list_tables are used as path delineators (#10863) (cdbbcb9)
  • mssql: ensure that we only escape passwords if the password is not None (e589344)
  • mysql: explicitly handle the zero integer -> timestamp case (f5e8c4f)
  • pyspark: avoid potentially different field names produced by SQL by using python-native APIs (#10877) (9538d51)
  • snowflake: use get instead of get_path; get_path does not support columns with spaces (#10836) (50c978b), closes #10835
  • sqlglot: ensure that sge.Median is only accessed when it exists (dc6b7e0)
  • sqlite: avoid generating double-quoted string literals (#10873) (76b0114)

Documentation

  • add blogpost for Athena backend (#10796) (f2f09eb)
  • add information about reading from cloud buckets (32e82c7)
  • add udf rewriting blog post (c6ecf6b)
  • blog: add post on SQL understanding and Ibis (#10762) (94425ec)
  • blog: convert case to cases in blog posts (#10560) (bbf98de)
  • blog: use more reliable URL for geospatial data (72b7673)
  • fix reference to incorrect value (1945237)
  • move __getitem__ docs so that quarto publishes them (#10870) (269cdfe)
  • release-notes: fixup release notes (fb0798e)
  • remove incorrect parameters (#10876) (a707778)
  • update post date (ea0cc95)

Refactors

  • duckdb: remove the pyarrow read_parquet fallback (5fa0103)