11.0.0

11.0.0 (2025-10-15)

⚠ BREAKING CHANGES

• clickhouse: adjust array indexing code for later versions of clickhouse
• trino: remove deprecated password parameter (#11538)
• datatypes: DataType.name is removed. Use DataType.__class__.__name__ instead.
• memtables: memtables can no longer be named explicitly, please use create_table or create_view to create a named object instead.
• api: Struct.destructure is removed in favor of Table.unpack.
• api: String.to_date is removed in favor of String.as_date.
• api: String.to_timestamp method is removed in favor of String.as_timestamp.
• api: IntegerValue.to_interval is removed. Use IntegerValue.as_interval instead.
• api: IntegerValue.to_timestamp is removed. Use IntegerValue.as_timestamp instead.
• api: ibis.case() is removed. Use ibis.cases() instead.
• pyspark: Type annotations using pd.Series/pd.DataFrame are now required per the non-deprecated PySpark Pandas UDF API.

Features

• databricks: allow reads of streaming tables (#11565) (75a944b)
• databricks: use temporary view for memtables to avoid polluting the catalog (#11562) (346504f)
• deps: support duckdb 1.4.0 (#11622) (aacf072)

Bug Fixes

• backends: ensure that memtable finalizers do not hold onto references to self (096efa3)
• clickhouse: adjust array indexing code for later versions of clickhouse (a16f13f)
• clickhouse: ensure that sqlglot does not capitalize the translate function (6fcabf5)
• clickhouse: reraise non-unknown-table exceptions when running get_schema queries (#11520) (c95436d)
• duckdb: allow creating tables with columns containing sql keywords (#11532) (db1a727)
• duckdb: support duckdb 1.4.1 (f9f2363)
• exasol: ensure back compat for exasol (93e847c)
• flink: update key_hack SQL for array handling (71b50a2)
• impala: compile to NOW instead of hive-default current_timestamp (563cb8e)
• lazily import packaging so we don't depend on it in a base install (bad3ca3)
• packaging: remove unnecessary base install requirement for packaging and cleanup packaging requirement for other backends (05397d7)
• pyspark: only register json unwraps if they are being used (#11503) (2fc3303)
• snowflake: allow expressions in window bounds (a82631b)
• typing: harmonize Table.join and Table.*_join type hints (c60431e)

Documentation

• contribute: add missing "Install just" in uv set-up guide (68e8c55)
• disable duckdb credential chain creation (393da16)
• improve Value.identical_to() docstring and example (7c8fcf7)

Refactors

• api: remove deprecated IntegerValue.to_interval method (378f7ee)
• api: remove deprecated IntegerValue.to_timestamp method (12b92a5)
• api: remove deprecated String.to_date method (0d22310)
• api: remove deprecated String.to_timestamp method (db4c83b)
• api: remove deprecated Struct.destructure method (95caf6e)
• api: remove deprecated top-level case function (ce927d7)
• binding: make DerefMap computation lazy and support multiple value inputs (#11540) (5757caa)
• datatypes: fully remove unused DataType.name (#11102) (8a7534c)
• memtables: remove support for problematic memtable properties (5d35ad5)
• split temp table support along finalizer lines (4e4a165)
• trino: remove deprecated password parameter (#11538) (a178459)

Performance

• athena: replace list_table_metadata with get_table_metadata in _get_schema_with_query (#11573) (40b73e7)
• athena: use get_table_metadata in get_schema (#11574) (e4e582a)
• binding: only create a dereference map in Table.bind when needed (#11531) (90e790c)

10.8.0

10.8.0 (2025-07-28)

Features

• datatypes: add Schema.from_sqlglot method to produce an Ibis schema from SQLGlot (#11351) (f6641e7)
• datatypes: allow dtype(Union[T, None]) (#11506) (c83f842)
• polars: support is_in queries from uncorrelated tables (476353e)

Bug Fixes

• clickhouse: return the correct type from InSubquery operations (69be180)

Documentation

• add join method docstring templates for various join types (#11507) (4798af4)
• improve docstrings for temporal Values (#11505) (8641087)

10.7.0

10.7.0 (2025-07-27)

Features

• api: add Expr.to_sql() method (#11357) (493d6e5)
• api: always prefer overwriting existing fields when unpacking with overlapping column names (#11470) (8027d51)
• backends: kurtosis (50c3132)
• ux: workaround for spurious display()s in jupyter (58dea60)

Bug Fixes

• api: do not assume that dir produces attributes that exist (#11497) (1a5b1a2)
• athena/trino/bigquery: make temporal diff robust to operand order (857758a)
• datafusion: ensure approx quantile works (195b4f6)
• distinct: allow deleting rows when on is unspecified (#11383) (4902ec6)
• duckdb/snowflake: decouple Pandas and PyArrow backend converters (#11431) (0f7f7c1)
• duckdb: disallow duckdb 1.3.1 due to hangs when loading the geospatial extension (#11409) (a67a244)
• exasol: ensure that rounding is not subverted by sqlglot cast (3aa3d34)
• polars: ensure that memtables can be compiled into LazyFrames by running pre-execute hooks (#11392) (7be448e)
• postgres: make pyarrow round trip of nans and nulls higher fidelity (#11396) (60a5ba9)
• psycopg: avoid specifying the psycopg dependency implementation (#11424) (038c339)
• sql: ensure that CTEs as part of .sql calls are compiled in topological order (#11439) (8e227ac)
• trino/athena: ensure that DateDiff operands are in the correct order (#11434) (e29881c)
• trino/athena: ensure that regex extract compiler dispatch method is spelled correctly (#11436) (944fa99)

Documentation

• improve docs for set ops (#11394) (6521a53)
• replace: clarify types and add link to Value.substitute (#11413) (1c05d2f)
• substitute: add to See Also section to fix rendered formatting (#11412) (d381536)
• to_sql: fix definition package (0b5daea)
• to_sql: fix see also reference from Value.to_sql to ibis.to_sql (7e0042c)
• udfs: add examples of udfs with struct types as inputs (#11385) (0e8a932)

Refactors

• rename to_sqlglot to to_sqlglot_columns_definition (#11455) (0e4cf43)
• sqlite: move sqlite3 imports local to avoid pyodide import issues with non-sqlite backends (#11500) (b8110d3)

Performance

• mutate: skip dereferencing cols preserved in .mutate (#11458) (70de1a1)

10.6.0

10.6.0 (2025-06-16)

Features

• add Deferred to signatures of many Table methods (d73a040)
• api: allow creating specific uuid values and add to docs (#11304) (476fa9f)
• athena: support partitioning in create_table (6ea5cd4)
• bigquery: add job_id_prefix to functions that produce bigquery jobs (#11265) (4006d68)
• bigquery: add QueryJobConfig properties to bigquery backend specified at query time (#11255) (6d7defc)
• bigquery: add the ability to specify a bigquery job id in a query (3a383d8)
• bigquery: add the ability to specify a bigquery job id in a query (e0ad6ea)
• datafusion: add BitwiseNot, Clip and IntervalFromInteger (1666e33)
• datafusion: add Greatest and Least (10aa7d7)
• datafusion: add support for anys and alls (#11162) (0f23dc5)
• improve typing and docstring for to_pyarrow (2600c65)
• make Backend._from_url() consistently use kwargs as overrides (14d103a)
• risingwave: add support for includes in create_source (f88c0dc)
• types: add raw_type to dt.Unknown (#11337) (9296107)
• typing: improve type hints of .isin and .notin (#11156) (c10b627)
• typing: return Self from .fill_null() and similar (4218644)
• ux: improve rich rendering (#11326) (1b8038c)

Bug Fixes

• api: deal with null literals in Struct.__getitem__ (#11299) (ccd9359)
• bigquery: maintain custom client project id when provided (12bc4c0)
• bigquery: raise error if bigquery project id cannot be identified (a93b7d9)
• bigquery: undo removal of job_id_prefix functionality from raw_sql (#11349) (cf404c2)
• bigquery: use job_id_prefix instead of job_id (364e33a)
• builds: ensure that python 3.13 builds (601ea3e)
• datafusion: override struct syntax to use bracket (dae8e7a)
• datatype: convert from sqlglot VARCHAR(MAX) correctly (#11202) (dc37796)
• duckdb: allow creating tables using data with a different column order (cf83480)
• duckdb: support materializing enum types to pyarrow (#11214) (d2f8839)
• flink: unit is ignored when str cast to timestamp (#11167) (cfa5768)
• impala: make create_database follow the CanCreateDatabase API (#11149) (15ff868)
• oracle: cast string to int for the precision and scale of decimal (969453e)
• polars: make sure sum is NULL to match other backends (#11321) (f778f03), closes #11318
• postgres: move more configuration to _post_connect() so that .from_connection() also uses it (15df443)
• postgres: set search path more compliantly (1c284a1)
• pyarrow: properly support round tripping of fixed size list types (#11330) (e2937a4)
• snowflake: allow pyarrow 20 to work (#11173) (0ed7862)
• snowflake: use newer sqlglot syntax for unnest (0b57361)
• sql: ensure that when distinct with order by is split that all fields are considered for rewrites (#11334) (e4df7a8)
• trino: fix nested row casting inside of lambda (#11146) (755e2a3)

Documentation

• actually include expression-misc.qmd in the navbar (#11306) (4a935dc)
• add concepts doc on casting (cea8ed8)
• add examples of .isin() (#11154) (b4e6272)
• databricks: add a backend documentation page (2665f53)
• improve docstrings on methods using the database param (#11112) (458f06d)
• postgres: show passing schema to Postgres.from_url() (46cce76)
• support-matrix: do not escape the HTML links (895eb58)

Refactors

• bigquery: remove unused connect function in ibis/__init__.py (#11305) (77a6dff)
• rich: consolidate rich logic into a more central place (#11283) (44d5e6b)
• test: centralize marks for no array, struct, etc support (#11315) (c581afe)

Performance

• duckdb: optimize ArrayIndex sql when index is a Literal (3f32794)

10.5.0

10.5.0 (2025-04-18)

Features

• api: expose DataType publicly at ibis.DataType (#11101) (4164265)
• backends: fixup drop_database() when passing a specific catalog (#11110) (57d79b7)
• databricks: add support for json via variant (a1f2bab)
• datafusion: basic map operations (16b0c4c)
• postgres: allow disablement of hstore extension loading (#11126) (35fc38b), closes #11103
• postgres: implement maps in terms of JSONB instead of HSTORE (ff6100a)
• pyflink: fix type mapping from BINARY to VARBINARY (bbcd5b9)
• risingwave: support map operations (e7d8c3a)
• temporal: implement casting floating values to timestamps (0356caf)

Bug Fixes

• athena: only replace commas instead of all special characters in column names (#11138) (4c74ddd)
• databricks: use AS JSON for programmatic output of schema information (d55a5ee)
• duckdb: always create temp tables in temp.main (#11092) (20bec13)
• ir: make ibis.to_sql() parse join limits and projections; fixes #11105 (a416755)
• oracle: return a sqlglot.DataType, not str, when compiling string dtype (#11124) (99be73b)
• postgres: clean up possible transaction hangs (6f9d141)
• scalar-subquery: ensure that scalar subqueries value is resolvable with no underlying projection (#11125) (a92c3cb)
• snowflake/bigquery/athena: ensure that timestamp casting works (#11135) (4bb0b78)
• typing: remove unnecessary generics in ibis.struct() typing (#11070) (9d3aece)

Documentation

• clarify NULL/NaN/inf distinction in docstring and examples (#11077) (3d4e741)
• fix SECURITY.md formatting (63aeff4)

10.4.0

10.4.0 (2025-03-27)

Features

• api: make topk() and value_counts() more flexible (#10928) (329ad7c)
• datatypes: add string length (#11045) (61dd7ea)
• flink: support StringSplit (#11049) (b83a88e)
• postgres: add support for reading enum types as strings (#11028) (dfb818a)
• postgres: implement support for asof_join API via a lateral join (#11024) (8eb9d33)
• ux: include type of bad value in SignatureValidationError (#10988) (a706afa)

Bug Fixes

• backends: array string join on empty array now consistently returns None across all supported backends (#10913) (a02a392)
• polars: use engine="streaming" instead of streaming=True (e1bbe55)
• postgres: clean up leaky cursor coming from raw_sql (#11001) (c3097a7)

Documentation

• add introduction to pandas users for adding columns with single text values (literal) (#11010) (6032e71)
• fix to_* methods docstrings typos (#11013) (523e4c3)
• ir: remove extra backticks to fix formatting (7c3e5fc)

Refactors

• postgres: use raw sql for get_schema invocation to simplify code (#11037) (9b25ab1)

Performance

• duckdb: optimize generated SQL for ArrayConcat (#10999) (d38db7e)
• struct: drop unused fields on Struct.getitem (#10996) (e5fe34c)

10.3.1

10.3.1 (2025-03-14)

Bug Fixes

• datatypes: ensure that newer sqlglot BLOB type is not accessed if it does not exist (6452a5e)
• deps: drop pytz from dependencies (#10976) (3ecf731)
• postgres: use .transaction method instead of managing our own (0edbab9)

Documentation

• link to our config file for conventional commits warning (#10986) (a945c96)

10.3.0

10.3.0 (2025-03-11)

Features

• duckdb: add read_xlsx implementation (705aa16)
• duckdb: add to_xlsx implementation (1800abd)
• python: support python 3.9 again (#10949) (786d8f0)

Bug Fixes

• bigquery: adjust codegen for newer version of sqlglot (8105a09)
• memtable: ensure that constructing an empty memtable from a dataframe works (#10945) (29c796a), closes #10940
• mysql: add regular blob to type mapping (365c157)
• mysql: ensure that unsigned integers are mapped to the ibis type system (#10947) (3eb76f8), closes #10946
• postgres: create server-side cursors in the iterator to ensure cursor is not cleaned up before it is used (f5f798e)
• postgres: ensure that cursor lifetime is at least as long as the generator (f42ab45)

Documentation

• mention theseus in the readme and docs landing page diagram (360c25b)
• setup: add just recipe and setup entry for uv (#10959) (dd33f47)
• tutorials: reorganize the "Getting started" tab (f78a8fb)

Refactors

• oracle: remove unused owner column from select metadata query (#10935) (56dcc42)
• postgres: split out hstore extension registration into separate method (0bdb5a0)

Performance

• postgres: improve to_pyarrow_batches by using server-side cursors (#10954) (cb17b8b), closes #10938

10.2.0

10.2.0 (2025-03-03)

Features

• examples: expand examples functionality to more backends (#10916) (90a174b)

Bug Fixes

• bigquery: ensure that count distinct can be used in window functions without a specified window (#10911) (c9120e1)
• build: ensure that legacy tools can recognize the validity of the Ibis license (#10905) (c29d83a)
• datatypes: document "!" as meaning non-nullable, enable nullable arg for type hints (#10893) (601aabe)
• examples: ensure that examples work on mssql and avoid using overwrite (#10909) (8336483)
• pyarrow-conversion: ensure that non-nullability is preserved (#10897) (b2c5a0c)
• snowflake: use schema when constructing pyarrow table (#10903) (8b9b143)

Documentation

• add an example with literal in "Getting started" (#10918) (b69061b)
• build: disable misbehaving duckdb spatial optimizer (#10898) (c37116f)
• selectors: render the module instead of hand-picking the documented components (#10892) (d841481)

Refactors

• duckdb: simplify loading and installation of extensions (#10900) (421972a)

Performance

• cache entry points (#10889) (62e9bca)
• disallow dunder attributes on Deferred (#10888) (40b20f1)

10.1.0

10.1.0 (2025-02-22)

Features

• pyspark: add partitionBy argument to create_table (c99cc23), closes #8900
• python: allow python 3.9 installations (#10859) (fbe8c8b)

Bug Fixes

• bigquery: allow sane use of params with raw_sql (#10874) (0a684c3)
• deps: update dependency datafusion to v44 (979cf59)
• deps: update dependency sqlglot to >=23.4,<26.5 (#10807) (f09e8e2)
• deps: update dependency sqlglot to >=23.4,<26.7 (15111f8)
• dev-tools: ensure that bump is minimal so that later release sort properly (#10878) (39729c7)
• duckdb: use the delta extension for reading deltalake data (#10833) (beeaa29), closes #10829
• join: error in more places on colliding column names (#10778) (ec06e1e)
• mssql: ensure that dots in database parameter to list_tables are used as path delineators (#10863) (cdbbcb9)
• mssql: ensure that we only escape passwords if the password is not None (e589344)
• mysql: explicitly handle the zero integer -> timestamp case (f5e8c4f)
• pyspark: avoid potentially different field names produced by SQL by using python-native APIs (#10877) (9538d51)
• snowflake: use get instead of get_path; get_path does not support columns with spaces (#10836) (50c978b), closes #10835
• sqlglot: ensure that sge.Median is only accessed when it exists (dc6b7e0)
• sqlite: avoid generating double-quoted string literals (#10873) (76b0114)

Documentation

• add blogpost for Athena backend (#10796) (f2f09eb)
• add information about reading from cloud buckets (32e82c7)
• add udf rewriting blog post (c6ecf6b)
• blog: add post on SQL understanding and Ibis (#10762) (94425ec)
• blog: convert case to cases in blog posts (#10560) (bbf98de)
• blog: use more reliable URL for geospatial data (72b7673)
• fix reference to incorrect value (1945237)
• move __getitem__ docs so that quarto publishes them (#10870) (269cdfe)
• release-notes: fixup release notes (fb0798e)
• remove incorrect parameters (#10876) (a707778)
• update post date (ea0cc95)

Refactors

• duckdb: remove the pyarrow read_parquet fallback (5fa0103)