10.8.0

10.8.0 (2025-07-28)

Features

• datatypes: add Schema.from_sqlglot method to produce an Ibis schema from SQLGlot (#11351) (f6641e7)
• datatypes: allow dtype(Union[T, None]) (#11506) (c83f842)
• polars: support is_in queries from uncorrelated tables (476353e)

Bug Fixes

• clickhouse: return the correct type from InSubquery operations (69be180)

Documentation

• add join method docstring templates for various join types (#11507) (4798af4)
• improve docstrings for temporal Values (#11505) (8641087)

10.7.0

10.7.0 (2025-07-27)

Features

• api: add Expr.to_sql() method (#11357) (493d6e5)
• api: always prefer overwriting existing fields when unpacking with overlapping column names (#11470) (8027d51)
• backends: kurtosis (50c3132)
• ux: workaround for spurious display()s in jupyter (58dea60)

Bug Fixes

• api: do not assume that dir produces attributes that exist (#11497) (1a5b1a2)
• athena/trino/bigquery: make temporal diff robust to operand order (857758a)
• datafusion: ensure approx quantile works (195b4f6)
• distinct: allow deleting rows when on is unspecified (#11383) (4902ec6)
• duckdb/snowflake: decouple Pandas and PyArrow backend converters (#11431) (0f7f7c1)
• duckdb: disallow duckdb 1.3.1 due to hangs when loading the geospatial extension (#11409) (a67a244)
• exasol: ensure that rounding is not subverted by sqlglot cast (3aa3d34)
• polars: ensure that memtables can be compiled into LazyFrames by running pre-execute hooks (#11392) (7be448e)
• postgres: make pyarrow round trip of nans and nulls higher fidelity (#11396) (60a5ba9)
• psycopg: avoid specifying the psycopg dependency implementation (#11424) (038c339)
• sql: ensure that CTEs as part of .sql calls are compiled in topological order (#11439) (8e227ac)
• trino/athena: ensure that DateDiff operands are in the correct order (#11434) (e29881c)
• trino/athena: ensure that regex extract compiler dispatch method is spelled correctly (#11436) (944fa99)

Documentation

• improve docs for set ops (#11394) (6521a53)
• replace: clarify types and add link to Value.substitute (#11413) (1c05d2f)
• substitute: add to See Also section to fix rendered formatting (#11412) (d381536)
• to_sql: fix definition package (0b5daea)
• to_sql: fix see also reference from Value.to_sql to ibis.to_sql (7e0042c)
• udfs: add examples of udfs with struct types as inputs (#11385) (0e8a932)

Refactors

• rename to_sqlglot to to_sqlglot_columns_definition (#11455) (0e4cf43)
• sqlite: move sqlite3 imports local to avoid pyodide import issues with non-sqlite backends (#11500) (b8110d3)

Performance

• mutate: skip dereferencing cols preserved in .mutate (#11458) (70de1a1)

10.6.0

10.6.0 (2025-06-16)

Features

• add Deferred to signatures of many Table methods (d73a040)
• api: allow creating specific uuid values and add to docs (#11304) (476fa9f)
• athena: support partitioning in create_table (6ea5cd4)
• bigquery: add job_id_prefix to functions that produce bigquery jobs (#11265) (4006d68)
• bigquery: add QueryJobConfig properties to bigquery backend specified at query time (#11255) (6d7defc)
• bigquery: add the ability to specify a bigquery job id in a query (3a383d8)
• bigquery: add the ability to specify a bigquery job id in a query (e0ad6ea)
• datafusion: add BitwiseNot, Clip and IntervalFromInteger (1666e33)
• datafusion: add Greatest and Least (10aa7d7)
• datafusion: add support for anys and alls (#11162) (0f23dc5)
• improve typing and docstring for to_pyarrow (2600c65)
• make Backend._from_url() consistently use kwargs as overrides (14d103a)
• risingwave: add support for includes in create_source (f88c0dc)
• types: add raw_type to dt.Unknown (#11337) (9296107)
• typing: improve type hints of .isin and .notin (#11156) (c10b627)
• typing: return Self from .fill_null() and similar (4218644)
• ux: improve rich rendering (#11326) (1b8038c)

Bug Fixes

• api: deal with null literals in Struct.__getitem__ (#11299) (ccd9359)
• bigquery: maintain custom client project id when provided (12bc4c0)
• bigquery: raise error if bigquery project id cannot be identified (a93b7d9)
• bigquery: undo removal of job_id_prefix functionality from raw_sql (#11349) (cf404c2)
• bigquery: use job_id_prefix instead of job_id (364e33a)
• builds: ensure that python 3.13 builds (601ea3e)
• datafusion: override struct syntax to use bracket (dae8e7a)
• datatype: convert from sqlglot VARCHAR(MAX) correctly (#11202) (dc37796)
• duckdb: allow creating tables using data with a different column order (cf83480)
• duckdb: support materializing enum types to pyarrow (#11214) (d2f8839)
• flink: unit is ignored when str cast to timestamp (#11167) (cfa5768)
• impala: make create_database follow the CanCreateDatabase API (#11149) (15ff868)
• oracle: cast string to int for the precision and scale of decimal (969453e)
• polars: make sure sum is NULL to match other backends (#11321) (f778f03), closes #11318
• postgres: move more configuration to _post_connect() so that .from_connection() also uses it (15df443)
• postgres: set search path more compliantly (1c284a1)
• pyarrow: properly support round tripping of fixed size list types (#11330) (e2937a4)
• snowflake: allow pyarrow 20 to work (#11173) (0ed7862)
• snowflake: use newer sqlglot syntax for unnest (0b57361)
• sql: ensure that when distinct with order by is split that all fields are considered for rewrites (#11334) (e4df7a8)
• trino: fix nested row casting inside of lambda (#11146) (755e2a3)

Documentation

• actually include expression-misc.qmd in the navbar (#11306) (4a935dc)
• add concepts doc on casting (cea8ed8)
• add examples of .isin() (#11154) (b4e6272)
• databricks: add a backend documentation page (2665f53)
• improve docstrings on methods using the database param (#11112) (458f06d)
• postgres: show passing schema to Postgres.from_url() (46cce76)
• support-matrix: do not escape the HTML links (895eb58)

Refactors

• bigquery: remove unused connect function in ibis/__init__.py (#11305) (77a6dff)
• rich: consolidate rich logic into a more central place (#11283) (44d5e6b)
• test: centralize marks for no array, struct, etc support (#11315) (c581afe)

Performance

• duckdb: optimize ArrayIndex sql when index is a Literal (3f32794)

10.5.0

10.5.0 (2025-04-18)

Features

• api: expose DataType publicly at ibis.DataType (#11101) (4164265)
• backends: fixup drop_database() when passing a specific catalog (#11110) (57d79b7)
• databricks: add support for json via variant (a1f2bab)
• datafusion: basic map operations (16b0c4c)
• postgres: allow disablement of hstore extension loading (#11126) (35fc38b), closes #11103
• postgres: implement maps in terms of JSONB instead of HSTORE (ff6100a)
• pyflink: fix type mapping from BINARY to VARBINARY (bbcd5b9)
• risingwave: support map operations (e7d8c3a)
• temporal: implement casting floating values to timestamps (0356caf)

Bug Fixes

• athena: only replace commas instead of all special characters in column names (#11138) (4c74ddd)
• databricks: use AS JSON for programmatic output of schema information (d55a5ee)
• duckdb: always create temp tables in temp.main (#11092) (20bec13)
• ir: make ibis.to_sql() parse join limits and projections; fixes #11105 (a416755)
• oracle: return a sqlglot.DataType, not str, when compiling string dtype (#11124) (99be73b)
• postgres: clean up possible transaction hangs (6f9d141)
• scalar-subquery: ensure that scalar subqueries value is resolvable with no underlying projection (#11125) (a92c3cb)
• snowflake/bigquery/athena: ensure that timestamp casting works (#11135) (4bb0b78)
• typing: remove unnecessary generics in ibis.struct() typing (#11070) (9d3aece)

Documentation

• clarify NULL/NaN/inf distinction in docstring and examples (#11077) (3d4e741)
• fix SECURITY.md formatting (63aeff4)

10.4.0

10.4.0 (2025-03-27)

Features

• api: make topk() and value_counts() more flexible (#10928) (329ad7c)
• datatypes: add string length (#11045) (61dd7ea)
• flink: support StringSplit (#11049) (b83a88e)
• postgres: add support for reading enum types as strings (#11028) (dfb818a)
• postgres: implement support for asof_join API via a lateral join (#11024) (8eb9d33)
• ux: include type of bad value in SignatureValidationError (#10988) (a706afa)

Bug Fixes

• backends: array string join on empty array now consistently returns None across all supported backends (#10913) (a02a392)
• polars: use engine="streaming" instead of streaming=True (e1bbe55)
• postgres: clean up leaky cursor coming from raw_sql (#11001) (c3097a7)

Documentation

• add introduction to pandas users for adding columns with single text values (literal) (#11010) (6032e71)
• fix to_* methods docstrings typos (#11013) (523e4c3)
• ir: remove extra backticks to fix formatting (7c3e5fc)

Refactors

• postgres: use raw sql for get_schema invocation to simplify code (#11037) (9b25ab1)

Performance

• duckdb: optimize generated SQL for ArrayConcat (#10999) (d38db7e)
• struct: drop unused fields on Struct.getitem (#10996) (e5fe34c)

10.3.1

10.3.1 (2025-03-14)

Bug Fixes

• datatypes: ensure that newer sqlglot BLOB type is not accessed if it does not exist (6452a5e)
• deps: drop pytz from dependencies (#10976) (3ecf731)
• postgres: use .transaction method instead of managing our own (0edbab9)

Documentation

• link to our config file for conventional commits warning (#10986) (a945c96)

10.3.0

10.3.0 (2025-03-11)

Features

• duckdb: add read_xlsx implementation (705aa16)
• duckdb: add to_xlsx implementation (1800abd)
• python: support python 3.9 again (#10949) (786d8f0)

Bug Fixes

• bigquery: adjust codegen for newer version of sqlglot (8105a09)
• memtable: ensure that constructing an empty memtable from a dataframe works (#10945) (29c796a), closes #10940
• mysql: add regular blob to type mapping (365c157)
• mysql: ensure that unsigned integers are mapped to the ibis type system (#10947) (3eb76f8), closes #10946
• postgres: create server-side cursors in the iterator to ensure cursor is not cleaned up before it is used (f5f798e)
• postgres: ensure that cursor lifetime is at least as long as the generator (f42ab45)

Documentation

• mention theseus in the readme and docs landing page diagram (360c25b)
• setup: add just recipe and setup entry for uv (#10959) (dd33f47)
• tutorials: reorganize the "Getting started" tab (f78a8fb)

Refactors

• oracle: remove unused owner column from select metadata query (#10935) (56dcc42)
• postgres: split out hstore extension registration into separate method (0bdb5a0)

Performance

• postgres: improve to_pyarrow_batches by using server-side cursors (#10954) (cb17b8b), closes #10938

10.2.0

10.2.0 (2025-03-03)

Features

• examples: expand examples functionality to more backends (#10916) (90a174b)

Bug Fixes

• bigquery: ensure that count distinct can be used in window functions without a specified window (#10911) (c9120e1)
• build: ensure that legacy tools can recognize the validity of the Ibis license (#10905) (c29d83a)
• datatypes: document "!" as meaning non-nullable, enable nullable arg for type hints (#10893) (601aabe)
• examples: ensure that examples work on mssql and avoid using overwrite (#10909) (8336483)
• pyarrow-conversion: ensure that non-nullability is preserved (#10897) (b2c5a0c)
• snowflake: use schema when constructing pyarrow table (#10903) (8b9b143)

Documentation

• add an example with literal in "Getting started" (#10918) (b69061b)
• build: disable misbehaving duckdb spatial optimizer (#10898) (c37116f)
• selectors: render the module instead of hand-picking the documented components (#10892) (d841481)

Refactors

• duckdb: simplify loading and installation of extensions (#10900) (421972a)

Performance

• cache entry points (#10889) (62e9bca)
• disallow dunder attributes on Deferred (#10888) (40b20f1)

10.1.0

10.1.0 (2025-02-22)

Features

• pyspark: add partitionBy argument to create_table (c99cc23), closes #8900
• python: allow python 3.9 installations (#10859) (fbe8c8b)

Bug Fixes

• bigquery: allow sane use of params with raw_sql (#10874) (0a684c3)
• deps: update dependency datafusion to v44 (979cf59)
• deps: update dependency sqlglot to >=23.4,<26.5 (#10807) (f09e8e2)
• deps: update dependency sqlglot to >=23.4,<26.7 (15111f8)
• dev-tools: ensure that bump is minimal so that later release sort properly (#10878) (39729c7)
• duckdb: use the delta extension for reading deltalake data (#10833) (beeaa29), closes #10829
• join: error in more places on colliding column names (#10778) (ec06e1e)
• mssql: ensure that dots in database parameter to list_tables are used as path delineators (#10863) (cdbbcb9)
• mssql: ensure that we only escape passwords if the password is not None (e589344)
• mysql: explicitly handle the zero integer -> timestamp case (f5e8c4f)
• pyspark: avoid potentially different field names produced by SQL by using python-native APIs (#10877) (9538d51)
• snowflake: use get instead of get_path; get_path does not support columns with spaces (#10836) (50c978b), closes #10835
• sqlglot: ensure that sge.Median is only accessed when it exists (dc6b7e0)
• sqlite: avoid generating double-quoted string literals (#10873) (76b0114)

Documentation

• add blogpost for Athena backend (#10796) (f2f09eb)
• add information about reading from cloud buckets (32e82c7)
• add udf rewriting blog post (c6ecf6b)
• blog: add post on SQL understanding and Ibis (#10762) (94425ec)
• blog: convert case to cases in blog posts (#10560) (bbf98de)
• blog: use more reliable URL for geospatial data (72b7673)
• fix reference to incorrect value (1945237)
• move __getitem__ docs so that quarto publishes them (#10870) (269cdfe)
• release-notes: fixup release notes (fb0798e)
• remove incorrect parameters (#10876) (a707778)
• update post date (ea0cc95)

Refactors

• duckdb: remove the pyarrow read_parquet fallback (5fa0103)

10.0.0

10.0.0 (2025-02-06)

⚠ BREAKING CHANGES

• api: change as_interval unit argument to be positional-only
• api: change as_timestamp unit argument to be positional-only
• api: standardize unnest and pivot_longer signatures
• api: remove deprecated Table.relabel method
• api: standardize StringValue method signatures
• api: standardize NumericValue methods
• api: make GeoSpatialValue.contains positional-only
• api: make Table.describe quantile argument keyword-only
• api: remove deprecated Table.relabel method
• api: make Table.drop_null/Table.fill_null/Table.window_by/Table.alias argument positional-only
• api: make Table.sample fraction argument positional-only
• api: make Table.aggregate metrics argument positional-only
• api: make Table set operation methods positional-only
• api: make Table.cast and Table.try_cast methods positional-only
• api: make nth positional-only
• api: make isin/notin/cases/identical_to positional-only
• api: make null-related methods and null function positional-only
• api: make Value.cast and Value.try_cast positional-only
• internals: make Value.name positional-only
• internals: make Expr.pipe positional-only
• internals: make Expr.equals positional-only
• api: align signatures of to_json methods
• api: align signatures of to_delta methods
• api: align signatures of to_csv/to_csv_dir methods
• api: align signatures of to_parquet/to_parquet_dir methods
• api: align .sql method signatures across polars and sql as well as the Table method
• api: top-level connect method now takes its first argument as positional-only
• duckdb: align signatures of read_sqlite/read_mysql/read_postgres methods in the duckdb backend
• api: align signatures of read_delta method; sources are positional-only, everything else is required-keyword
• api: canonicalize has_operation backend method; single argument is positional-only
• api: canonicalize read_kafka and to_kafka methods of the PySpark backend
• api: canonicalize drop_table_or_view method of the impala backend
• api: canonicalize to_geo signature of the the DuckDB backend
• api: canonicalize read_geo signature of the the DuckDB backend
• api: align signatures of list_catalogs; like` argument is now keyword-only
• bigquery: canonicalize set_database signature
• api: make list_databases arguments all required-keyword
• risingwave: canonicalize signatures of risingwave-specific create_* methods
• polars: canonicalize signature of read_pandas method
• api: align signatures of drop_table method; name is positional-only; the rest are keyword-only
• api: align signatures of create_catalog and drop_catalog methods; name is positional-only; the rest are keyword-only
• api: compile method is now the same across backends
• api: align signatures of create_table method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
• api: align signatures of create_view method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
• api: align signatures of drop_view method; name is positional-only; the rest are keyword-only
• api: align signatures of truncate_table method; name is positional-only; the rest are keyword-only
• api: align signatures of insert method; name is positional-only; obj is positional-or-keyword; the rest are keyword-only
• api: align signatures of read_json method; sources are positional-only, everything else is required-keyword
• api: align signatures of read_csv method; sources are positional-only, everything else is required-keyword
• api: align signatures of read_parquet method; sources are positional-only, everything else is required-keyword
• api: align signatures of to_torch method
• api: align signatures of to_polars method
• api: align signatures of Backend.list_tables method; all arguments are now keyword-only
• api: align signatures of Backend.table method; name is positional-only; everything else is required-keyword
• api: align signatures of create_database and drop_database; name is positional-only; everything else is required-keyword
• api: standardize MapValue method signatures
• api: standardize ArrayValue method signatures
• api: type argument of struct function is now required-keyword
• api: standardize TemporalValue APIs
• api: where argument of aggregate functions is now required-keyword
• api: hashbytes and hexdigest are now positional-only
• api: standardize how argument to join methods as keyword-only and standardize remaining arguments
• api: ibis.coalesce/ibis.greatest/ibis.least are now positional-only
• api: Expr.ifelse is now positional-only
• api: top-level set operation functions are now positional-only
• api: set_backend and get_backend functions are now positional-only
• api: ntile function and method is now positional-only
• api: ibis.preceding/ibis.following` are now positional-only
• api: expr argument of ibis.asc/ibis.desc is now positional-only; nulls_first is keyword-only
• api: data argument of ibis.memtable is now positional-only; the rest are keyword-only
• api: pairs argument of ibis.schema is now positional-only; the rest are keyword-only
• api: ibis.param is now positional-only
• api: n argument in Table.limit and Table.head is now required-positional
• api: offset argument in Table.limit is now required-keyword
• api: temporal window expression APIs now require all arguments as keywords
• api: to_pyarrow and to_pyarrow_batches requires expr as positional-only and keyword for everything else
• api: to_pandas_batches requires expr as positional-only
• api: execute and to_pandas methods now require expr as positional-only
• api: distance is now a required keyword argument for the d_within api
• duckdb: The duckdb backend's read_csv method accepts only DuckDB types for the values components of the columns and types arguments. You may need need to adjust existing code. For example, the string "float64" should be replaced with the string "double".
• duckdb: The read_in_memory method is removed from the duckdb backend. Use ibis.memtable instead.
• api: The how parameter of the Value.arbitrary method is removed. call Value.first or Value.last explicitly
• api: The StringValue.initcap method is removed. Use StringValue.capitalize instead.
• api: IntegerValue.label is redundant with the IntegerValue.cases method, use that instead. Replace expr.label(labels) with expr.cases(*enumerate(labels))
• register: The deprecated register method has been removed. Please use the file-specific read_* methods instead. For in-memory objects, pass them to ibis.memtable or create_table.
• duckdb: Special handling of the temp_directory argument passed to Ibis is removed in favor of passing the argument through directly to duckdb.connect. Interior nodes of directory trees must be created, e.g., using Path.mkdir(exists_ok=True, parents=True), mkdir -p, etc.
• config: option_context is removed. Use contextlib.contextmanager to create your own version of this functionality if necessary.
• duckdb: The DuckDB lower bound has been bumped to a version that has storage backwards compatibility. You may need to migrate your DuckDB database files.
• api: has_name has always returned True since 9.0. It is safe to remove any calls to has_name.
• backends: execute now returns non-numpy objects for scalar values.
• api: ibis.negate is removed. Use the negate method on a
  specific column, instead.
• api: All ibis.geo_* functions are removed. Equivalent
  methods are available on all geo columns.
• api: where is removed. Use ibis.ifelse instead.
• value: Value.greatest and Value.least are removed. Use
  ibis.greatest and ibis.least, instead.
• joins: Passing a pyarrow.Table or a pandas.DataFrame as
  the right-hand-side of a join is no longer supported.

To join against in-memory data, you can pass the in-memory object to
ibis.memtable or con.create_table and use the resulting table object
instead.

Issues closed

• api: Removed hierarchical usage of schema.
  Ibis uses the following naming conventions:

  • schema: a mapping of column names to datatypes
  • database: a collection of tables
  • catalog: a collection of databases

• mysql: Ibis now uses the MySQLdb driver. You may need to install MySQL client libraries to build the extension.

• padding: String padding operations now follow Python semantics and leave strings greater than the padding length untouched.

• pandas: The pandas backend is removed. Note that pandas DataFrames are STILL VALID INPUTS AND OUTPUTS and will remain so for the foreseeable future. Please use one of the other local backends like DuckDB, Polars, or DataFusion to perform operations directly on pandas DataFrames.

• dask: The dask backend is removed. Please use one of the
  other backends that Ibis supports.

• api: remove deprecated where methodism (886b2d1)

• api: remove top-level negate function (c8c37dd)

• api: remove top-level geo functions ([6b18...