v1.2.682-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(ci): flaky test by @zhyass in #17149

Code Refactor 🎉

• refactor: make fuse_amend implementation more convenient for on-premise deployments by @dantengsky in #17147

Others 📒

• chore(query): bump ast 0.1.3 by @sundy-li in #17145

Full Changelog: v1.2.681-nightly...v1.2.682-nightly

v1.2.681-nightly

What's Changed

Exciting New Features ✨

• feat: add server to admin udfs api by @everpcpc in #17141

Others 📒

• chore: Split functions crate into multi crates by @TCeason in #17133

Full Changelog: v1.2.680-nightly...v1.2.681-nightly

v1.2.680

What's Changed

Exciting New Features ✨

• feat(query): optimize min/max aggregation use accurate column stats by @sundy-li in #16270
• feat: new table option data_retention_period_in_hours by @dantengsky in #16266
• feat(query): add config udf_server_allow_insecure by @everpcpc in #16290
• feat(query): Inverted index search function support options by @b41sh in #16256
• feat(geo): add geography data type by @forsaken628 in #16286
• feat(query): Support create/drop dictionary ddl. by @Winnie-Hong0927 in #16280
• feat: add endpoint /v1/session/logout and record client session ID in meta. by @youngsofun in #16281
• feat(query): ST_GEOGRAPHYFROMEWKT by @forsaken628 in #16302
• feat: support temporary table by @SkyFan2002 in #16250
• feat(query): Add jq set returning function for processing Variant data with jq filters by @maxjustus in #16288
• feat: Add KVPbApi::get_id_and_value() by @drmingdrmer in #16325
• feat: Check essential permissions when creating table with external location by @dantengsky in #16315
• feat(query): add is_local to admin /v1/tables info by @everpcpc in #16329
• feat(query): add is_external to /v1/tables info by @everpcpc in #16336
• feat(query): add plan cache by @sundy-li in #16333
• feat: login and renew return session_token_validity_in_secs too. by @youngsofun in #16313
• feat(query): add discovery nodes api by @zhang2014 in #16353
• feat: hilbert clustering by @zhyass in #16296
• feat: add error stack to ErrorCode by @andylokandy in #16343
• feat: support vacuum temporary table by @SkyFan2002 in #16364
• feat: http handler support temp table. by @youngsofun in #16375
• feat: batch size hint of stream consumption by @zhyass in #16372
• feat(query): make external server parallel by batch by @sundy-li in #16390
• feat: estimate selectivity by table sample by @xudong963 in #16362
• feat: introduce an intermediate version that is compatible with vacuum2 by @SkyFan2002 in #16354
• feat(query): Procedure (Part1) by @TCeason in #16348
• feat: add limit push down rule by @xudong963 in #16403
• feat: discovery_nodes not need auth. by @youngsofun in #16392
• feat(query): Support SHOW VARIABLES by @TCeason in #16409
• feat: compact after merge sort by @zhyass in #16401
• feat(query): Support access Redis data from dictionaries via the dict_get function. by @Winnie-Hong0927 in #16389
• feat: continue vacuum drop table on per-table cleanup failures by @SkyFan2002 in #16424
• feat: Load DATABEND_ENTERPRISE_LICENSE_EMBEDDED while building by @Xuanwo in #16440
• feat(query): Support access Mysql data from dictionaries via the dict_get function. by @Winnie-Hong0927 in #16444
• feat: Auto compact(re-cluster) for multiple table insertion statement by @SkyFan2002 in #16443
• feat(query): Procedure Part2 support arguments by @TCeason in #16453
• feat(query): refactor window spill by @Dousir9 in #16448
• feat(query): push rank limit into aggregate partial node by @sundy-li in #16466
• feat: add eliminate union optimizer rule by @xudong963 in #16478
• feat(query): parallel k way merge sort by @forsaken628 in #16340
• feat: add fast path for cross join to optimize tpcds q23 (~50x) by @xudong963 in #16504
• feat(catalog): ignore error when listing databases by @BohuTANG in #16514
• feat: limit stage name to plain_identifier by @youngsofun in #16522
• feat(query): Initial implementation of postgres-compatible number to_char function by @forsaken628 in #16507
• feat(query): Procedure support drop if exists and create or replace by @TCeason in #16500
• feat(query): DATE_ADD Functions to support week as a unit by @TracyZYJ in #16530
• feat: http handler add X-DATABEND-VERSION in each response. by @youngsofun in #16518
• feat: short sql length setting by @arkzuse in #16502
• feat: unset table options by @dantengsky in #16544
• feat(query): window partition by spill to disk by @forsaken628 in #16441
• feat(query): Simplified version Pattern S/L/D/G for number to_char by @forsaken628 in #16569
• feat(query): add json array lambda functions by @sundy-li in #16573
• feat(query): support describe user statement by @b41sh in #16579
• feat: copy option 'pattern' support variable. by @youngsofun in #16525
• feat(query): support json array functions by @b41sh in #16571
• feat(query): add date_diff function by @chagelo in #16560
• feat(query): Reduce serialisation memory usage when spilling to local disk by @forsaken628 in #16580
• feat(query): Support show dictionaries DQL. by @Winnie-Hong0927 in #16602
• feat: support sampling table with block and row level simultaneously by @xudong963 in #16613
• feat: add unload options overwrite, include_query_id and use_raw_path. by @youngsofun in #16614
• feat(function): Add mode Aggregate function. by @Freejww in #16627
• feat(query): Support use parquet format when spilling by @forsaken628 in #16612
• feat(query): support json_object_insert function by @b41sh in #16636
• feat: http handler support request forwarding. by @youngsofun in #16637
• feat: Implemented a new sql explain analyze graphical by @Maricaya in #16543
• feat: add deterministic block level sampling for small datasets by @xudong963 in #16670
• feat: support explain decorrelated plan by @xudong963 in #16681
• feat: add ARRAY_SIZE alias for ARRAY_LENGTH by @ding-young in #16685
• feat(query): support json_object_delete and json_object_pick function by @b41sh in #16682
• feat: Bump OpenDAL to 0.50.1 by @Xuanwo in #16661
• feat: support map lambda functions by @Dragonliu2018 in #16683
• feat(query): add api /v1/verify by @everpcpc in #16665
• feat: support calculating table storage size(including time travel) by @SkyFan2002 in #16698
• feat(query): add table function udf_echo && expose more metrics of udf client by @sundy-li in #16696
• feat(query): Settings clause by @TCeason in #16669
• feat: implement is_not_null selectivity based on null count in stats by @xudong963 in #16730
• feat: HTTP handler use cookie to pass session id to and from clients. by @youngsofun in #16735
• feat: impl opendal::HttpFetch for StorageHttpClient by @everpcpc in #16738
• feat(query): function about convert_timezone #16177 by @florann in #16181
• feat(functions): add new function: map_pick by @hanxuanliang in #15573
• feat(metrics): new storage_http_...

v1.2.680-nightly

What's Changed

Others 📒

• chore(query): fix udf metrics by @sundy-li in #17134

Full Changelog: v1.2.679-nightly...v1.2.680-nightly

v1.2.679-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(log): prefix_filter not working. by @youngsofun in #17136

Code Refactor 🎉

• refactor: simplify alter and drop cluster key logic by @zhyass in #17128
• refactor: integrate segment pruning into pipeline by @dqhl76 in #17126

Others 📒

• chore: rename default_cluster_key_id to cluster_key_seq by @zhyass in #17135

Full Changelog: v1.2.678-nightly...v1.2.679-nightly

v1.2.678-nightly

What's Changed

Exciting New Features ✨

• feat: support vacuum leaked table data (#17022)-Revert by @dantengsky in #17123

Thoughtful Bug Fix 🔧

• fix(query): fix udf script return type is arrow DataType::LargeList caused failed by @b41sh in #17127
• fix: ignore case when matching function name by @notauserx in #16912
• fix: analyze table contain keyword bug by @zhyass in #17131
• fix(query): Disabled the shrink_scalar optimization for cast expressions during the resolve procedure args by @TCeason in #17119

Others 📒

• chore(query): bump federated mysql version by @TCeason in #17118
• chore(query): reduce-functions-code-expand by @sundy-li in #17114
• chore(query): improve hook vacuum temp after query by @sundy-li in #17125
• chore(query): bump enum-as-inner 0.6 by @forsaken628 in #17120

New Contributors

• @notauserx made their first contribution in #16912

Full Changelog: v1.2.677-nightly...v1.2.678-nightly

v1.2.677-nightly

What's Changed

Exciting New Features ✨

• feat(query): refactor vacuum temp files by @sundy-li in #17089
• refactor: Serialize iceberg table inside engine options to reduce catalog requests by @Xuanwo in #17049
• feat(query): support interval function by @TCeason in #17093
• feat: new settings stream_consume_batch_size_hint by @dantengsky in #17102
• feat(query): support javascript/python script User Defined Aggregate Function by @forsaken628 in #17054
• feat: databend-meta watch API provides initial_flush field by @drmingdrmer in #17098
• feat(query): support javascript/python script User Defined Aggregate Function by @forsaken628 in #17108
• feat(query): support interval 'string' by @TCeason in #17105
• feat(query): Add external_block_rows metrics by @b41sh in #17116

Thoughtful Bug Fix 🔧

• fix(query): union all panic in mysql client by @zhyass in #17095
• fix: name resolution bug in case-sensitive mode by @zhyass in #17097
• fix(query): fix a number of bugs in the spill config. by @forsaken628 in #17103
• fix(query): uri path should not be percent-encoded by @youngsofun in #17109
• fix(query): fix default values of SpillConfig may not take effect by @forsaken628 in #17113
• fix(query): table field data type in show create table incorrect by @forsaken628 in #17112

Code Refactor 🎉

• refactor(query): refactor udf server batch rows by @b41sh in #17092
• refactor: set name for all runtime worker threads. by @youngsofun in #17084
• refactor(query): refactor udf server connection reuse endpoint by @b41sh in #17106

Others 📒

• chore(query): Revert "feat(query): support javascript/python script User Defined Aggregate Function" by @sundy-li in #17107
• chore: add more log by @SkyFan2002 in #17110
• chore(planner): fix except and intersect operation by @Dousir9 in #17111
• chore(query): add fuzz tests by @sundy-li in #17100

Full Changelog: v1.2.676-nightly...v1.2.677-nightly

v1.2.676-nightly

What's Changed

Exciting New Features ✨

• feat: databend-meta transaction support generic bool-expression and else-if chain by @drmingdrmer in #17064

Thoughtful Bug Fix 🔧

• fix: check if the table option is valid according to the engine by @SkyFan2002 in #17076
• fix(ci): flaky test by @zhyass in #17090
• fix(query): double panic if broken state in block builder by @zhang2014 in #17091
• fix(query): Correct sign extension handling in months_days_micros struct by @TCeason in #17086

Others 📒

• chore: remove unused dev by @zhyass in #17088

Full Changelog: v1.2.675-nightly...v1.2.676-nightly

v1.2.675-nightly

What's Changed

Exciting New Features ✨

• feat(query): support global network policy by @everpcpc in #17050
• feat(query): add config network_policy_whitelist by @everpcpc in #17065
• feat(query): pre-check cast type by @andylokandy in #16836
• feat(functions): add jaro_winkler string similarity function by @maxjustus in #16993
• feat(query): add config jwks_refresh_interval & jwks_refresh_timeout by @everpcpc in #17087

Thoughtful Bug Fix 🔧

• fix(query): fix read small parquet files in cluster mode by @sundy-li in #17063
• fix(query): fix thread leak by @zhang2014 in #17077
• fix(query): fix thread leak and remove all usage by @dqhl76 in #17081
• fix: change tracking stream failed by @zhyass in #17072
• fix(query): fix join on not null column cause panic by @b41sh in #17069
• fix: create as query respect ddl_column_type_nullable by @TCeason in #17071

Others 📒

• chore: refmt crash log message by @TCeason in #17059
• chore: bump dtparse by @TCeason in #17073

Full Changelog: v1.2.674-nightly...v1.2.675-nightly

v1.2.674-nightly

What's Changed

Exciting New Features ✨

• feat: add key prefix count assertions to transactions by @drmingdrmer in #17053
• feat(query): null type infer as Nullabel(String) by default in create as statement by @TCeason in #17052

Others 📒

• chore(binder): fix bind materialized cte by @Dousir9 in #17046

Full Changelog: v1.2.673-nightly...v1.2.674-nightly