v1.2.874-nightly

What's Changed

Exciting New Features ✨

• feat: support independent schema for table branches by @zhyass in #19269
• feat: preliminary support for common subexpression elimination by @SkyFan2002 in #19351
• feat: CSV support multi-bytes field delimiter. by @youngsofun in #19355
• feat: support LATERAL for generate_series by @KKould in #19349
• feat: Sandbox UDF by @KKould in #19301
• feat(query): Support Geography functions by @b41sh in #19320
• feat: support prune columns for materialized cte by @SkyFan2002 in #19362

Thoughtful Bug Fix 🔧

• fix(query): respect null ordering in topn pruner by @sundy-li in #19317
• fix: enhance feature verification logic by @dqhl76 in #19359

Code Refactor 🎉

• refactor: extract HTTP admin API to standalone crate databend-meta-admin by @drmingdrmer in #19335
• refactor: group gRPC and admin config fields into nested structs by @drmingdrmer in #19336
• refactor: use TlsConfig in HttpServiceConfig by @drmingdrmer in #19337
• refactor: embed KvApiArgs in Config by @drmingdrmer in #19338
• refactor: move CLI config parsing to separate crate by @drmingdrmer in #19339
• refactor: remove databend-common-base dependency from raft-store by @drmingdrmer in #19341
• refactor: move CLI fields out of MetaServiceConfig into MetaConfig by @drmingdrmer in #19342
• refactor: replace local count module with databend-base counter by @drmingdrmer in #19343
• refactor: add trace_request to SpawnApi for server-side tracing by @drmingdrmer in #19346
• refactor: simplify port allocation in meta-store to use OS-assigned ports by @drmingdrmer in #19352
• refactor(meta): reorganize tests and use lightweight runtime for meta service by @drmingdrmer in #19354
• refactor: move HTTP endpoint tests to admin crate by @drmingdrmer in #19356
• refactor: move module tag registration from client to store crate by @drmingdrmer in #19365
• refactor: add connect method to SpawnApi for channel creation by @drmingdrmer in #19370
• refactor: decouple meta core crates from databend dependencies by @drmingdrmer in #19372
• refactor: remove databend-common-meta-api dependency from meta-service by @drmingdrmer in #19374

Documentation 📔

• docs: update meta README to reflect current directory structure by @drmingdrmer in #19348
• docs: rebrand Databend as AI agent-ready enterprise data warehouse by @bohutang in #19379

Others 📒

• chore: set correct engine name for StageTable and StageSinkTable by @SkyFan2002 in #19347
• chore(query): replacing closures in FunctionEval with traits by @forsaken628 in #19316
• chore(sqllogic): add tutorial test by @TCeason in #19353
• chore: fix lint by @drmingdrmer in #19361
• chore: fix lint by @drmingdrmer in #19363
• chore: fix stateless test timeout by @b41sh in #19367
• chore: fix lint by @drmingdrmer in #19371

Full Changelog: v1.2.873-nightly...v1.2.874-nightly

v1.2.873-nightly

What's Changed

Code Refactor 🎉

• refactor: decouple meta-service from databend-common-base runtime by @drmingdrmer in #19319
• refactor: remove MetaStorageError and delete stoerr crate by @drmingdrmer in #19322
• refactor: use DatabendRuntime::spawn in meta binaries by @drmingdrmer in #19324
• refactor: make meta-client generic over RuntimeApi by @drmingdrmer in #19327
• refactor: add SpawnApi::prepare_request() for gRPC metadata injection by @drmingdrmer in #19329
• refactor: move 7 server-side meta crates to src/meta/core/ by @drmingdrmer in #19332
• refactor: remove environment variable configuration for metasrv by @drmingdrmer in #19333

Documentation 📔

• docs: clean up and improve README formatting by @bohutang in #19331

Others 📒

• chore: adjust lazy read setting by @SkyFan2002 in #19318

Full Changelog: v1.2.872-nightly...v1.2.873-nightly

v1.2.872-nightly

What's Changed

Exciting New Features ✨

• feat(query): add nested loop join for new experimental hash join by @forsaken628 in #18961
• feat: support prewhere for fuse parquet by @SkyFan2002 in #19209
• feat(query): Support CREATE/DROP SPATIAL INDEX (Geometry/Geography types) by @b41sh in #19314

Thoughtful Bug Fix 🔧

• ci(test): enable profile configuration during test runs by @dqhl76 in #19245
• fix: preserve IO error kind in codec decoder for raft log recovery by @drmingdrmer in #19315
• fix: drop/replace stage fail to delete files on OSS. by @youngsofun in #19321

Code Refactor 🎉

• refactor(query): refactor hybrid hash join for new join by @zhang2014 in #19307
• refactor(query): optimize system tables filter with lightweight permission check by @TCeason in #19293

Build/Testing/CI Infra Changes 🔌

• ci: reopen compat tests for py driver. by @youngsofun in #19310

Full Changelog: v1.2.871-nightly...v1.2.872-nightly

v1.2.871-nightly

What's Changed

Exciting New Features ✨

• feat: add table function for inspecting Parquet pages by @SkyFan2002 in #19278
• feat: introducing DataBlockVec to optimize multiple data block take by @forsaken628 in #19249
• feat: eliminate constant subquery by @KKould in #19289
• feat: enhance binary input and output format by @KKould in #19246
• feat: enable lazy materialization across joins with outer-join NULL handling by @SkyFan2002 in #19295

Thoughtful Bug Fix 🔧

• fix: compile error by @SkyFan2002 in #19298
• fix: add TrackingPayloadExt trait for ergonomic future tracking by @drmingdrmer in #19296
• fix: drain gRPC stream in get_kv() to prevent h2 reset errors by @drmingdrmer in #19302
• fix(cloud): clamp task run pagination by @sundy-li in #19297
• fix: in_memory_table_data_cache should use memory size of array instead of buffer size. by @youngsofun in #19305
• fix: cluster key consistency during ALTER TABLE column by @zhyass in #19300
• chore(query): reapply iceberg bump by @sundy-li in #19304

Code Refactor 🎉

• refactor: use hickory-resolver directly in raft-store for DNS resolution by @drmingdrmer in #19288
• refactor: replace custom JoinHandle wrapper with From impl by @drmingdrmer in #19291
• refactor(query): support pushdown rules with SecureFilter for row access policy by @TCeason in #19274
• refactor: use io::Error instead of MetaStorageError in sled-store by @drmingdrmer in #19292

Full Changelog: v1.2.870-nightly...v1.2.871-nightly

v1.2.870-nightly

What's Changed

Exciting New Features ✨

• feat: table branch support insert by @zhyass in #19225
• feat: adjust columns of system.stages and show stages by @youngsofun in #19257

Thoughtful Bug Fix 🔧

• fix: out of bound on recursive cte & coercion_types left right not match on recursive scan by @KKould in #19212
• fix: send responses only after state machine commit by @drmingdrmer in #19286

Code Refactor 🎉

• refactor(meta): decouple meta crates and improve stream error handling by @drmingdrmer in #19263
• refactor: simplify location capture in location_future by @drmingdrmer in #19267
• refactor: move Pool from common-base to meta-client by @drmingdrmer in #19273
• refactor: remove implicit From<MetaError> for ErrorCode trait by @drmingdrmer in #19275
• refactor: simplify Pool struct and retry loop by @drmingdrmer in #19276
• refactor: replace BuildInfoRef with semver::Version in meta-client by @drmingdrmer in #19277
• refactor: simplify KVAppError handling with into_nested() by @drmingdrmer in #19280
• refactor: replace local uniq_id and drop_callback with databend-base by @drmingdrmer in #19279
• refactor: replace BuildInfoRef with semver::Version in RpcClientConf by @drmingdrmer in #19281
• refactor: simplify Runtime by extracting common spawn and builder logic by @drmingdrmer in #19282
• refactor: use anyhow::Result for test function return types by @drmingdrmer in #19283
• refactor: simplify control flow in Runtime methods by @drmingdrmer in #19285

Others 📒

• chore: update version for kv_list and kv_get_many features by @drmingdrmer in #19270
• chore: remove sqllogicttest warnings by @KKould in #19256

Full Changelog: v1.2.869-nightly...v1.2.870-nightly

v1.2.869-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: roll back h2 upgrade and ineffective reset workarounds by @bohutang in #19264

Code Refactor 🎉

• refactor: support row/bucket shuffle for aggregation by @dqhl76 in #19155

Full Changelog: v1.2.868-nightly...v1.2.869-nightly

v1.2.868-nightly

What's Changed

Exciting New Features ✨

• feat: add configurable connection TTL for meta client by @bohutang in #19259

Thoughtful Bug Fix 🔧

• fix: stream diff SQL alias generation for quoted columns by @zhyass in #19258

Code Refactor 🎉

• refactor(meta): remove unused KVApi wrappers and traits by @drmingdrmer in #19251
• refactor: rename get_kv_stream to get_many_kv with streaming input by @drmingdrmer in #19254

Full Changelog: v1.2.867-nightly...v1.2.868-nightly

v1.2.867-nightly

What's Changed

Exciting New Features ✨

• feat(iceberg): bump iceberg-rust to v0.8.0 and add write support by @sundy-li in #19200
• feat(query): support tag_reference table function by @TCeason in #19221
• feat(meta): add KvList gRPC API for listing keys by prefix by @drmingdrmer in #19235
• feat: add limit parameter to KVApi::list_kv() and related methods by @drmingdrmer in #19240
• feat(meta): add KvGetMany gRPC API; refactor: split chained awaits by @drmingdrmer in #19244
• feat: support alter stage. by @youngsofun in #19236
• feat(ci): add sha input to release workflow for custom commit targeting by @bohutang in #19250

Thoughtful Bug Fix 🔧

• fix(optimizer): add recursive identifier to prevent stack overflow by @sundy-li in #19228
• fix(ci): make ci run databend query unit test by @TCeason in #19233
• fix(query): fix rust-analyzer cyclic deps between common-functions and common-formats by @TCeason in #19227
• fix(storage): prevent parquet page overflow by splitting large batches by @dantengsky in #19206
• fix(query): normalize type names in DESC PROCEDURE by @TCeason in #19237
• fix(test): ignore explain memo cost by @dqhl76 in #19238
• fix(meta): vacuum dropped table cleans policy refs by @TCeason in #19239
• fix(ci): increase pool_max_idle_per_host by @sundy-li in #19241
• fix(test): force HTTP/1.1 for sqllogictests client by @sundy-li in #19243

Code Refactor 🎉

• refactor(query): restructure hash join memory implementations by @zhang2014 in #19199
• refactor: encapsulate list parameters into ListOptions struct by @drmingdrmer in #19242
• refactor(optimizer): refactor SelectivityEstimator by @forsaken628 in #19186

Others 📒

• chore: remove fastrace::trace attributes from test suite by @drmingdrmer in #19229
• chore: enhance EXPLAIN to display table branch qualifiers by @zhyass in #19232
• chore: add setting for table ref by @zhyass in #19234
• chore(query): bump ast to 0.2.4 and optimize gRPC configs by @sundy-li in #19253

Full Changelog: v1.2.866-nightly...v1.2.867-nightly

v1.2.866-nightly

What's Changed

Exciting New Features ✨

• feat: support variant cast to timestamp_tz by @KKould in #19190
• feat: initial support for table branching and tagging by @zhyass in #19035
• feat: to_decimal support more types by @KKould in #19195
• feat(query): Variant timestamp_tz offset support seconds by @b41sh in #19194
• feat(query): support ALTER SET/UNSET TAG = value by @TCeason in #19197
• feat: database level default connection configuration by @dantengsky in #18886
• feat(formats): trim trailing zeros from decimal output in CSV/TSV exports by @sundy-li in #19211
• feat: add request latency histogram tracking for meta service by @drmingdrmer in #19215
• feat(query): support scalar subquery arguments in table functions by @sundy-li in #19213
• fix: distinguish quoted and unquoted empty and null string when loading CSV by @youngsofun in #19207

Thoughtful Bug Fix 🔧

• fix: use full join keys for hash shuffle to avoid skew by @dqhl76 in #19198
• fix(query): enable query result cache for queries with scalar subqueries by @TCeason in #19202
• fix: auto-increment and computed columns in multi-table INSERT by @zhyass in #19205
• fix(query): Ensure result cache keys include secure filter predicates by @TCeason in #19210
• fix(query): fix parse float number error by @b41sh in #19223

Code Refactor 🎉

• refactor(query): rename skip duplicate hash tables to unique by @zhang2014 in #19192
• refactor(meta): upgrade openraft to v0.10.0-alpha.13 by @drmingdrmer in #19193
• refactor(query): reduce string allocations in hot paths by @sundy-li in #19201
• refactor: extract SchemaApiTestSuite to standalone crate by @drmingdrmer in #19219
• refactor(query): use snapshot_location for DummyTableScan cache invalidation by @TCeason in #19214
• refactor: split SchemaTestSuite into several sub tests by @drmingdrmer in #19224

Others 📒

• chore(query): clean up the useless count_distinct by @forsaken628 in #19191

Full Changelog: v1.2.865-nightly...v1.2.866-nightly

v1.2.865-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: avoid committing new snapshot if table not changed by @dantengsky in #19174

Others 📒

• chore: fix misleading comment about CTAS and allow_append_only_skip by @dantengsky in #19189

Full Changelog: v1.2.864-nightly...v1.2.865-nightly