Update dependency org.rocksdb:rocksdbjni to v10

[renovate]

Note: This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
org.rocksdb:rocksdbjni (source)	6.29.5 -> 10.4.2

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

---

Release Notes

facebook/rocksdb (org.rocksdb:rocksdbjni)

v10.4.2

10.4.2 (07/09/2025)

Bug Fixes

• Fix a race condition between concurrent DB::Open sharing the same SstFileManager instance.

10.4.1 (07/01/2025)

Behavior Changes

• RocksDB now triggers eligible compactions every 12 hours when periodic compaction is configured. This solves a limitation of the compaction trigger mechanism, which would only trigger compaction after specific events like flush, compaction, or SetOptions.

Bug Fixes

• Fix a bug in BackupEngine that can crash backup due to a null FSWritableFile passed to WritableFileWriter.

10.4.0 (06/20/2025)

New Features

• Add a new CF option memtable_avg_op_scan_flush_trigger that supports triggering memtable flush when an iterator scans through an expensive range of keys, with the average number of skipped keys from the active memtable exceeding the threshold.
• Vector based memtable now supports concurrent writers (DBOptions::allow_concurrent_memtable_write) #​13675.
• Add new experimental TransactionOptions::large_txn_commit_optimize_byte_threshold to enable optimizations for large transaction commit by transaction batch data size.
• Add a new option CompactionOptionsUniversal::reduce_file_locking and if it's true, auto universal compaction picking will adjust to minimize locking of input files when bottom priority compactions are waiting to run. This can increase the likelihood of existing L0s being selected for compaction, thereby improving write stall and reducing read regression.
• Add new format_version=7 to aid experimental support of custom compression algorithms with CompressionManager and block-based table. This format version includes changing the format of TableProperties::compression_name.

Public API Changes

• Change NewExternalTableFactory to return a unique_ptr instead of shared_ptr.
• Add an optional min file size requirement for deletion triggered compaction. It can be specified when creating CompactOnDeletionCollectorFactory.

Behavior Changes

• TransactionOptions::large_txn_commit_optimize_threshold now has default value 0 for disabled. TransactionDBOptions::txn_commit_bypass_memtable_threshold now has no effect on transactions.

Bug Fixes

• Fix a bug where CreateColumnFamilyWithImport() could miss the SST file for the memtable flush it triggered. The exported CF then may not contain the updates in the memtable when CreateColumnFamilyWithImport() is called.
• Fix iterator operations returning NotImplemented status if disallow_memtable_writes and paranoid_memory_checks CF options are both set.
• Fixed handling of file checksums in IngestExternalFile() to allow providing checksums using recognized but not necessarily the DB's preferred checksum function, to ease migration between checksum functions.

v10.2.1

10.2.1 (2025-04-24)

Bug Fixes

• Fix improper initialization of ExternalTableOptions

10.2.0 (2025-04-21)

New Features

• Provide histogram stats COMPACTION_PREFETCH_BYTES to measure number of bytes for RocksDB's prefetching (as opposed to file
  system's prefetch) on SST file during compaction read
• A new API DB::GetNewestUserDefinedTimestamp is added to return the newest user defined timestamp seen in a column family
• Introduce API IngestWriteBatchWithIndex() for ingesting updates into DB while bypassing memtable writes. This improves performance when writing a large write batch to the DB.
• Add a new CF option memtable_op_scan_flush_trigger that triggers a flush of the memtable if an iterator's Seek()/Next() scans over a certain number of invisible entries from the memtable.

Public API Changes

• AdvancedColumnFamilyOptions.max_write_buffer_number_to_maintain is deleted. It's deprecated since introduction of a better option max_write_buffer_size_to_maintain since RocksDB 6.5.0.
• Deprecated API DB::MaxMemCompactionLevel().
• Deprecated ReadOptions::ignore_range_deletions.
• Deprecated API experimental::PromoteL0().
• Added arbitrary string map for additional options to be overriden for remote compactions
• The fail_if_options_file_error option in DBOptions has been removed. The behavior now is to always return failure in any API that fails to persist the OPTIONS file.

Behavior Changes

• Make stats PREFETCH_BYTES_USEFUL, PREFETCH_HITS, PREFETCH_BYTES only account for prefetching during user initiated scan

Bug Fixes

• Fix a bug in Posix file system that the FSWritableFile created via FileSystem::ReopenWritableFile internally does not track the correct file size.
• Fix a bug where tail size of remote compaction output is not persisted in primary db's manifest

v10.1.3

10.1.3 (2025-04-09)

Bug Fixes

• Fix a bug where resurrected full_history_ts_low from a previous session that enables UDT is used by this session that disables UDT.

10.1.2 (2025-04-07)

Bug Fixes

• Fix a bug where tail size of remote compaction output is not persisted in primary db's manifest

10.1.0 (2025-03-24)

New Features

• Added a new DBOptions.calculate_sst_write_lifetime_hint_set setting that allows to customize which compaction styles SST write lifetime hint calculation is allowed on. Today RocksDB supports only two modes kCompactionStyleLevel and kCompactionStyleUniversal.
• Add a new field num_l0_files in CompactionJobInfo about the number of L0 files in the CF right before and after the compaction
• Added per-key-placement feature in Remote Compaction
• Implemented API DB::GetPropertiesOfTablesByLevel that retrieves table properties for files in each LSM tree level

Public API Changes

• GetAllKeyVersions() now interprets empty slices literally, as valid keys, and uses new OptSlice type default value for extreme upper and lower range limits.
• DeleteFilesInRanges() now takes RangeOpt which is based on OptSlice. The overload taking RangePtr is deprecated.
• Add an unordered map of name/value pairs, ReadOptions::property_bag, to pass opaque options through to an external table when creating an Iterator.
• Introduced CompactionServiceJobStatus::kAborted to allow handling aborted scenario in Schedule(), Wait() or OnInstallation() APIs in Remote Compactions.
• format_version < 2 in BlockBasedTableOptions is no longer supported for writing new files. Support for reading such files is deprecated and might be removed in the future. CompressedSecondaryCacheOptions::compress_format_version == 1 is also deprecated.

Behavior Changes

• ldb now returns an error if the specified --compression_type is not supported in the build.
• MultiGet with snapshot and ReadOptions::read_tier = kPersistedTier will now read a consistent view across CFs (instead of potentially reading some CF before and some CF after a flush).
• CreateColumnFamily() is no longer allowed on a read-only DB (OpenForReadOnly())

Bug Fixes

• Fixed stats for Tiered Storage with preclude_last_level feature

v10.0.1: RocksDB 10.0.1 Release

10.0.1 (2025-03-05)

Public API Changes

• Add an unordered map of name/value pairs, ReadOptions::property_bag, to pass opaque options through to an external table when creating an Iterator.
• Introduced CompactionServiceJobStatus::kAborted to allow handling aborted scenario in Schedule(), Wait() or OnInstallation() APIs in Remote Compactions.
• Added a column family option disallow_memtable_writes to safely fail any attempts to write to a non-default column family. This can be used for column families that are ingest only.

10.0.0 (2025-02-21)

New Features

• Introduced new auto_refresh_iterator_with_snapshot opt-in knob that (when enabled) will periodically release obsolete memory and storage resources for as long as the iterator is making progress and its supplied read_options.snapshot was initialized with non-nullptr value.
• Added the ability to plug-in a custom table reader implementation. See include/rocksdb/external_table_reader.h for more details.
• Experimental feature: RocksDB now supports FAISS inverted file based indices via the secondary indexing framework. Applications can use FAISS secondary indices to automatically quantize embeddings and perform K-nearest-neighbors similarity searches. See FaissIVFIndex and SecondaryIndex for more details. Note: the FAISS integration currently requires using the BUCK build.
• Add new DB property num_running_compaction_sorted_runs that tracks the number of sorted runs being processed by currently running compactions
• Experimental feature: added support for simple secondary indices that index the specified column as-is. See SimpleSecondaryIndex and SecondaryIndex for more details.
• Added new TransactionDBOptions::txn_commit_bypass_memtable_threshold, which enables optimized transaction commit (see TransactionOptions::commit_bypass_memtable) when the transaction size exceeds a configured threshold.

Public API Changes

• Updated the query API of the experimental secondary indexing feature by removing the earlier SecondaryIndex::NewIterator virtual and adding a SecondaryIndexIterator class that can be utilized by applications to find the primary keys for a given search target.
• Added back the ability to leverage the primary key when building secondary index entries. This involved changes to the signatures of SecondaryIndex::GetSecondary{KeyPrefix,Value} as well as the addition of a new method SecondaryIndex::FinalizeSecondaryKeyPrefix. See the API comments for more details.
• Minimum supported version of ZSTD is now 1.4.0, for code simplification. Obsolete CompressionType kZSTDNotFinalCompression is also removed.

Behavior Changes

• VerifyBackup in verify_with_checksum=true mode will now evaluate checksums in parallel. As a result, unlike in case of original implementation, the API won't bail out on a very first corruption / mismatch and instead will iterate over all the backup files logging success / degree_of_failure for each.
• Reversed the order of updates to the same key in WriteBatchWithIndex. This means if there are multiple updates to the same key, the most recent update is ordered first. This affects the output of WBWIIterator. When WriteBatchWithIndex is created with overwrite_key=true, this affects the output only if Merge is used (#​13387).
• Added support for Merge operations in transactions using option TransactionOptions::commit_bypass_memtable.

Bug Fixes

• Fixed GetMergeOperands() API in ReadOnlyDB and SecondaryDB
• Fix a bug in GetMergeOperands() that can return incorrect status (MergeInProgress) and incorrect number of merge operands. This can happen when GetMergeOperandsOptions::continue_cb is set, both active and immutable memtables have merge operands and the callback stops the look up at the immutable memtable.

v9.11.2: RocksDB 9.11.2 Release

Rocksdb Change Log

NOTE: Entries for next release do not go here. Follow instructions in unreleased_history/README.txt

9.11.2 (2025-03-29)

Bump patch version to fix a mistake in the previous 9.11 release tag

9.11.1 (2025-02-19)

New Features

• Added the ability to plug-in a custom table reader implementation. See include/rocksdb/external_table_reader.h for more details.

9.11.0 (2025-01-17)

New Features

• Introduce CancelAwaitingJobs() in CompactionService interface which will allow users to implement cancellation of running remote compactions from the primary instance
• Experimental feature: RocksDB now supports defining secondary indices, which are automatically maintained by the storage engine. Secondary indices provide a new customization point: applications can provide their own by implementing the new SecondaryIndex interface. See the SecondaryIndex API comments for more details. Note: this feature is currently only available in conjunction with write-committed pessimistic transactions, and Merge is not yet supported.
• Provide a new option track_and_verify_wals to track and verify various information about WAL during WAL recovery. This is intended to be a better replacement to track_and_verify_wals_in_manifest.

Public API Changes

• Add io_buffer_size to BackupEngineOptions to enable optimal configuration of IO size
• Clean up all the references to random_access_max_buffer_size, related rules and all the clients wrappers. This option has been officially deprecated in 5.4.0.
• Add file_ingestion_nanos and file_ingestion_blocking_live_writes_nanos in PerfContext to observe file ingestions
• Offer new DB::Open and variants that use std::unique_ptr<DB>* output parameters and deprecate the old versions that use DB** output parameters.
• The DB::DeleteFile API is officially deprecated.

Behavior Changes

• For leveled compaction, manual compaction (CompactRange()) will be more strict about keeping compaction size under max_compaction_bytes. This prevents overly large compactions in some cases (#​13306).
• Experimental tiering options preclude_last_level_data_seconds and preserve_internal_time_seconds are now mutable with SetOptions(). Some changes to handling of these features along with long-lived snapshots and range deletes made this possible.

Bug Fixes

• Fix a longstanding major bug with SetOptions() in which setting changes can be quietly reverted.

v9.10.0

New Features

• Introduce TransactionOptions::commit_bypass_memtable to enable transaction commit to bypass memtable insertions. This can be beneficial for transactions with many operations, as it reduces commit time that is mostly spent on memtable insertion.

Public API Changes

• Deprecated Remote Compaction APIs (StartV2, WaitForCompleteV2) are completely removed from the codebase

Behavior Changes

• DB::KeyMayExist() now follows its function comment, which means value parameter can be null, and it will be set only if value_found is passed in.

Bug Fixes

• Fix the issue where compaction incorrectly drops a key when there is a snapshot with a sequence number of zero.
• Honor ConfigOptions.ignore_unknown_options in ParseStruct()

Performance Improvements

• Enable reuse of file system allocated buffer for synchronous prefetching.
• In buffered IO mode, try to align writes on power of 2 if checksum handoff is not enabled for the file type being written.

v9.9.3: RocksDB release 9.9.3

9.9.3 (2024-12-03)

Performance Improvements

• In buffered IO mode, try to align writes on power of 2 if checksum handoff is not enabled for the file type being written.

9.9.2 (2024-11-22)

Bug Fixes

• Honor ConfigOptions.ignore_unknown_options in ParseStruct()

9.9.1 (2024-11-30)

Behavior Changes

• Updates the hidden hook RocksDbThreadYieldAndCheckAbort() to support MySQL to abort long-running query.

9.9.0 (2024-11-18)

New Features

• Multi-Column-Family-Iterator (CoalescingIterator/AttributeGroupIterator) is no longer marked as experimental
• Adds a new table property "rocksdb.newest.key.time" which records the unix timestamp of the newest key. Uses this table property for FIFO TTL and temperature change compaction.

Public API Changes

• Added a new API Transaction::GetAttributeGroupIterator that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.
• Added a new API Transaction::GetCoalescingIterator that can be used to create a multi-column-family coalescing iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.

Behavior Changes

• BaseDeltaIterator now honors the read option allow_unprepared_value.

Bug Fixes

• BaseDeltaIterator now calls PrepareValue on the base iterator in case it has been created with the allow_unprepared_value read option set. Earlier, such base iterators could lead to incorrect values being exposed from BaseDeltaIterator.
• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.
• Fix missing cases of corruption retry during DB open and read API processing.
• Fix a bug for transaction db with 2pc where an old WAL may be retained longer than needed (#​13127).
• Fix leaks of some open SST files (until DB::Close()) that are written but never become live due to various failures. (We now have a check for such leaks with no outstanding issues.)
• Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions.

Performance Improvements

• Fix regression in issue #​12038 due to Options::compaction_readahead_size greater than max_sectors_kb (i.e, largest I/O size that the OS issues to a block device defined in linux)

v9.8.4: RocksDB 9.8.4

9.8.4 (2024-11-18)

Behavior Changes

• When Remote Compaction is enabled, do not purge OPTIONS file immediately by DeleteObsoleteOptionsFiles() after SetOptions(). Rely on PurgeObsoleteFiles() to clean up obsolete OPTIONS file after each compaction.

9.8.3 (2024-11-12)

Bug Fixes

• Fix missing cases of corruption retry during DB open and read API processing.

9.8.2 (2024-11-06)

Public API Changes

• Added a new API Transaction::GetAttributeGroupIterator that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.

Behavior Changes

• BaseDeltaIterator now honors the read option allow_unprepared_value.

Bug Fixes

• BaseDeltaIterator now calls PrepareValue on the base iterator in case it has been created with the allow_unprepared_value read option set. Earlier, such base iterators could lead to incorrect values being exposed from BaseDeltaIterator.
• Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions.

9.8.1 (2024-10-31)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version
  9.4.0.

9.8.0 (2024-10-25)

New Features

• All non-block_cache options in BlockBasedTableOptions are now mutable with DB::SetOptions().
  See also Bug Fixes below.
• When using iterators with BlobDB, it is now possible to load large values on an on-demand basis, i
  .e. only if they are actually needed by the application. This can save I/O in use cases where the va
  lues associated with certain keys are not needed. For more details, see the new read option allow_u nprepared_value and the iterator API PrepareValue.
• Add a new file ingestion option IngestExternalFileOptions::fill_cache to support not adding bloc
  ks from ingested files into block cache during file ingestion.
• The option allow_unprepared_value is now also supported for multi-column-family iterators (i.e.
  CoalescingIterator and AttributeGroupIterator).
• When a file with just one range deletion (standalone range deletion file) is ingested via bulk loa
  ding, it will be marked for compaction. During compaction, this type of files can be used to directl
  y filter out some input files that are not protected by any snapshots and completely deleted by the
  standalone range deletion file.

Behavior Changes

• During file ingestion, overlapping files level assignment are done in multiple batches, so that th
  ey can potentially be assigned to lower levels other than always land on L0.
• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the
  primary host. A similar technique as how we are preserving new SST files from getting purged is used
  for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.
• Trim readahead_size during scans so data blocks containing keys that are not in the same prefix as
  the seek key in Seek() are not prefetched when ReadOptions::auto_readahead_size=true (default v
  alue) and ReadOptions::prefix_same_as_start = true
• Assigning levels for external files are done in the same way for universal compaction and leveled
  compaction. The old behavior tends to assign files to L0 while the new behavior will assign the file
  s to the lowest level possible.

Bug Fixes

• Fix a longstanding race condition in SetOptions for block_based_table_factory options. The fix h
  as some subtle behavior changes because of copying and replacing the TableFactory on a change with S
  etOptions, including requiring an Iterator::Refresh() for an existing Iterator to use the latest opt
  ions.
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the com
  pressed block size rather than the actual memory allocated, which could be larger due to internal fr
  agmentation.
• GetApproximateMemTableStats() could return disastrously bad estimates 5-25% of the time. The fun
  ction has been re-engineered to return much better estimates with similar CPU cost.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option
  is kVolatileTier.
• Fix an issue in level compaction where a small CF with small compaction debt can cause the DB to a
  llow parallel compactions. (#​13054)
• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as w
  ell. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_ identity_file, prefix_seek_opt_in_only

v9.7.4: RocksDB 9.7.4

9.7.4 (2024-10-31)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.

9.7.3 (2024-10-16)

Behavior Changes

• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.

9.7.2 (2024-10-08)

Bug Fixes

• Fix a bug for surfacing write unix time: Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.

9.7.1 (2024-09-26)

Bug Fixes

• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as well. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_identity_file, prefix_seek_opt_in_only
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the compressed block size rather than the actual memory allocated, which could be larger due to internal fragmentation.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option is kVolatileTier.

9.7.0 (2024-09-20)

New Features

• Make Cache a customizable class that can be instantiated by the object registry.
• Add new option prefix_seek_opt_in_only that makes iterators generally safer when you might set a prefix_extractor. When prefix_seek_opt_in_only=true, which is expected to be the future default, prefix seek is only used when prefix_same_as_start or auto_prefix_mode are set. Also, prefix_same_as_start and auto_prefix_mode now allow prefix filtering even with total_order_seek=true.
• Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion.

Behavior Changes

• Changed the semantics of the BlobDB configuration option blob_garbage_collection_force_threshold to define
  a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (accordin
  g to blob_garbage_collection_age_cutoff). This can provide better control over space amplification at the cos
  t of slightly higher write amplification.
• Set write_dbid_to_manifest=true by default. This means DB ID will now be preserved through backups, checkpo
  ints, etc. by default. Also add write_identity_file option which can be set to false for anticipated future b
  ehavior.
• In FIFO compaction, compactions for changing file temperature (configured by option file_temperature_age_thr esholds) will compact one file at a time, instead of merging multiple eligible file together (#​13018).
• Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files a
  nd IngestExternalFileOptions::allow_db_generated_files.
• Add a new file ingestion option IngestExternalFileOptions::link_files to hard link input files and preserve
  original files links after ingestion.
• DB::Close now untracks files in SstFileManager, making avaialble any space used
  by them. Prior to this change they would be orphaned until the DB is re-opened.

Bug Fixes

• Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only h
  appen when users configure CompactRangeOptions::change_level to true and the change level step of manual compac
  tion fails (#​13009).
• Fix handling of dynamic change of prefix_extractor with memtable prefix filter. Previously, prefix seek cou
  ld mix different prefix interpretations between memtable and SST files. Now the latest prefix_extractor at th
  e time of iterator creation or refresh is respected.
• Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsisten
  t (#​12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disa
  bles auto error recovery from these errors.

v9.7.3: RocksDB 9.7.3

9.7.3 (2024-10-16)

Behavior Changes

• OPTIONS file to be loaded by remote worker is now preserved so that it does not get purged by the primary host. A similar technique as how we are preserving new SST files from getting purged is used for this. min_options_file_numbers_ is tracked like pending_outputs_ is tracked.

9.7.2 (2024-08-10)

Bug Fixes

• Fix a bug for surfacing write unix time: Iterator::GetProperty("rocksdb.iterator.write-time") for non-L0 files.

9.7.1 (2024-09-26)

Bug Fixes

• Several DB option settings could be lost through GetOptionsFromString(), possibly elsewhere as well. Affected options, now fixed:background_close_inactive_wals, write_dbid_to_manifest, write_identity_file, prefix_seek_opt_in_only
• Fix under counting of allocated memory in the compressed secondary cache due to looking at the compressed block size rather than the actual memory allocated, which could be larger due to internal fragmentation.
• Skip insertion of compressed blocks in the secondary cache if the lowest_used_cache_tier DB option is kVolatileTier.

9.7.0 (2024-09-20)

New Features

• Make Cache a customizable class that can be instantiated by the object registry.
• Add new option prefix_seek_opt_in_only that makes iterators generally safer when you might set a prefix_extractor. When prefix_seek_opt_in_only=true, which is expected to be the future default, prefix seek is only used when prefix_same_as_start or auto_prefix_mode are set. Also, prefix_same_as_start and auto_prefix_mode now allow prefix filtering even with total_order_seek=true.
• Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion.

Behavior Changes

• Changed the semantics of the BlobDB configuration option blob_garbage_collection_force_threshold to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to blob_garbage_collection_age_cutoff). This can provide better control over space amplification at the cost of slightly higher write amplification.
• Set write_dbid_to_manifest=true by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add write_identity_file option which can be set to false for anticipated future behavior.
• In FIFO compaction, compactions for changing file temperature (configured by option file_temperature_age_thresholds) will compact one file at a time, instead of merging multiple eligible file together (#​13018).
• Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files.
• Add a new file ingestion option IngestExternalFileOptions::link_files to hard link input files and preserve original files links after ingestion.
• DB::Close now untracks files in SstFileManager, making avaialble any space used
  by them. Prior to this change they would be orphaned until the DB is re-opened.

Bug Fixes

• Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#​13009).
• Fix handling of dynamic change of prefix_extractor with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest prefix_extractor at the time of iterator creation or refresh is respected.
• Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#​12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors.

v9.6.2: RocksDB 9.6.2

9.6.2 (10/31/2024)

Bug Fixes

• Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0.

9.6.1 (08/24/2024)

Bug Fixes

• Fix correctness of MultiGet across column families with user timestamp.

9.6.0 (08/19/2024)

New Features

• *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
• New option BlockBasedTableOptions::decouple_partitioned_filters should improve efficiency in serving read queries because filter and index partitions can consistently target the configured metadata_block_size. This option is currently opt-in.
• Introduce a new mutable CF option paranoid_memory_checks. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch
• Adds optional installation callback function for remote compaction

Behavior Changes

• There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
• Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
• Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
• Fixed a data race involving the background error status in unordered_write mode.
• *Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest
  file. #​12882
• Fix a bug where per kv checksum corruption may be ignored in MultiGet().
• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to
  be registered simultaneously, resulting in a crash or other unpredictable behavior.

v9.6.1: RocksDB 9.6.1

9.6.1 (2024-08-24)

Bug Fixes

• Fix correctness of MultiGet across column families with user timestamp.

9.6.0 (2024-08-19)

New Features

• *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
• New option BlockBasedTableOptions::decouple_partitioned_filters should improve efficiency in serving read queries because filter and index partitions can consistently target the configured metadata_block_size. This option is currently opt-in.
• Introduce a new mutable CF option paranoid_memory_checks. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch
• Adds optional installation callback function for remote compaction

Behavior Changes

• There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.
• Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
• Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
• Fixed a data race involving the background error status in unordered_write mode.
• *Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest file. #​12882
• Fix a bug where per kv checksum corruption may be ignored in MultiGet().
• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.

v9.5.2: RocksDB 9.5.2

9.5.2 (2024-08-13)

Bug Fixes

• Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.

Public API Changes

• Add ticker stats to count file read retries due to checksum mismatch

9.5.1 (2024-08-02)

Bug Fixes

• *Make DestroyDB supports slow deletion when it's configured in SstFileManager. The slow deletion is subject to the configured rate_bytes_per_sec, but not subject to the max_trash_db_ratio.

9.5.0 (2024-07-19)

Public API Changes

• Introduced new C API function rocksdb_writebatch_iterate_cf for column family-aware iteration over the contents of a WriteBatch
• Add support to ingest SST files generated by a DB instead of SstFileWriter. This can be enabled with experimental option IngestExternalFileOptions::allow_db_generated_files.

Behavior Changes

• When calculating total log size for the log_size_for_flush argument in CreateCheckpoint API, the size of the archived log will not be included to avoid unnecessary flush

Bug Fixes

• Fix a major bug in which an iterator using prefix filtering and SeekForPrev might miss data when the DB is using whole_key_filtering=false and partition_filters=true.
• Fixed a bug where OnErrorRecoveryBegin() is not called before auto recovery starts.
• Fixed a bug where event listener reads ErrorHandler's bg_error_ member without holding db mutex(#​12803).
• Fixed a bug in handling MANIFEST write error that caused the latest valid MANIFEST file to get deleted, resulting in the DB being unopenable.
• Fixed a race between error recovery due to manifest sync or write failure and external SST file ingestion. Both attempt to write a new manifest file, which causes an assertion failure.

Performance Improvements

• Fix an issue where compactions were opening table files and reading table properties while holding db mutex_.
• Reduce unnecessary filesystem queries and DB mutex acquires in creating backups and checkpoints.

v9.4.0

New Features

• Added a CompactForTieringCollectorFactory to auto trigger compaction for tiering use case.
• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the GetEntityForUpdate API.
• Added a new "count" command to the ldb repl shell. By default, it prints a count of keys in the database from start to end. The options --from= and/or --to= can be specified to limit the range.
• Add rocksdb_writebatch_update_timestamps, rocksdb_writebatch_wi_update_timestamps in C API.
• Add rocksdb_iter_refresh in C API.
• Add rocksdb_writebatch_create_with_params, rocksdb_writebatch_wi_create_with_params to create WB and WBWI with all options in C API

Public API Changes

• Deprecated names LogFile and VectorLogPtr in favor of new names WalFile and VectorWalPtr.
• Introduce a new universal compaction option CompactionOptionsUniversal::max_read_amp which allows user to define the limit on the number of sorted runs separately from the trigger for compaction (level0_file_num_compaction_trigger) #​12477.

Behavior Changes

• Inactive WALs are immediately closed upon being fully sync-ed rather than in a background thread. This is to ensure LinkFile() is not called on files still open for write, which might not be supported by some FileSystem implementations. This should not be a performance issue, but an opt-out is available with with new DB option background_close_inactive_wals.

Bug Fixes

• Fix a rare case in which a hard-linked WAL in a Checkpoint is not fully synced (so might lose data on power loss).
• Fixed the output of the ldb dump_wal command for PutEntity records so it prints the key and correctly resets the hexadecimal formatting flag after printing the wide-column entity.
• Fixed an issue where PutEntity records were handled incorrectly while rebuilding transactions during recovery.
• Various read operations could ignore various ReadOptions that might be relevant. Fixed many such cases, which can result in behavior change but a better reflection of specified options.

Performance Improvements

• Improved write throughput to memtable when there's a large number of concurrent writers and allow_concurrent_memtable_write=true(#​12545)

v9.3.1: RocksDB 9.3.1

9.3.1 (2024-05-25)

Bug Fixes

• [internal only] Build script improvement

9.3.0 (2024-05-17)

New Features

• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the GetEntity API.
• Added new Iterator property, "rocksdb.iterator.is-value-pinned", for checking whether the Slice returned by Iterator::value() can be used until the Iterator is destroyed.
• Optimistic transactions and WriteCommitted pessimistic transactions now support the MultiGetEntity API.
• Optimistic transactions and pessimistic transactions with the WriteCommitted policy now support the PutEntity API. Support for read APIs and other write policies (WritePrepared, WriteUnprepared) will be added later.

Public API Changes

• Exposed block based metadata cache options via C API
• Exposed compaction pri via c api.
• Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.

Behavior Changes

• CompactRange() with change_level=true on a CF with FIFO compaction will return Status::NotSupported().
• External file ingestion with FIFO compaction will always ingest to L0.

Bug Fixes

• Fixed a bug for databases using DBOptions::allow_2pc == true (all TransactionDBs except OptimisticTransactionDB) that have exactly one column family. Due to a missing WAL sync, attempting to open the DB could have returned a Status::Corruption with a message like "SST file is ahead of WALs".
• Fix a bug in CreateColumnFamilyWithImport() where if multiple CFs are imported, we were not resetting files' epoch number and L0 files can have overlapping key range but the same epoch number.
• Fixed race conditions when ColumnFamilyOptions::inplace_update_support == true between user overwrites and reads on the same key.
• Fix a bug where CompactFiles() can compact files of range conflict with other ongoing compactions' when preclude_last_level_data_seconds > 0 is used
• Fixed a false positive Status::Corruption reported when reopening a DB that used DBOptions::recycle_log_file_num > 0 and DBOptions::wal_compression != kNoCompression.
• While WAL is locked with LockWAL(), some operations like Flush() and IngestExternalFile() are now blocked as they should have been.
• Fixed a bug causing stale memory access when using the TieredSecondaryCache with an NVM secondary cache, and a file system that supports return an FS allocated buffer for MultiRead (FSSupportedOps::kFSBuffer is set).

v9.2.1: RocksDB 9.2.1

9.2.1 (2024-05-03)

Public API Changes

• Add a kAdmPolicyAllowAll option to TieredAdmissionPolicy that admits all blocks evicted from the primary block cache into the compressed secondary cache.

9.2.0 (2024-05-01)

New Features

• Added two options deadline and max_size_bytes for CacheDumper to exit early
• Added a new API GetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for wide-column point lookups with read-your-own-writes consistency. Similarly to GetFromBatchAndDB, the API can combine data from the write batch with data from the underlying database if needed. See the API comments for more details.
• [Experimental] Introduce two new cross-column-family iterators - CoalescingIterator and AttributeGroupIterator. The CoalescingIterator enables users to iterate over multiple column families and access their values and columns. During this iteration, if the same key exists in more than one column family, the keys in the later column family will overshadow the previous ones. The AttributeGroupIterator allows users to gather wide columns per Column Family and create attribute groups while iterating over keys across all CFs.
• Added a new API MultiGetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for batched wide-column point lookups with read-your-own-writes consistency. Similarly to MultiGetFromBatchAndDB, the API can combine data from the write batch with data from the underlying database if needed. See the API comments for more details.
• *Adds a SstFileReader::NewTableIterator API to support programmatically read a SST file as a raw table file.
• Add an option to WaitForCompactOptions - wait_for_purge to make WaitForCompact() API wait for background purge to complete

Public API Changes

• DeleteRange() will return NotSupported() if row_cache is configured since they don't work together in some cases.
• Deprecated CompactionOptions::compression since CompactionOptions's API for configuring compression was incomplete, unsafe, and likely unnecessary
• Using OptionChangeMigration() to migrate from non-FIFO to FIFO compaction
  with Options::compaction_options_fifo.max_table_files_size > 0 can cause
  the whole DB to be dropped right after migration if the migrated data is larger than
  max_table_files_size

Behavior Changes

• Enabling BlockBasedTableOptions::block_align is now incompatible (i.e., APIs will return Status::InvalidArgument) with more ways of enabling compression: CompactionOptions::compression, ColumnFamilyOptions::compression_per_level, and ColumnFamilyOptions::bottommost_compression.
• Changed the default value of CompactionOptions::compression to kDisableCompressionOption, which means the compression type is determined by the ColumnFamilyOptions.
• BlockBasedTableOptions::optimize_filters_for_memory is now set to true by default. When partition_filters=false, this could lead to somewhat increased average RSS memory usage by the block cache, but this "extra" usage is within the allowed memory budget and should make memory usage more consistent (by minimizing internal fragmentation for more kinds of blocks).
• Dump all keys for cache dumper impl if SetDumpFilter() is not called
• CompactRange() with CompactRangeOptions::change_level = true and CompactRangeOptions::target_level = 0 that ends up moving more than 1 file from non-L0 to L0 will return Status::Aborted().
• On distributed file systems that support file system level checksum verification and reconstruction reads, RocksDB will now retry a file read if the initial read fails RocksDB block level or record level checksum verification. This applies to MANIFEST file reads when the DB is opened, and to SST file reads at all times.

Bug Fixes

• Fix a bug causing VerifyFileChecksums() to return false-positive corruption under BlockBasedTableOptions::block_align=true
• Provide consistent view of the database across the column families for NewIterators() API.
• Fixed feature interaction bug for DeleteRange() together with ColumnFamilyOptions::memtable_insert_with_hint_prefix_extractor. The impact of this bug would likely be corruption or crashing.
• Fixed hang in DisableManualCompactions() where compactions waiting to be scheduled due to conflicts would not be canceled promptly
• Fixed a regression when ColumnFamilyOptions::max_successive_merges > 0 where the CPU overhead for deciding whether to merge could have increased unless the user had set the option ColumnFamilyOptions::strict_max_successive_merges
• Fixed a bug in MultiGet() and MultiGetEntity() together with blob files (ColumnFamilyOptions::enable_blob_files == true). An error looking up one of the keys could cause the results to be wrong for other keys for which the statuses were Status::OK.
• Fixed a bug where wrong padded bytes are used to generate file checksum and DataVerificationInfo::checksum upon file creation
• Correctly implemented the move semantics of PinnableWideColumns.
• Fixed a bug when the recycle_log_file_num in DBOptions is changed from 0 to non-zero when a DB is reopened. On a subsequent reopen, if a log file created when recycle_log_file_num==0 was reused previously, is alive and is empty, we could end up inserting stale WAL records into the memtable.
• *Fix a bug where obsolete files' deletion during DB::Open are not rate limited with SstFilemManager's slow deletion feature even if it's configured.

v9.1.1: RocksDB 9.1.1

9.1.1 (2024-04-17)

Bug Fixes

• Fixed Java SstFileMetaData to prevent throwing java.lang.NoSuchMethodError
• Fixed a regression when ColumnFamilyOptions::max_successive_merges > 0 where the CPU overhead for deciding whether to merge could have increased unless the user had set the option ColumnFamilyOptions::strict_max_successive_merges

v9.1.0

New Features

• Added an option, GetMergeOperandsOptions::continue_cb, to give users the ability to end GetMergeOperands()'s lookup process before all merge operands were found.
• Add sanity checks for ingesting external files that currently checks if the user key comparator used to create the file is compatible with the column family's user key comparator.
  *Support ingesting external files for column family that has user-defined timestamps in memtable only enabled.
• On file systems that support storage level data checksum and reconstruction, retry SST block reads for point lookups, scans, and flush and compaction if there's a checksum mismatch on the initial read.
• Some enhancements and fixes to experimental Temperature handling features, including new default_write_temperature CF option and opening an SstFileWriter with a temperature.
• WriteBatchWithIndex now supports wide-column point lookups via the GetEntityFromBatch API. See the API comments for more details.
• Implement experimental features: API Iterator::GetProperty("rocksdb.iterator.write-time") to allow users to get data's approximate write unix time and write data with a specific write time via WriteBatch::TimedPut API.

Public API Changes

• Best-effort recovery (best_efforts_recovery == true) may now be used together with atomic flush (atomic_flush == true). The all-or-nothing recovery guarantee for atomically flushed data will be upheld.
• Remove deprecated option bottommost_temperature, already replaced by last_level_temperature
• Added new PerfContext counters for block cache bytes read - block_cache_index_read_byte, block_cache_filter_read_byte, block_cache_compression_dict_read_byte, and block_cache_read_byte.
• Deprecate experimental Remote Compaction APIs - StartV2() and WaitForCompleteV2() and introduce Schedule() and Wait(). The new APIs essentially does the same thing as the old APIs. They allow taking externally generated unique id to wait for remote compaction to complete.
• For API WriteCommittedTransaction::GetForUpdate, if the column family enables user-defined timestamp, it was mandated that argument do_validate cannot be false, and UDT based validation has to be done with a user set read timestamp. It's updated to make the UDT based validation optional if user sets do_validate to false and does not set a read timestamp. With this, GetForUpdate skips UDT based validation and it's users' responsibility to enforce the UDT invariant. SO DO NOT skip this UDT-based validation if users do not have ways to enforce the UDT invariant. Ways to enforce the invariant on the users side include manage a monotonically increasing timestamp, commit transactions in a single thread etc.
• Defined a new PerfLevel kEnableWait to measure time spent by user threads blocked in RocksDB other than mutex, such as a write thread waiting to be added to a write group, a write thread delayed or stalled etc.
• RateLimiter's API no longer requires the burst size to be the refill size. Users of NewGenericRateLimiter() can now provide burst size in single_burst_bytes. Implementors of RateLimiter::SetSingleBurstBytes() need to adapt their implementations to match the changed API doc.
• Add write_memtable_time to the newly introduced PerfLevel kEnableWait.

Behavior Changes

• RateLimiters created by NewGenericRateLimiter() no longer modify the refill period when SetSingleBurstBytes() is called.
• Merge writes will only keep merge operand count within ColumnFamilyOptions::max_successive_merges when the key's merge operands are all found in memory, unless strict_max_successive_merges is explicitly set.

Bug Fixes

• Fixed kBlockCacheTier reads to return Status::Incomplete when I/O is needed to fetch a merge chain's base value from a blob file.
• Fixed kBlockCacheTier reads to return Status::Incomplete on table cache miss rather than incorrectly returning an empty value.
• Fixed a data race in WalManager that may affect how frequent PurgeObsoleteWALFiles() runs.
• Re-enable the recycle_log_file_num option in DBOptions for kPointInTimeRecovery WAL recovery mode, which was previously disabled due to a bug in the recovery logic. This option is incompatible with WriteOptions::disableWAL. A Status::InvalidArgument() will be returned if disableWAL is specified.

Performance Improvements

• Java API multiGet() variants now take advantage of the underlying batched multiGet() performance improvements.
  Before

Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 6315.541 ± 8.106 ops/s
MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 6975.468 ± 68.964 ops/s

After

Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 64 thrpt 25 7046.739 ± 13.299 ops/s
MultiGetBenchmarks.multiGetList10 no_column_family 10000 16 100 1024 thrpt 25 7654.521 ± 60.121 ops/s

v9.0.1: RocksDB 9.0.1

9.0.1 (2024-04-11)

Bug Fixes

• Fixed CMake Javadoc and source jar builds
• Fixed Java SstFileMetaData to prevent throwing java.lang.NoSuchMethodError

v9.0.0

New Features

• Provide support for FSBuffer for point lookups. Also added support for s

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.