facebook/rocksdb v11.8.1

RocksDB 11.8.1

on GitHub

11.8.1 (07/28/2026)

Public API Changes

• Add DBOptions::read_io_executor_threads to increase the maximum thread count of the shared filesystem read I/O executor used for asynchronous reads when opening a DB. Opening a DB never reduces the shared executor thread count. Change the experimental FSRandomAccessFile::SubmitReadAsync() API to return whether the asynchronous path was used, and add the FILE_SUBMIT_ASYNC_READ_FALLBACK ticker to count calls that fall back to synchronous reads.

Performance Improvements

• When DBOptions::avoid_unnecessary_blocking_io is true, obsolete OPTIONS-* files found during DB open are deleted by background purge instead of synchronously on the opening thread.

11.8.0 (07/28/2026)

Public API Changes

• Add callback-based asynchronous read APIs, DB::GetAsync() and DB::MultiGetAsync(). The idea is that when IO is required, RocksDB can suspend its internal coroutine read path, allowing the read executor thread to do other work. When the IO is complete, RocksDB invokes the user callback. This requires filesystem support for full performance benefits. A new filesystem API FSRandomAccessFile::SubmitReadAsync() is introduced for this. Unlike ReadAsync, the filesystem is responsible for eventually calling the callback. FileSystem::GetReadExecutor() returns the IO executor whose EventBases run coroutine read processing. In the Posix filesystem, an IO-uring based event loop is used to complete the IO.
• Added FlushOptions::listener_wait (default false). When set together with FlushOptions::wait == true, DB::Flush() will not return until the registered EventListener::OnFlushCompleted callbacks for the flushed memtables have finished running. By default (false), Flush(wait=true) may return as soon as the flush result is committed, which can be before (or while) the OnFlushCompleted callbacks execute on the background flush thread. Also added the corresponding C API rocksdb_flushoptions_set_listener_wait() / rocksdb_flushoptions_get_listener_wait().
• Deprecated PinnableWideColumns::serialized_size() in favor of the new PinnableWideColumns::payload_size(), which returns the total size of the columns' names and values (for a plain value this equals the value size). Relatedly, wide-column read accounting changed subtly: read statistics for entities (BYTES_READ, BYTES_PER_READ, and the per-key byte counts for GetEntity/MultiGetEntity) and the ReadOptions::value_size_soft_limit threshold now measure entities by this payload size instead of the serialized entity size. This drops serialization framing bytes from those measurements (so reported/limited sizes for entity reads are slightly smaller than before) and makes plain-value and wide-column accounting consistent.

Behavior Changes

• ReadOptions::value_size_soft_limit now also bounds the blob values resolved from pre-flush blob direct write references during MultiGet/MultiGetEntity (previously these were resolved after the limit check and read without bound). Enforcement is "always make progress": at least one key is read even if its value alone exceeds the limit, and only once the returned size exceeds the limit do subsequent keys get Status::Aborted -- so a caller retrying the aborted keys cannot loop forever on a single value that by itself exceeds the limit.

Performance Improvements

• Reduced copying when reading wide-column entities that have blob-referenced columns, whether stored in separate blob files or embedded (same-file) as blob records in the same SST. On the point-lookup (GetEntity/MultiGetEntity) and blob-backed memtable/direct-write read paths, resolving these columns no longer re-serializes the whole entity into a fresh buffer: inline columns are referenced in place in the pinned entity and each resolved blob value is referenced in place from its fetched buffer. This is enabled by an internal change to PinnableWideColumns that lets its columns reference multiple backing buffers.

11.7.0 (07/16/2026)

Public API Changes

• Added RangeLockManagerHandle::SetIsKilledCallback(), a std::function<bool()> predicate polled during range-lock waits. When it returns true, a blocked GetRangeLock() wait returns promptly with Status::Aborted() instead of waiting for the lock timeout. Range-lock users that install no callback keep the original wait-until-grant-or-timeout behavior.
• Added MultiScanArgs::reverse to support reverse MultiScan reads over bounded scan ranges.

Behavior Changes

• When using LockWAL(), live-file capture APIs such as checkpoint and backup now flush WAL-disabled unpersisted data; they can block until UnlockWAL() or return Aborted when called from the same thread that holds LockWAL() to avoid deadlock. UnlockWAL() must now also be called from the same thread that called LockWAL().

Performance Improvements

• Fixed slow DB open with best_efforts_recovery, and slow catch-up of secondary and follower (read-only) instances, in which time and memory grew quadratically with the number of files in the DB (so the slowdown was worst for large DBs with long history).

11.6.0 (07/02/2026)

New Features

• Added EXPERIMENTAL embedded blob SST support through SstFileWriter::OpenWithEmbeddedBlobs(), storing eligible large values as same-file blob records in block-based SST files and resolving them transparently for reads. This niche feature currently supports uncompressed embedded blobs only; compression options are placeholders and compression support is deferred to follow-up work.

Public API Changes

• Expanded the C API (include/rocksdb/c.h) with a large set of new rocksdb_* functions, mostly option getters/setters plus table-properties, job/event-listener, and metadata accessors, a WAL filter, a ReadOptions table filter, and a backup exclude-files callback. Many are now produced by a new semi-automated generator (tools/c_api_gen/) from the C++ headers; include/rocksdb/c.h remains a single self-contained header and the signatures of pre-existing functions are unchanged.

Bug Fixes

• Reverted PR14831 that made range_lock_manager aware of reverse-order CF
• Fixed a bug in RandomAccessFileReader::ReadAsync where an already-aligned direct-IO read request with a null scratch and a caller-provided aligned_buf would take the "already aligned" fast path and submit the null buffer to the underlying async read (e.g. a null iovec base to io_uring, failing with EFAULT). This could surface as spurious iterator failures during async prefetch (MultiScan with async IO) on direct-IO databases. The async path now allocates a backing buffer in this case, matching the synchronous Read path.
• Fixed a bug where closing a read-only DB instance could delete live SST files created by a concurrent read-write DB sharing the same directory.

11.5.0 (06/16/2026)

New Features

• External table readers that open files through ExternalTableOptions::fs now update RocksDB SST/file-read statistics and file IO listener callbacks, making external file IO activity visible in existing read metrics.
• Added DB::PrepareFileIngestion(), a two-phase form of IngestExternalFile/IngestExternalFiles. PrepareFileIngestion() performs all of the work that does not require the DB mutex (validating the arguments, reading each external file's metadata, and linking/copying the files into the DB) and returns an opaque FileIngestionHandle. DB::CommitFileIngestionHandle() (or DB::CommitFileIngestionHandles() for several handles at once) then makes the prepared files visible; multiple handles are committed atomically in a single MANIFEST write, or none are. FileIngestionHandle::Abort() (or simply destroying the handle) cancels a prepared ingestion and rolls it back. This gives flexibility to the application to Prepare a file separately from when its committed, and may shorten an applications critical path on IngestExternalFile.
• Added two stats histograms reporting the per-call latency (in microseconds) of each successful IngestExternalFile/IngestExternalFiles call, split by phase: rocksdb.ingest.external.file.prepare.micros (argument validation and reading/validating/linking the external files, which does not block live writes) and rocksdb.ingest.external.file.run.micros (the ingestion performed under the DB mutex while live writes are blocked). Failed ingestions are not recorded.
• Added DBOptions::use_direct_io_for_compaction_reads (default false). When enabled, compaction-input SST reads use O_DIRECT while user reads remain buffered, avoiding page-cache eviction of hot user-read data by sequential compaction scans. Pair with use_direct_io_for_flush_and_compaction = true on write-heavy workloads for direct I/O on both compaction inputs and outputs. Rejected at Open when combined with allow_mmap_reads = true. No-op when use_direct_reads = true is already set (since all reads are already direct).

Public API Changes

• DB::GetPreparedFileInfoForExternalSstIngestion() can now prepare metadata for a live DB-generated SST file so it can be passed to IngestExternalFileArg::file_infos. The input path must exactly match a live table file owned by the source DB, and the returned metadata handle must outlive the ingestion.
• ExternalSstFileInfo gained a prepared_file_info member produced by SstFileWriter::Finish, and IngestExternalFileArg gained a file_infos field: a vector of borrowed const PreparedFileInfo* pointers, parallel to external_files. The owning ExternalSstFileInfo::prepared_file_info handle must outlive the ingestion. When set, IngestExternalFiles() reuses the supplied per-file metadata instead of re-opening and scanning each file to recompute it, avoiding that extra I/O.
• Corrected the public C++ option spelling from memtable_veirfy_per_key_checksum_on_seek to memtable_verify_per_key_checksum_on_seek. RocksDB continues to accept the old misspelled OPTIONS-file key for compatibility, while newly serialized OPTIONS files use the corrected key.
• Added experimental read-scoped block buffer provider API, configured through ReadOptions::read_scoped_block_buffer_provider, for supported block-based table iterator scans and MultiScan reads to use caller-provided read-scoped storage for final data-block contents. When configured, supported provider-backed scan data-block reads bypass the data-block cache. The provider is ignored when mmap reads are enabled. RocksDB may still use ordinary temporary scratch for serialized block bytes, such as when a block may be compressed. Get and MultiGet do not currently provide API guarantees for this provider.
• Added FileSystem::SyncFile() and Env::SyncFile() public APIs for syncing or fsyncing a file by name without requiring callers to reopen it as writable.

Behavior Changes

• The default compression for column families is now kLZ4Compression rather than kSnappyCompression. This affects only column families that do not explicitly set compression, and only newly-written SST files. The change is fully compatible: existing data remains readable with no migration needed, as RocksDB selects the decompressor per block. LZ4 offers slightly better compression ratios and decompression CPU efficiency vs. Snappy and similar compression CPU, tested across various server CPUs. When support is not compiled in, the fallback path for default compression is LZ4 -> Snappy -> NoCompression.
• Disable parallel compression (ignore CompressionOptions::parallel_threads) for fast built-in compressors: Snappy, LZ4 (accelerated, not LZ4HC), and accelerated levels of ZSTD (level < 0). These do not generally benefit from parallel compression. This behavior can be overridden with a custom Compressor from a custom CompressionManager.
• PosixFileSystem::OptimizeForCompactionTableRead now delegates to the base FileSystem::OptimizeForCompactionTableRead implementation before applying its Linux-only compaction-readahead clamp (added for #12038). The returned FileOptions::use_direct_reads therefore reflects DBOptions::use_direct_reads, consistent with the base class and with other FileSystem implementations. Previously the override ignored the base implementation and keyed both the returned options and the Linux readahead clamp off the incoming FileOptions rather than DBOptions. At the in-tree call site the incoming FileOptions::use_direct_reads already matches DBOptions::use_direct_reads, so this is effectively a no-op for existing configurations; it removes a latent inconsistency where a caller passing FileOptions whose use_direct_reads disagreed with DBOptions would have had the global flag silently ignored for compaction-input reads on Linux. (Note: the #12038 readahead clamp still keys off the global use_direct_reads; it is not skipped for the compaction-only use_direct_io_for_compaction_reads path, since that flag enables O_DIRECT after this hook runs.)
• Brought more unity to kLZ4Compression and kLZ4HCCompression by giving each access to the other's compression levels. Negative (fast) values previously only available to LZ4 and positive (slow) values previously only available to LZ4HC are now available to both, so configuring a non-default compression_opts.level now selects the LZ4 compressor variant. This is a behavior change for previously tolerated but dubious configurations such as positive compression level with kLZ4Compression. kLZ4Compression and kLZ4HCCompression keep their effective default levels, equivalent to -1 and 9 respectively.
• Removed a discontinuity in kZSTD compression levels at compression_opts.level == 0 by mapping that setting to level = -1 rather than to level = 3 as the ZSTD library does internally. This improves the compression auto-tuning landscape.

Bug Fixes

• Fixed a bug where the range lock manager would crash with an assertion failure when using a reverse comparator column family.

Performance Improvements

• Reduced commit latency when ingesting many external files by allowing IngestExternalFileOptions::file_opening_threads to open table readers for committed ingested files using multiple threads.
• Reduced commit latency for large external file ingestions into the last level by adding IngestExternalFileOptions::prefetch_lmax_index_and_filter_blocks, which can skip commit-time index and filter block prefetching for cache-backed table metadata.

11.4.0 (06/02/2026)

Public API Changes

• Added rocksdb_options_set_memtable_batch_lookup_optimization() and rocksdb_options_get_memtable_batch_lookup_optimization() to the C API, exposing the existing AdvancedColumnFamilyOptions::memtable_batch_lookup_optimization field. This allows C API users (and downstream language bindings) to enable the skip-list memtable's batch-lookup optimization for MultiGet, which caches the search path between consecutive keys and reduces per-key cost from O(log N) to O(log d) where d is the distance between consecutive keys.
• Added rocksdb_readoptions_set_optimize_multiget_for_io() and rocksdb_readoptions_get_optimize_multiget_for_io() to the C API, exposing the existing ReadOptions::optimize_multiget_for_io field. This allows C API users (and downstream language bindings) to opt out of the multi-level parallel MultiGet path, which only takes effect when the library is built with USE_COROUTINES.
• Added rocksdb_block_based_table_index_block_search_type_auto enum constant and the rocksdb_block_based_options_set_uniform_cv_threshold() setter to the C API. The constant exposes BlockSearchType::kAuto, and the setter exposes BlockBasedTableOptions::uniform_cv_threshold. Both are required for kAuto index-block search to take effect from the C API: without setting uniform_cv_threshold >= 0, the per-block is_uniform footer bit is never written, and kAuto falls back to binary search at read time.
• Added EventListener::OnDBShutdownBegin, a callback that fires once when a DB begins shutdown, including when RocksDB cleans up a failed DB::Open() attempt.
• Added a new PerfContext counter blob_cache_read_byte for blob cache bytes read.

Behavior Changes

• Remote compaction now falls back to local compaction when the primary cannot deserialize a successful CompactionServiceResult before installing any remote output files.
• StringToMap (rocksdb/convenience.h) now preserves the outer braces of nested values so each map entry is in self-contained form and can be embedded directly into another key=value; string (e.g. SetOptions) without losing inner ';' delimiters. A new symmetric MapToString utility is provided.

Bug Fixes

• Fixed a bug where AbortAllCompactions() could leave automatic compaction work unscheduled when aborting before the background worker picked it, causing compaction and write stalls until DB restart.
• Fix a bug where db had a false positive compaction corruption error due to remote compaction service result with has_accurate_num_input_records=false not being serialized.
• Fixed WritePrepared TransactionDB cleanup after retryable commit or rollback write failures so prepared transactions remain rollbackable instead of leaving unresolved prepared state.
• Fix a rare corruption bug for tiered compaction that incorrectly moved last level range tombstones into proximal level. This corruption error is surfaced when force_consistency_checks is enabled.

Performance Improvements

• Reduce the likelihood of user-facing metadata operations blocking on MANIFEST rotation when file creation is slow, such as on remote storage. MANIFEST write batches containing foreground edits from CreateColumnFamily(), DropColumnFamily(), CreateColumnFamilyWithImport(), IngestExternalFile(), and DeleteFilesInRanges() now get a relaxed file size threshold for triggering MANIFEST rotation; background-only MANIFEST write batches, such as compaction and flush, continue to use the normal threshold. This change should make auto-tuning manifest file size more attractive (see max_manifest_file_size and max_manifest_space_amp_pct options).

11.3.0 (05/15/2026)

New Features

• Add experimental DB option async_wal_precreate to precreate the next WAL file in a background thread and reduce foreground WAL rotation latency. The option is sanitized to false when WAL recycling is enabled.
• Added a new EventListener::OnCompactionPreCommit callback that fires after a compaction job finishes but before its input files are released (i.e. while FileMetaData::being_compacted is still true). Listeners that maintain bookkeeping of which files are currently being compacted can clean up such state in this new callback to avoid races with concurrent compaction picking, where another thread might pick up the same files for a new compaction immediately after being_compacted is flipped back to false but before OnCompactionCompleted fires. The default implementation is a no-op so this is not a breaking change.
• Add mutable DBOption optimize_manifest_for_recovery (default false). When enabled, RocksDB can reduce recovery work after a clean shutdown, which may lower DB::Open latency on warm reopens.
• Added public utility APIs ParseCompressionNameForDisplay() to convert TableProperties::compression_name into a human-readable compression name for both legacy and format_version 7+ SST metadata, including custom CompressionManager-provided display names for custom compression types.
• Add reuse_manifest_on_open DBOption (default false). When enabled, DB::Open reuses the existing MANIFEST file for append instead of creating a fresh one, avoiding the cost of serializing the entire database state into a new MANIFEST on the first post-open write. To prevent this feature from interfering with manifest file size auto-tuning, an extra forward-compatible field is now always added to the MANIFEST (to track the last "compacted" size).

Behavior Changes

• Read-only open with error_if_wal_file_exists=true now tolerates empty WAL files so empty precreated WALs do not prevent inspection.
• WriteCommitted TransactionDB now matches WritePrepared and WriteUnprepared compaction filtering in both single- and two-write-queue modes: a compaction filter's FilterMergeOperand will not be invoked on merge operands at or below the latest published sequence number.

Bug Fixes

• Fixed blob-backed wide-column merge reads to preserve correct status
  propagation and resolution across memtable, read-only, and secondary DB
  paths.
• Fixed a bug where DB::GetCreationTimeOfOldestFile() could return inaccurate results instead of the real creation time when called shortly after opening a legacy DB (one whose manifest lacks file_creation_time) with open_files_async = true. The API now waits for background SST file loading to complete only when needed; modern DBs are unaffected.
• Fixed merge reads against wide-column/blob-backed base values to preserve precise failure statuses, including GetMergeOperands() and direct-write memtable reads.
• Fix bug in range tombstone synthesis that covers live keys added during an IngestExternalFile
• Reject the empty string as a column family name in DB::CreateColumnFamily / DB::CreateColumnFamilies. Previously such calls returned OK and a usable handle, but the column family was not persisted in the manifest, so any data written to it was silently lost on DB reopen.
• Fixed a bug where a WriteCommitted TransactionDB using commit-bypass WBWI ingestion could drop an entry that is still visible at the published sequence boundary.

11.2.0 (04/18/2026)

New Features

• Added experimental DBOptions::fast_sst_open option. When enabled, RocksDB retrieves opaque file system metadata for SST files after flush, compaction, and external file ingestion, persists it in the MANIFEST, and passes it back to the file system on subsequent file opens to accelerate DB open time.
• Added new option min_tombstones_for_range_conversion in AdvancedColumnFamilyOptions. When set to a non-zero value N, forward or reverse iteration will convert N or more contiguous point tombstones into a range tombstone in the mutable memtable. Future read operations will then be able to benefit from range tombstone optimizations. There are some limitations when it comes to table_filters, prefix_filters, and UDTs. See header comments for more details.
• Added read-triggered compaction: a new column family option read_triggered_compaction_threshold (default 0, disabled) that marks SST files for compaction when their read frequency (num_collapsible_entry_reads_sampled / file_size) exceeds the threshold. This helps reduce read amplification for frequently-read ("hot") keys. A new DB option max_compaction_trigger_wakeup_seconds (default 43200s / 12 hours) controls the maximum interval for periodic compaction score re-evaluation, which is necessary for this feature to work on quiet (no-write) databases.

Public API Changes

• Added new WideColumnBlobResolver interface and CompactionFilter::FilterV4() method, including ResolveColumn() / ResolveColumns() helpers for lazy loading blob column values in wide-column entities during compaction. This allows compaction filters to resolve blob values on-demand, avoiding unnecessary I/O for blob columns they don't need to access.
• Changed experimental feature ExternalTableReader::Get and ExternalTableReader::MultiGet to use PinnableSlice instead of std::string for output values, enabling zero-copy pinning. This will break existing implementations.
• Added SstFileReader::Get and SstFileReader::MultiGet overloads that accept PinnableSlice/std::vector<PinnableSlice>*, enabling zero-copy reads when the underlying TableReader supports pinning.

Behavior Changes

• Prefix filter changes - when seeking to a key that is out of domain, and total_order_seek is false, total_order_seek is treated as if it were true. When prefix_same_as_start = true, now iterating past a key that is out of domain invalidates the iterator. The existing behavior does not check for InDomain in the DBIter, so Transform() can produce undefined behavior (e.g. key of size 3 on FixedPrefixTransform(4)).
• Wide-column entities with blob-backed columns now use a new V2 on-disk encoding; older RocksDB versions that do not support wide-column blob separation will reject DBs or SSTs containing those entities.

Bug Fixes

• Fix blob garbage accounting for blob direct-write flushes so flush-time filtering and overwrite elision correctly register obsolete blob bytes in blob metadata.
• Fix a memory accounting leak in IODispatcher where ReadIndex() moved block values out of ReadSet without releasing the associated prefetch memory, causing subsequent prefetches to be blocked when max_prefetch_memory_bytes was set.
• Fix MultiScan to fall back to synchronous coalesced reads when async I/O is unsupported at runtime.