facebook/rocksdb v11.12

on GitHub

New Features

• Added FullExternalTableFactory and related full-mode external table APIs that preserve RocksDB internal keys, enabling external tables to support nonzero sequence numbers, point and single deletions, merge operands, blob indexes, wide-column entities, snapshots, live DB writes, and multi-level compactions.

Public API Changes

• Added experimental coroutine capability interfaces for basic and full ExternalTableReader implementations. Coroutine-capable external readers can initiate file reads through FSRandomAccessFile::SubmitReadAsync().
• ExternalTableFactory::NewTableReader() now receives an already-opened FSRandomAccessFile and its file size instead of requiring implementations to reopen file_path. The supplied file preserves RocksDB's direct-I/O alignment handling, rate limiting, read statistics and listener notifications, prefetching, and synchronous, asynchronous, and batched reads.
• Added PutRangeDeletionBlock(), GetRangeDeletionBlock(), and IsDeleteRangeSupported() to the external table interfaces so formats can opt in to range deletions using RocksDB's raw range-deletion metadata block.
• Opaque PreparedFileInfo metadata can now be serialized to a checksummed representation with SerializePreparedFileInfo() and restored with DeserializePreparedFileInfo() for reuse across processes.
• Removed the deprecated IngestExternalFileOptions::write_global_seqno option and its C, Java, and ldb APIs, TableProperties::external_sst_file_global_seqno_offset, and the file_offset output from ExternalTableReader::GetPropertiesBlock(). Existing external SST files whose global sequence number was rewritten without updating the properties-block checksum remain readable.
• TablePropertiesNames has no apparent use in the public API so has been deprecated there.
• TableProperties::creation_time is renamed to oldest_ancestor_time, which better describes its meaning, along with TablePropertiesNames::kCreationTime to kOldestAncestorTime. The misspelled SstFileMetaData::oldest_ancester_time is renamed to oldest_ancestor_time. The old names remain available as deprecated aliases and will be removed in a future release.
• Added WriteBufferFlushPolicy and WriteBufferManager policy selection for flushing the oldest or largest column family, including configurable batched flushing of the largest column families across DBs sharing a manager, with C and Java bindings.

Behavior Changes

• When ExternalTableBuilderBase::PutPropertiesBlock() returns NotSupported, RocksDB now overrides num_entries, raw_key_size, and raw_value_size from ExternalTableBuilderBase::GetTableProperties() with the logical totals tracked by the external-table adapter. Format-specific properties such as data_size remain unchanged.
• The FSWritableFile passed to ExternalTableFactory::NewTableBuilder() now reports logical file size including RocksDB-buffered data and exposes the underlying file's direct-I/O mode and required buffer alignment.
• Iterator::Refresh() now returns NotSupported on iterators from a secondary DB's DB::NewIterators() only, because those iterators are pinned to one cross-column-family read view. Iterators from DB::NewIterator() are unaffected; to advance a batch, call DB::NewIterators() again after TryCatchUpWithPrimary().
• SstFileReader now reports standard SST read statistics and file I/O listener notifications, and honors the configured read rate limiter.

Bug Fixes

• Fixed full recovery from retryable background errors to atomically advance WAL replay boundaries for all column families, preventing partial recovery of multi-column-family write batches after a failed recovery flush.
• Fixed RocksDB builds on big-endian architectures such as s390x by disabling the optional persistent fault-injection log, whose binary format is little-endian-only.
• Fixed DB::ClipColumnFamily() deleting a non-L0 SST file whose largest key equals begin_key, which lost keys at the start of the kept range.
• Fixed swapped oldest_key_time and newest_key_time values in the table_file_creation event log entry.
• Fixed ExternalTableReader::VerifyChecksum() results not being honored by the external table adapter.
• Fixed a bug in manifest handling that could lead to a hang or memory corruption in the case of LSM sanity checks being fatally violated (exceptional; see force_consistency_checks).
• Fixed MultiGet() with ReadOptions::read_tier == kBlockCacheTier reading data blocks from disk when allow_mmap_reads is false. Keys whose data block is not in the block cache now return Status::Incomplete(), the same as Get().
• Fixed trace replay (DB::NewDefaultReplayer(), db_bench replay) failing every read record (Get, MultiGet, iterator seek) on the default column family with Corruption: Invalid Column Family ID when the default column family handle was not passed, which is always the case in db_bench with --num_column_families=1.
• Fixed multi-threaded Replayer::Replay() ignoring errors from executing trace records and returning OK. It now stops and returns the error, including Status::NotSupported() from executing a record, as single-threaded replay does.
• Fixed secondary DB NewIterators() returning iterators from different database states or blocking for TryCatchUpWithPrimary() by serving the last completed catch-up view.

Performance Improvements

• Single Get coroutine reads using non-mmap, non-direct I/O on a kFSBuffer-capable FileSystem now use FSRandomAccessFile::SubmitReadAsync() instead of falling back to synchronous Read(). When rate-limited, these reads acquire their full token budget before submission to preserve a single filesystem-owned buffer, which can make physical I/O burstier than reads using caller-provided buffers.