Dragonfly v1.31.0

This release is mostly focused on stability features and bug fixes.

Important stability fixes and changes:

1. Introduce partial sync feature that avoids full replica reload during master/replica short disconnects #5182
2. Add support for GAT/GATS commands #5257
3. Introduce experimental huffman encoding for keys #5179

What's Changed

• chore: support fetching a huffman encoded string from CompactObj by @romange in #5086
• fix(hset_family): Fix val being overwritten by TTL by @abhijat in #5094
• fix: memcached authentication handling by @vyavdoshenko in #5098
• fix: memcached port has been already used crash by @vyavdoshenko in #5097
• chore: Add suffix and infix search syntax support in query parser by @vyavdoshenko in #5091
• chore(server): cleanup unused noop opcode in journal by @adiholden in #5101
• chore: add arch type to dfly bench release binary by @kostasrim in #5090
• feat: add HELP SCAN and MINMSZ filter by @romange in #5099
• fix test: rdb test keep loading expired keys by @adiholden in #5105
• refactor: provide CommandContext into InvokeCmd by @BorysTheDev in #5100
• dfly_bench: Send auth if required before cluster setup by @abhijat in #5107
• chore: tls improvements by @romange in #5116
• fix: XTRIM command crash by @vyavdoshenko in #5121
• chore: update module versions by @romange in #5126
• chore: provide metrics for huffman encoding by @romange in #5117
• fix: make search keywords case insensitive by @romange in #5129
• chore: replica clean ups by @romange in #5127
• fix: collapsing logic in CapturingReplyBuilder by @romange in #5133
• fix: improved socket error logging for connection diagnostics by @vyavdoshenko in #5062
• fix: out of bounds SIMD access by @romange in #5140
• chore: improve logs around the parser error by @romange in #5144
• feat: introduce resp validator by @romange in #5147
• chore: minor fixes in geo_family by @romange in #5149
• refactor: geosearch cmd with CmdArgParser by @BorysTheDev in #5153
• feat: add Georadius command by @BorysTheDev in #5158
• pytest: Add sleep to let master stop propagate to replica by @abhijat in #5157
• chore: enable simd optimizations for aarch64 by @romange in #5150
• fix: prevent crash in GEORADIUS argument parsing by @vyavdoshenko in #5166
• chore: more tests around dash table by @romange in #5168
• chore: Add option to ignore parsing errors in replay tool by @vyavdoshenko in #5167
• fix: Prevent crash in PFCOUNT when using keys of different types by @vyavdoshenko in #5161
• chore: lower log level for harmless "Invalid Json path" warnings by @vyavdoshenko in #5163
• chore: remove global current_reply_size by @romange in #5057
• replication: Ensure snapshot fiber joined in all code paths by @abhijat in #5171
• chore: refactorings in dash code. by @romange in #5178
• chore(dash_table): pass memory resource to segment by @romange in #5176
• fix: show proactor kind on test failures by @kostasrim in #5170
• fix(parser): Parse correctly split simple response or error msg in client mode by @mkaruza in #5177
• chore: introduce azure snapshot storage skeleton by @romange in #5104
• fix: improve parsing of client facing responses by @romange in #5181
• server(fix) : SCAN command reduce latency by @vyavdoshenko in #5151
• chore(json_family): Clean up the memory tracking code for JSON mutate operations. FOURTH PR by @BagritsevichStepan in #5070
• feat(search_family): Ignore unsupported options in the FT.CREATE command by @BagritsevichStepan in #5188
• fix (server): fix replication partial sync flow by @adiholden in #5189
• feat: add flags support for huffman encoding by @romange in #5179
• fix server: expire if need check valid condition fix by @adiholden in #5193
• chore: minor dash fixes by @romange in #5191
• fix(json_family): Fix memory tracking for the JSON.DEL command. FIFTH PR by @BagritsevichStepan in #5075
• fix: prevent crash on KEYLOCK_ACQUIRED check for NO_KEY_TRANSACTIONAL commands by @vyavdoshenko in #5185
• fix(cluster_family): Cancel slot migration from incoming node on OOM by @mkaruza in #5000
• fix: timeout for test_seeder_key_target by @BorysTheDev in #5198
• fix: crash in HLL PFMERGE with corrupted data by @vyavdoshenko in #5201
• fix: ZCOUNT crash when min > max by @vyavdoshenko in #5205
• fix: SearchStats race condition crash during concurrent FT operations by @vyavdoshenko in #5207
• fix(pytest): Sleep in sporadically failing test_slot_migration_oom pytest by @mkaruza in #5214
• fix(transaction): DCHECK fail in non-atomic transactions by @romange in #5217
• fix: seeder hset commands generation by @BorysTheDev in #5220
• fix(http_api): Return valid JSON string by @mkaruza in #5210
• feat: add Lua GC flags by @BorysTheDev in #5194
• fix: Unsupported status for Copy/Move/Rename operations by @vyavdoshenko in #5223
• (string_family): Add priliminary support for GAT by @abhijat in #5229
• chore: reduce leaky interface of DashCursor by @romange in #5230
• chore: minor changes in dash code by @romange in #5224
• test: add migration_restart_test with another slots set by @BorysTheDev in #5231
• chore: bring back partial sync by @kostasrim in #5182
• fix: macos build by @vyavdoshenko in #5234
• chore: maintain meta information around continous transactions by @romange in #5232
• fix: dcheck trigger in JournalFlushGuard constructor by @kostasrim in #5241
• fix: typo in AclFamily by @vyavdoshenko in #5244
• fix(pubsub): Skip stale pubsub messages by @dranikpg in #5246
• chore: improve partial sync and test by @kostasrim in #5195
• chore(search): Add self time and resp3 format to ft.profile by @dranikpg in #5250
• fix: test_pubsub_unsubscribe test timeout increased by @vyavdoshenko in #5253
• chore: add corruption check for lpSetTotalBytes by @romange in #5252
• chore: Pull latest helio by @mkaruza in #5254
• fix: BITFIELD: overflow handling, type validation, and return values by @vyavdoshenko in #5242
• fix: prevent writing zero size fields into log file by @BorysTheDev in #5256
• fix(transaction): Fix non-transactional multi/exec transaction access by @dranikpg in #5260
• fix(search_family): Fix empty key bug by @BagritsevichStepan in #5264
• chore: remove old ZIPLIST encoding during snapshotting by @romange in #5267
• chore: stats for lsn buffer by @kostasrim in #5243
• chore: minor fixes in dash table by @romange in #5274
• fix(json): Ensure full deletion for multi-match JSONPaths by @vyavdoshenko in #5270
• fix(reply_builder): Flush if last message didn't reply and flush but supose to by @mkaruza in #5269
• fix(workflow): upload dev release to aws s3 also by @Abhra303 in #5262
• chore: move MemoryBytesFlag to facade by @romange in #5281
• dfly_bench: Support terminating early by @abhijat in #5285
• refactor: remove is_oom flag from ConnectionContext by @BorysTheDev in #5216
• fix(protocol_client): Check if sock_ is not null before calling class method by @mkaruza in #5291
• chore: add -skip-time-min flag to skip initial N minutes during replay traffic by @vyavdoshenko in #5288
• fix(json_family): Fix bugs where JSON commands were modifying values of other data types by @BagritsevichStepan in #5266
• fix: Unsupported status 4 (XCLAIM command) by @vyavdoshenko in #5300
• Revert "fix(stream_family): Fix stream memory tracking issues (#5024)" by @romange in #5301
• chore(dash): pass segment id to segment by @romange in #5298
• Revert "fix(stream_family): Fix memory tracking for the STREAMS (#4781)" by @romange in #5304
• feat: Added store option for sort command. by @H4R5H1T-007 in #5095
• fix: prevent rename cmd crash by @BorysTheDev in #5307
• fix(string_family): Add RemoveDoc for the SET command by @BagritsevichStepan in #5306
• chore: minor refactoring around AutoUpdater by @romange in #5310
• fix(transaction): Fix schedule queues by @dranikpg in #4925
• chore(tiering): Test utils for faulty io by @dranikpg in #4928
• fix: huge sbf deserialization by @BorysTheDev in #5313
• fix: partial replication data loss by @kostasrim in #5297
• chore(string_family): Refactor to prep for adding the GAT command by @abhijat in #5289
• chore: add another benchmark test for the GlobMatcher by @romange in #5317
• fix: memory tracking bug in rename and SET by @romange in #5315
• chore: Improve scope/aggregator usage in replies by @dranikpg in #5286
• chore(db_slice): Add object type to AddOrFind by @BagritsevichStepan in #5305
• fix: fix db_capacity metrics description by @romange in #5318
• Implementation of memcache GAT and GATS commands by @abhijat in #5257
• fix: client kill preempts in atomic section on shutdown by @kostasrim in #5283
• fix(json): JSON.DEL recursive descent duplicate counting for nested same-key elements by @vyavdoshenko in #5312
• chore(search_family): Add AddGenericDoc/RemoveGenericDoc methods by @BagritsevichStepan in #5320
• chore: integrate Valkey fixes into stringmatchlen_impl by @romange in #5323
• docs(readme): add translated documentation for pt-br by @Santosl2 in #5276
• feat(json_family): Copy json_test.py::test_reset_key_to_string to json_family_test by @BagritsevichStepan in #5324
• fix: Handle bonus key + init error in squashing by @dranikpg in #5303
• feat (server): add on move hook to dash insert new and bumpup by @adiholden in #5278

New Contributors

• @Santosl2 made their first contribution in #5276

Full Changelog: v1.30.0...v1.31.0