Dragonfly v1.35.0

Important stability fixes and changes:

• Defragmentation covers list and json data types: #5790, #6023
• FT.Search can now search tags based on prefix and suffix globs: #5871, #5895
• Added support for FT.Search GEO index: #5854
• Optional Huffman compression for keys and values: #6025, #6037

What's Changed

• feat(core): support defragmentation for quicklist by @abhijat in #5790
• chore: skip arguments when logging failed auth cmd by @kostasrim in #5820
• fix(server): Fix AllocatingTracker crash due to VLOG by @dranikpg in #5827
• fix(server): Resest global state on loading error by @dranikpg in #5825
• core: use explicit types and minor cleanups by @abhijat in #5826
• server: Fix metric name by @abhijat in #5829
• fix(server): Support FLUSH(ALL) SYNC by @dranikpg in #5821
• fix(server): Use correct messages for sharded pubsub by @dranikpg in #5818
• fix: stack overflow in hincrbyfloat by @BorysTheDev in #5835
• fix: mac-os build by @romange in #5840
• pytest: Retry set until failure after ACL update by @abhijat in #5823
• ci: Allow overriding log level for modules when repeating tests by @abhijat in #5824
• feat: SimSIMD update and integration by @vyavdoshenko in #5846
• feat(search): FT.SEARCH FILTER option by @mkaruza in #5830
• refactor: restore stack size for debug builds by @BorysTheDev in #5857
• feat: add FT.CONFIG command by @BorysTheDev in #5855
• chore(deps): bump the actions group with 9 updates by @dependabot[bot] in #5841
• chore(deps): bump gopkg.in/yaml.v3 from 3.0.0 to 3.0.1 in /tools/replay by @dependabot[bot] in #5850
• chore(deps): bump golang.org/x/sys from 0.0.0-20210330210617-4fbd30eecc44 to 0.1.0 in /tools/replay by @dependabot[bot] in #5848
• tests: Add test for prometheus metrics sanity check by @abhijat in #5843
• fix: skip full sync during partial sync by @kostasrim in #5580
• fix: clang build for macos by @vyavdoshenko in #5862
• chore: new ReplicaOf algorithm by @kostasrim in #5774
• chore: update go version by @romange in #5859
• chore(deps): bump github.com/redis/go-redis/v9 from 9.7.0 to 9.7.3 in /tools/replay by @dependabot[bot] in #5847
• fix: crash when WITH_SEARCH=OFF by @BorysTheDev in #5867
• actions: Set node version by @abhijat in #5868
• Clean up blocking by @dranikpg in #5836
• feat: add field prefix/suffix search by @BorysTheDev in #5871
• fix: SimSIMD dynamic dispatch by @vyavdoshenko in #5870
• fix: allow 5% tolerance if snapshot used memory greater than maxmemory limit by @BorysTheDev in #5869
• feat(search): Implement GEO index and support for RADIUS search by @mkaruza in #5854
• fix: Fix Fedora 30 and MacOS build by @mkaruza in #5880
• chore(hset_family): Code cleaning by @dranikpg in #5882
• core: Add data structure for estimate which decays over time by @abhijat in #5879
• fix(search): Ensure RemoveDoc before SORT STORE overwrite to keep indices consi stent by @vyavdoshenko in #5887
• fix: Add type safety to FindMutable calls by @vyavdoshenko in #5886
• refactor: Add DelMutable() helper to unify post_updater.Run() + Del() pattern by @vyavdoshenko in #5890
• Fix: environment variable typo in comment by @maets in #5866
• feat: add symbols search in tag by @BorysTheDev in #5895
• feat(search): Add DD option support for FT.DROPINDEX command by @vyavdoshenko in #5885
• chore(hset_family): Listpack wrapper by @dranikpg in #5889
• fix: SINTERSTORE/SUNIONSTORE/SDIFFSTORE crash when overwriting indexed keys by @vyavdoshenko in #5903
• fix: ZINTERSTORE/ZUNIONSTORE crash when overwriting indexed HASH/JSON keys by @vyavdoshenko in #5902
• server: Remove unused variable and field by @abhijat in #5904
• fix(evicition): Don't accumulate deleted bytes if there is no RSS eviction by @mkaruza in #5894
• chore(server): Small dead code removal by @dranikpg in #5907
• feat(hset_family): create HMapWrap by @dranikpg in #5900
• chore(deps): bump actions/setup-node from 4 to 5 in the actions group by @kostasrim in #5909
• server: Support RESP3 map response for xread by @abhijat in #5898
• fix(docker): prevent zombie processes by @vyavdoshenko in #5878
• chore(deps): bump actions/setup-node from 5 to 6 in the actions group by @dependabot[bot] in #5910
• fix(search): prevent crash on HSET with same key across databases by @vyavdoshenko in #5913
• fix(stream): prevent assert failure in XDEL with non-existent ID by @vyavdoshenko in #5918
• feat: json debug memory command added by @vyavdoshenko in #5884
• feat(tiering): Decoders by @dranikpg in #5883
• fix: Allow JSON.DEBUG HELP to work without key parameter by @vyavdoshenko in #5919
• feat(server): Add shutdown watchdog by @abhijat in #5915
• chore(server): Better hash tests (Part 1) by @dranikpg in #5922
• server: Yield fiber if running for over 1ms by @abhijat in #5924
• feat(tiering): Serialized map by @dranikpg in #5896
• feat(core): External representation enum by @dranikpg in #5912
• test: fix shutdown_test by waiting for server to be ready after restart by @vyavdoshenko in #5930
• chore(hset_family): Add mutable functions to listpack wrapper by @dranikpg in #5901
• chore: code redundancies in replication metrics by @kostasrim in #5926
• fix: Handle non-STRING types in ParseRedis to prevent crash by @vyavdoshenko in #5937
• chore(server): Allow using background fibers for DEBUG OBJHIST by @dranikpg in #5933
• chore: non preemptive ProtocolClient destructor by @kostasrim in #5927
• chore(hset): Mutable HMapWrap functions by @dranikpg in #5928
• feat(tools): Vector search benchmark by @dranikpg in #5938
• search: Return a default_score with ft.info index definition by @abhijat in #5941
• server: Exit consume-redis-stream loop on error by @abhijat in #5948
• chore: reduce frequency of clock invocations by @romange in #5946
• fix(set_family): Don't replicate SPOP if nothing was removed by @mkaruza in #5936
• feat: Add AFL++ fuzzing integration with persistent mode by @vyavdoshenko in #5932
• fix: reject non-STRING arguments in RESP protocol to prevent crash by @vyavdoshenko in #5955
• fix: preserve correct command order in MULTI/EXEC transactions by @vyavdoshenko in #5954
• chore: reduce smoothing interval for local monitoring by @romange in #5965
• fix(tiering): Remove disk store files on exit by @dranikpg in #5966
• fix(tiering): Limit offload budget by time by @dranikpg in #5970
• Update helio to pull in epoll socket fix by @abhijat in #5974
• fix(search): improve FT.INFO flexibility and error messages by @vyavdoshenko in #5983
• test: Add valkey-search integration test infrastructure by @vyavdoshenko in #5981
• feat(search): Add support for multiple prefixes in search indexes by @vyavdoshenko in #5985
• chore: separate shutdown and close for ProtocolClient by @kostasrim in #5969
• fix(list_family): Fix test by @dranikpg in #5997
• fix(tiering): Basic stash backpressure by @dranikpg in #5973
• fix: macOS build by @vyavdoshenko in #6004
• chore(server): Remove OpResultTyped by @dranikpg in #6005
• fix(tiering): Grow backing async by @dranikpg in #6007
• fix(search): persist FT.SYNUPDATE synonyms across server restarts by @vyavdoshenko in #5988
• feat(ci): Add AFL++ fuzzing workflows for smoke and long tests by @vyavdoshenko in #5972
• feat: update HEXPIRE flags and behavior by @BorysTheDev in #6001
• fix(search): Fix integration test failures by @vyavdoshenko in #5991
• fix: zdiff WITHSCORES param and resp3 response by @BorysTheDev in #6013
• feat(tiering): Two phase stash by @dranikpg in #5984
• fix(search): Allow colon in tag values for query syntax by @vyavdoshenko in #6014
• tests: Fix order of used memory counter reset by @abhijat in #6012
• test: add spop benchmark by @BorysTheDev in #6016
• chore: fix dfly_bench qps calculation by @romange in #6019
• fix(test): Lower tiering test limits by @dranikpg in #6017
• chore: Enable valkey-search tests in CI by @vyavdoshenko in #6021
• feat(search): Add query string length limit for FT.SEARCH and FT.AGGREGATE by @vyavdoshenko in #6018
• chore(dfly_bench): perf improvements and support for sequential access by @romange in #5943
• fix(eviction): Limit accumulation deleted bytes during eviction by @mkaruza in #5995
• test: integration python test enabled by @vyavdoshenko in #6024
• chore: support huffman compression for string values by @romange in #6025
• feat(core): Add defrag support for json objects by @abhijat in #6023
• fix: update SPOP benchmark by @vyavdoshenko in #6038
• chore: minor fixes around huffman encoding by @romange in #6037
• fix(tiering): Recompute size before upload by @dranikpg in #6031
• feat: add KEY dimenstion for memory usage by @romange in #6043
• fix: Optimize GetRandomChain() in DenseSet by @vyavdoshenko in #6033
• chore: add LOG_XXX constants for lua redis table by @kostasrim in #6036
• fix: Implement chunking for large SBF filters by @EricHayter in #5944
• actions: Add a volume mount to container by @abhijat in #6048
• chore: remove RecvProvided from dragonfly connection by @kostasrim in #6050
• fix(rdb_load): Store streamed items per rdb lodaer by @dranikpg in #6049
• fix(search): Result set cutoff by @dranikpg in #5906
• fix: remove duplicate expired/evicted metric header by @romange in #6058

Huge thanks to all the contributors! ❤️

New Contributors

• @maets made their first contribution in #5866

Full Changelog: v1.34.0...v1.35.0