Dragonfly v1.25.0

Some prominent changes include:

• FT.SEARCH fixes: #3956 #3955 #4043
• Prefix search - supporting queries like "@color:{gre* | orange | yellow*}: #3972
• GCS save/load support #4006
• Few Improvements around replication/snapshotting - it should be more efficient around large entries #4041
• zrevrank/zrank support withscore option #4001

What's Changed

• chore: ignore applying the same cluster config twice by @BorysTheDev in #3932
• test: add test to reproduce a lot of memory consumtion during migration by @BorysTheDev in #3939
• chore(refactoring): header clean ups by @romange in #3943
• chore: get rid of ToUpper call and use AsciiStrToUpper by @romange in #3944
• chore: remove ToUpper calls in main_service by @romange in #3947
• chore: get rid of ToUpper/ToLower mutations on arguments by @romange in #3950
• chore(facade): Fix bad new IO glue by @dranikpg in #3940
• feat: track request sizes histograms by @romange in #3951
• chore: pass SinkReplyBuilder and Transaction explicitly. Part1 by @romange in #3946
• chore: disable test_cluster_memory_consumption_migration by @kostasrim in #3948
• fix(search_family): Temporary remove the error when a field name does not have the '@' sign at the beginning in the FT.AGGREGATE command by @BagritsevichStepan in #3956
• chore: tune test_rss_used_mem_gap by @kostasrim in #3958
• test: add test to test big collections or collections with big values by @BorysTheDev in #3959
• chore: increase load in test_noreply_pipeline by @kostasrim in #3960
• chore(fakeredis): Enable JSON tests in the Fakeredis tests by @BagritsevichStepan in #3773
• chore: add defrag logic for zsets by @kostasrim in #3836
• chore: pass SinkReplyBuilder and Transaction explicitly. Part2 by @romange in #3954
• chore: pass SinkReplyBuilder and Transaction explicitly. Part3 by @romange in #3966
• fix: crash if we OOM during migration process by @BorysTheDev in #3968
• fix(cluster): fix backoff on unknown migration by @andydunstall in #3965
• chore: pass SinkReplyBuilder and Transaction explicitly. Part4 by @romange in #3967
• chore: get rid of MutableSlice by @romange in #3952
• chore: reduce dependency of debug/memory commands on ConnectionContext by @romange in #3977
• fix: properly set object with DenseLinkKey by @romange in #3980
• fix(memory-leak): add missing destructor to RaxTreeMap by @kostasrim in #3975
• fix: dangling string in DebugCmd::Reload by @kostasrim in #3981
• chore: pass SinkReplyBuilder and Transaction explicitly. Part5 by @romange in #3976
• chore: pass SinkReplyBuilder and Transaction explicitly. Part6 by @romange in #3987
• fix(build): Reduced checks permission in build to read #3821 by @irufus in #3916
• fix: properly set ttl bit during object replacement by @romange in #3991
• fix: replace string_view with string in RaxTreeMap::FindIterator by @kostasrim in #3982
• fix: prevent starting migrations from replica by @BorysTheDev in #3985
• feat(search): Prefix search for tags by @dranikpg in #3972
• chore: pass SinkReplyBuilder and Transaction explicitly. Part8 by @romange in #3992
• chore: pass SinkReplyBuilder and Transaction explicitly. Part7 by @romange in #3988
• chore: pass SinkReplyBuilder and Transaction explicitly. Part9 by @romange in #3996
• fix: keep the ttl bit when dense_set grows with expiring items by @romange in #3995
• refactor(workflow): changes default permissions to read for few workflows increasing security by @BLANKatGITHUB in #3905
• fix(search_family): Fix indexes loading in the FT.SEARCH and Ft.AGGREGATE commands by @BagritsevichStepan in #3955
• feat: introduce metrics/logs of when pipelining is being throttled by @romange in #4000
• chore: clean up conn_use_incoming_cpu heuristic by @romange in #3978
• chore: refactor snapshot expanding logic by @romange in #4003
• chore: introduce GcsSnapshotStorage by @romange in #4004
• fix: flaky test_failover test by @romange in #4007
• chore: pass SinkReplyBuilder and Transaction explicitly. Part10 by @romange in #3998
• fix: build break in search_family by @romange in #4008
• feat(zset_family): support WITHSCORE in zrevrank/zrank commands (#3921) by @Diskein in #4001
• fix: separate Heartbeat and ShardHandler to fibers by @kostasrim in #3936
• chore: improve verbosity logging during shutdown by @romange in #4010
• chore: skip AggregateWithLoadOptionHard on sanitizers (false positive) by @kostasrim in #4013
• chore: pass RedisReplyBuilder explicitly from dragonfly connection by @romange in #4009
• chore: print info stats if test_noreply_pipeline fails by @romange in #4016
• chore: flush journal to channel for non auto journal commands by @kostasrim in #3945
• chore: support load/save from GCS by @romange in #4006
• chore: pull helio by @romange in #4036
• chore: decrease logs severity for OOM from error to warning by @BorysTheDev in #4031
• feat(pre-commit): adds senstive data check like api keys by @BLANKatGITHUB in #4030
• test: update test_big_containers by @BorysTheDev in #4025
• fix: reduce memory consumption during migration by @BorysTheDev in #4017
• chore: add tx queue head debug info in AnalyzeTxQueue by @kostasrim in #4026
• chore: refactor part of s3 logic for loading a snapshot. by @romange in #4044
• chore: add retry to test_noreply_pipeline test by @romange in #4045
• chore(DenseSet): defrag all links in a chain by @kostasrim in #4019
• chore: add ForceUpdate to BPTree by @kostasrim in #3993
• fix(search_family): Fix crash in FT.PROFILE command for invalid queries by @BagritsevichStepan in #4043
• feat(dependabot): Add dependabot to check for vulnerabilies and updat… by @BLANKatGITHUB in #4035
• chore: pass reply_builder explicitly to pubsub module by @romange in #4021
• chore(deps): bump numpy from 1.24.1 to 2.1.3 in /tools by @dependabot in #4063
• chore(deps): bump github.com/redis/go-redis/v9 from 9.5.1 to 9.7.0 in /tools/replay by @dependabot in #4062
• chore(deps): bump tomli from 2.0.1 to 2.0.2 in /tests/dragonfly by @dependabot in #4059
• chore(deps): bump github.com/gruntwork-io/terratest from 0.41.11 to 0.47.2 in /contrib/charts/dragonfly by @dependabot in #4058
• chore(deps): bump redis-om from 0.2.2 to 0.3.3 in /tests/dragonfly by @dependabot in #4060
• chore(deps): bump mikepenz/action-junit-report from 4 to 5 by @dependabot in #4052
• chore(deps): bump mozilla-actions/sccache-action from 0.0.4 to 0.0.6 by @dependabot in #4051
• chore(deps): bump ossf/scorecard-action from 2.3.1 to 2.4.0 by @dependabot in #4053
• chore(deps): bump github/codeql-action from 3.24.9 to 3.27.0 by @dependabot in #4054
• chore(deps): bump docker/build-push-action from 5 to 6 by @dependabot in #4055
• chore(deps): bump pytest-repeat from 0.9.1 to 0.9.3 in /tests/dragonfly by @dependabot in #4057
• chore: eliminate redundant ConnectionContext arguments by @romange in #4065
• feat(server): dont use channel for replication / save df by @adiholden in #4041
• feat: support GCS storage for saving/loading snapshots by @romange in #4064
• chore: parser clean ups by @romange in #4077
• chore: add QList class by @romange in #4078
• chore: add push functionality to QList class by @romange in #4079
• chore: Add QList::Iterator by @romange in #4082
• Add index based iterator to QList interface by @romange in #4083
• chore(dependabot): Allows only security patches by @BLANKatGITHUB in #4075
• chore: implement QList::Iterate by @romange in #4086
• chore(acl): Implicit categories by @dranikpg in #3411
• chore: implement QList::Insert function by @romange in #4087
• fix(sanitizers): disable false positive FtProfileInvalidQuery by @kostasrim in #4090
• chore: implement QList::Erase functionality by @romange in #4092
• chore: Add initial bindings for QList in list_family by @romange in #4093
• fix(server): handle running script load inside multi by @adiholden in #4074

New Contributors

• @irufus made their first contribution in #3916
• @BLANKatGITHUB made their first contribution in #3905

Huge thanks to all the contributors! ❤️

Full Changelog: v1.24.0...v1.25.0