Dragonfly v1.26.0 - Chrismukkah edition 🔯 🎄🔯 🎄🔯

This marks our final release for the year, featuring numerous stability fixes and performance enhancements.

Important features and changes:

1. Numerous Search and Json fixes #4070 #4012 #4205 #4228 #4231 #4311 #4343
2. Account for list type memory, fixes #3800
3. Native support of GCS storage! Just point your --dir to gs::\mybucket\df_snapshots\
4. Fix metrics compatibility with prometheus 3.0 scraping #4277
5. Series of STREAM improvements bringing Dragonfly to be compatible with redis/valkey 7.2 streams: #4192, #4237, #4239 #4240 #4281 #4285

What's Changed

• chore: remove old io by @dranikpg in #3953
• chore: RdbSerializer::SaveListObject supports QList by @romange in #4101
• chore: Implement list Pop/Erase functionality with QList by @romange in #4099
• chore: change Namespaces to be a global pointer by @romange in #4032
• fix: update instruction how to build from the source for ubuntu by @BorysTheDev in #4103
• fix: test_noreply_pipeline flakiness by @romange in #4102
• chore: Implement LMOVE over QList by @romange in #4104
• fix(search_family): Process wrong field types in indexes for the FT.SEARCH and FT.AGGREGATE commands by @BagritsevichStepan in #4070
• chore: implement Erase for a range by @romange in #4106
• chore: decouple reply_builder from ConnectionContext by @romange in #4069
• chore: support rdb loading and container utils with QList. by @romange in #4109
• feat(server) : snapshot traverse physical buckets by @adiholden in #4084
• fix: skip Send() in SinkReplyBuilder::Flush() when vec is empty by @kostasrim in #4114
• fix: regression in squashing code when determining eval commands by @romange in #4116
• fix: slow regression tests tests by @kostasrim in #4117
• test: add test for snapshoting during migration by @BorysTheDev in #4108
• feat server: experimental_new_io flag add as deprecated by @adiholden in #4127
• chore: simplify BumpUps deduplication by @romange in #4098
• fix: Do not use cc_ in connection if it's null by @chakaz in #4131
• fix: Use MOVED error type for moved replies by @chakaz in #4125
• chore(deps): bump github/codeql-action from 3.27.0 to 3.27.1 in the actions group by @dependabot in #4115
• fix server: fix stats of pipeline squashed commands by @adiholden in #4132
• chore: fix plain node insertion by @romange in #4134
• chore: improve the state machine of RedisParser by @romange in #4085
• chore: run tests with list_experimental_v2 enabled by @romange in #4112
• chore: optimize info command by @romange in #4137
• fix server: fix write to slowlog on squashing flow by @adiholden in #4138
• chore:update fakeredis, remove irrelevant tests by @cunla in #4014
• chore: fix a potential crash during client list by @romange in #4141
• fix: migration ACK response processing by @BorysTheDev in #4140
• feat: add INFO memory section for squashing replies memory consuming by @BorysTheDev in #4147
• fix: allow SELECT in multi/exec if it's a noop by @romange in #4146
• chore(deps): bump github/codeql-action from 3.27.1 to 3.27.4 in the actions group by @dependabot in #4148
• chore: tune logs and improve restrict denied error by @romange in #4145
• fix: UB during cmd squashing reply size calculation by @BorysTheDev in #4149
• fix: enforce load limits when loading snapshot by @romange in #4136
• fix: test_replication_all failure by @romange in #4155
• fix: Huge entries fail to load outside RDB / replication by @chakaz in #4154
• chore: allow slow and precise memory measurement of an object by @romange in #4160
• chore: fix a regression build break by @romange in #4162
• chore: log exit code of failing dragonfly in tests by @romange in #4166
• fix: remove DenseSet::IteratorBase::TraverseApply by @kostasrim in #4170
• adjusted grafana dashboard to be more user friendly by @applike-ss in #4165
• chore: fix shutdown sequence in Dragonfly server by @romange in #4168
• feat(server): add eval_ro and evalsha_ro by @andydunstall in #4091
• chore: Hide replicas from CLUSTER subcmds in managed mode by @chakaz in #4174
• fix(tools): Prevent dragonfly.logrotate from stopping logrotate service by @s0xDk in #4176
• chore: more parser improvements by @romange in #4177
• fix(release/helm): allow empty commits for rerun by @Pothulapati in #4163
• fix(search_family): Fix LOAD fields parsing in the FT.AGGREGATE and FT.SEARCH commands by @BagritsevichStepan in #4012
• feat: Huge values breakdown in cluster migration by @chakaz in #4144
• feat: cluster migrations restarts if timeout happens by @BorysTheDev in #4081
• fix: deduplicate mget response by @romange in #4175
• fix: stop using openssl for container healthchecks by @romange in #4181
• chore: produce core files in regtests by @romange in #4185
• chore: stop periodic task earlier during the shutdown process by @romange in #4187
• chore(deps): bump github/codeql-action from 3.27.4 to 3.27.5 in the actions group by @dependabot in #4186
• test: skip test_cluster_mgr because of unclosed instance by @BorysTheDev in #4191
• chore: fix false positives sanitizers by @kostasrim in #4190
• fix: debug object encoding names by @romange in #4188
• feat: add ability reading stream_listpacks_2/3 rdb types by @BorysTheDev in #4192
• chore: get back on the decision to put a hard limit on command interface by @romange in #4203
• chore: qlist improvements by @romange in #4194
• fix: stream memory tracking by @kostasrim in #4067
• chore: fully support qlist in all list_family commands by @romange in #4201
• feat(contrib/helm): evaluate the provided passwordSecretName value as a template by @oOraph in #4046
• fix(search_family): Fix crash when no SEPARATOR is specified in the FT.CREATE command by @BagritsevichStepan in #4205
• chore: update command interface for hset/set families by @romange in #4209
• fix: cluster_mgr script by @BorysTheDev in #4210
• bug(replication): snapshot cleanup fix in transition to stable sync by @adiholden in #4211
• test: fix and unskip test_migration_timeout_on_sync by @BorysTheDev in #4216
• chore: Add more qlist tests by @romange in #4217
• chore: change the interface of stream and server commands by @romange in #4219
• chore: add noop docs subcmd by @kostasrim in #4214
• chore: remove tail field from qlist by @romange in #4220
• chore: qlist code clean ups by @romange in #4223
• chore: Track QList memory by @romange in #4226
• chore: change the interface of hll, generic and list commands by @romange in #4227
• fix: command docs by @kostasrim in #4229
• chore: support qlist compression when accounting for memory by @romange in #4233
• chore: update command interface for bitops, bloom, zset families by @romange in #4235
• Feat(Helm chart): Add env vars into pods by @andreibe24 in #4196
• chore: fix bugs in stream_family by @romange in #4237
• fix: bugs in stream code by @romange in #4239
• fix: Fix test_network_disconnect_during_migration test by @chakaz in #4224
• fix: RdbTest.LoadStream3 incorrect file usage by @BorysTheDev in #4242
• chore: Hide replica info in real cluster if --managed_service_info by @chakaz in #4241
• chore: Hide managed service info in INFO by @chakaz in #4248
• fix: stream bugs by @romange in #4240
• fix: command interface for json_family by @romange in #4243
• fix(server): fix memory leak on lua error by @adiholden in #4236
• chore: update command interface for acl, hll families by @romange in #4253
• chore: split RecordExpiry preemptive and non-preemptive flows by @kostasrim in #4252
• fix(rax_tree): Fix crash caused by destructor in RaxTreeMap by @BagritsevichStepan in #4228
• chore: update command interface for cluster and search families by @romange in #4258
• refactor: remove redundant allocations for streamer by @BorysTheDev in #4225
• fix(save): don't create dirs for cloud dir by @andydunstall in #4259
• feat: add support for big values in SeederV2 by @kostasrim in #4222
• chore: remove DbSlice mutex and add ConditionFlag in SliceSnapshot by @kostasrim in #4073
• fix(eviction): Tune eviction threshold in cache mode by @BagritsevichStepan in #4142
• fix: test_network_disconnect_during_migration data size was too big f… by @BorysTheDev in #4260
• fix(geo): support case insensitive units by @andydunstall in #4264
• chore: monotonically increasing ports for cluster tests by @kostasrim in #4268
• fix: close socket to prevent onCompletion call after the journal stre… by @BorysTheDev in #4270
• chore: Fix test_rss_used_mem_gap for all types by @chakaz in #4254
• feat server: support config set serialization_max_chunk_size by @adiholden in #4274
• fix: prohibit read commands during takeover by @BorysTheDev in #4267
• fix cluster: migration traverse bug by @adiholden in #4279
• fix(server) : dont apply eviction on rss over limit by @adiholden in #4276
• chore: let resp parser provide more useful logs by @romange in #4273
• chore: update command interface for main_service commands by @romange in #4265
• fix: support loading of 7.x streams correctly by @romange in #4281
• chore: fixes the parse error for xread/xreadgroup with unbalanced ids by @romange in #4266
• fix: Do not attempt to defrag StringSet as a StringMap by @chakaz in #4283
• chore: add active time to stream consumers by @romange in #4285
• fix: mismatch new-delete in unit test by @chakaz in #4288
• fix(search_family): Remove the output of extra fields in the FT.AGGREGATE command by @BagritsevichStepan in #4231
• fix: increase cluster migration default timeout by @BorysTheDev in #4293
• chore: test metrics for huge value serialization by @kostasrim in #4262
• chore: update reflex version and fix its build on alpine by @romange in #4304
• fix: circular dependency in qlist by @romange in #4302
• feat: add migration_finalization_timeout_ms flag by @BorysTheDev in #4301
• fix: Protect BumpUp() from running in parallel to serialization by @chakaz in #4307
• fix(search_family): Fix FT.AGGREGATE output by @BagritsevichStepan in #4311
• chore: improve parser state machine by @romange in #4313
• test: update logs and test for debug purpose by @BorysTheDev in #4309
• server: report redis version 7.2.0 to support Sidekiq by @adiholden in #4286
• chore: factor out rdb_load utilities into separate files by @romange in #4315
• fix: Remove hardcoded @assert_eventually 100 times retry by @chakaz in #4318
• refactor: remove serialization_max_chunk_size for cluster tests by @BorysTheDev in #4316
• chore: factor out CompressorImpl into separate files by @romange in #4319
• fix: Do not preempt on dispatcher fiber by @chakaz in #4323
• fix(test): seeder test remove check by @adiholden in #4320
• fix: potential OOM when first request sent in small bits by @romange in #4325
• fix: crash during getting info about replication by @BorysTheDev in #4328
• chore: add ability to track connections stuck at send by @romange in #4330
• fix: do not check-fail in OpRestore by @romange in #4332
• test: skip test_network_disconnect_during_migration because it is uns… by @BorysTheDev in #4334
• chore: fix wording around the dispatch fiber in dragonfly_connection by @romange in #4333
• fix (regression tests): skip flaky test by @adiholden in #4336
• chore: Disable failing test by @chakaz in #4337
• fix: cluster tests stability by @BorysTheDev in #4338
• fix(pytest): call stop for all instances even if stop raise exception by @adiholden in #4339
• fix: add content-type for metrics response by @romange in #4340
• fix(server): use compression for non big values by @adiholden in #4331
• fix: Stack overflow in DFLYCLUSTER CONFIG by @chakaz in #4342
• fix(search_family): Fix logging in ParseFieldWithAtSign by @BagritsevichStepan in #4343

Huge thanks to all the contributors! ❤️

New Contributors

• @s0xDk made their first contribution in #4176
• @oOraph made their first contribution in #4046
• @andreibe24 made their first contribution in #4196

Full Changelog: v1.25.0...v1.26.0