Dragonfly v1.4.0

New Dragonfly release! Some prominent changes include:

• Support for several STREAM commands (xinfo, xtrim, read) - Thanks @Abhra303 and @andydunstall !
• Implementation of bzpopmin and zdiff functionalities - thanks to @royjacobson and @kostasrim !
• Resolving high latency issues encountered in specific corner cases: @royjacobson
• Addressing stability issues reported by our community.
  I would like specifically mention the help of one of the discord members that assisted with finding
  the root cause with BLPOP bug. Thank you, glitch!
  In addition, @andydunstall contributed many fixes across the board, especially with BullMQ compatibility.

What's Changed

• fix(docker): Install libxml2 as part of ubuntu Dockerfile by @Pothulapati in #1239
• refactor(cluster): Move CLUSTER commands to their own methods. by @chakaz in #1230
• feat(stream) Add support for XREAD by @andydunstall in #1235
• fix: Dragonfly ignoring NOREPLY option for Memcached commands by @talbii in #1233
• fix(replication): Don't open a replication flow in non-shard threads by @royjacobson in #1234
• fix(stream): avoid using designated initializer by @Abhra303 in #1247
• fix: fix multi test by @dranikpg in #1236
• chore: factor out commandid from the server lib by @romange in #1253
• Feat bzpopmin by @royjacobson in #1232
• feature(server): Bring back inline scheduling by @royjacobson in #1130
• refactor(cluster): Move management commands from DFLY CLUSTER prefix to DFLYCLUSTER. by @chakaz in #1251
• feat(cluster): cluster config delete now owned slots data by @adiholden in #1241
• chore: Update helio by @royjacobson in #1257
• fix: actually zero initialize the struct by @royjacobson in #1256
• fix: support XREAD ... STREAMS ... keys derivation by @romange in #1250
• perf: Use InlinedVector in TransactionData for the common case of single commands by @royjacobson in #1260
• chore: Try to turn on Werror in the CI by @royjacobson in #1261
• fix: S3 writes are now working with new format as well by @romange in #1252
• feat(cluster): Implement CLUSTER INFO. by @chakaz in #1258
• feat(cluster): Do not allow node IDs to appear more than once. by @chakaz in #1271
• fix(cluster): Fix breaking test by @chakaz in #1273
• feat: Use journal LSNs for absolute replication offsets by @royjacobson in #1242
• fix: remove xread required arguments by @andydunstall in #1263
• Update README.md by @worldsoup in #1279
• chore: Add clang workflow to CI by @royjacobson in #1220
• feat(cluster): support dflycluster myid by @adiholden in #1272
• Refactor search AST by @dranikpg in #1222
• fix: move version check earlier in the flow by @romange in #1255
• Reply with MOVED error for non-owned slots. by @chakaz in #1283
• fix: fix dispatch ordering by @dranikpg in #1254
• feat(cluster): dflycluster commands only under admin port by @adiholden in #1281
• feat(cluster): Implement CLUSTER SHARDS. by @chakaz in #1284
• feat(streams): Add support for XREAD BLOCK by @andydunstall in #1291
• feat(cluster): select not allowed in cluster mode by @adiholden in #1288
• feat(cluster): Implement CLUSTER SLOTS. by @chakaz in #1289
• bug(memory): read LXC memory.limit_in_bytes too by @hermanbanken in #1293
• refactor(cluster): Unify cluster-disabled error message. by @chakaz in #1296
• feat(cluster): Implement CLUSTER NODES command. by @chakaz in #1298
• chore: Silence unwanted asyncio clutter by @royjacobson in #1299
• Search: Basic shard state by @dranikpg in #1249
• bug(cluster): Fix failing cluster test. by @chakaz in #1300
• chore: improve search tests + fixes by @dranikpg in #1303
• fix: fix search test by @dranikpg in #1306
• feat(cluster): first config set drop all entries not owned by node by @adiholden in #1301
• Add bison to build process, and libxml2 to runtime by @chakaz in #1308
• fix: Batch array length before sending its contents by @romange in #1287
• Add latest dependencies. by @chakaz in #1310
• feat(server): Add versioning to dragonfly replication by @royjacobson in #1307
• fix: werror in ci and warning that slipped by by @royjacobson in #1297
• feat: Support ACKs from replica to master by @royjacobson in #1243
• refactor(cluster): Use GetEmulatedShardInfo() for CLUSTER INFO. by @chakaz in #1317
• fix: Clarify logging for snapshots loading. by @royjacobson in #1312
• fix: remove redundant list assertions by @andydunstall in #1338
• feat(streams): Add support for XTRIM by @andydunstall in #1337
• Use search indices by @dranikpg in #1294
• fix: Skip optional WEIGHT param in ft.create by @dranikpg in #1340
• Chore: Clean up interface of Replica::ReadRespReply by @royjacobson in #1330
• feat(cluster): Allow actions on non-owned keys on replication port. by @chakaz in #1334
• chore: Double regression tests timeout (10m->20m) by @chakaz in #1350
• feat: Support tags in search by @dranikpg in #1341
• fix(cluster): Fix crash when getting DFLYCLUSTER GETSLOTINFO with no other args. by @chakaz in #1351
• fix(server): Return null when HRANDKEY doesn't find a key. by @chakaz in #1354
• feat(server): debug poplulate add slots option by @adiholden in #1348
• fix: fix search_test after incorrect rebase by @dranikpg in #1356
• feat(zset family): Implement ZDiff command issue #1311 by @kostasrim in #1333
• feat(cluster): Add total_reads and total_writes to GETSLOTSINFO. by @chakaz in #1355
• fix(cluster): Fix failing test due to automatic merge by @chakaz in #1359
• fix(replication): Restart replication upon receiving FLUSHALL during full sync by @chakaz in #1347
• test(cluster): Test that cluster works with a standard cluster client. by @chakaz in #1336
• chore(server): Move tests to their respective subdir by @chakaz in #1360
• chore: Add blpop crash diagnostics by @dranikpg in #1358
• fix: fix linsert pivot by @andydunstall in #1365
• fix(alpine): Add libxml2 into the build pipeline by @Pothulapati in #1363
• fix(replication): Destroy rdb saver on the same thread it was created by @chakaz in #1366
• feat(ci): Test the built docker image by @Pothulapati in #1240
• CGroup v1 detection by @talbii in #1329
• bug(debug populate): fix debug populate keys by @adiholden in #1370
• feat(cluster): Use thread-local cluster config by @chakaz in #1361
• fix(server): Do more aggressive batching. by @royjacobson in #1375
• fix(xinfo): send integer info in correct type by @Abhra303 in #1379
• feat: search limit by @dranikpg in #1367
• Fix reply builder access issue by @dranikpg in #1378
• fix: aggregate xread and xrange replys by @andydunstall in #1383
• fix: aggregate eval replys by @andydunstall in #1386
• feat: Basic search optimizations by @dranikpg in #1385
• fix: A more gracefull shutdown by @royjacobson in #1388
• chore: fix some of FreeBsd compile issues. by @romange in #1374
• fix: Allow readonly commands in replica script by @dranikpg in #1392
• feat: track update and delete for search by @dranikpg in #1325
• Doc: Warn people about benchmarking/deploying without LTO by @royjacobson in #1396
• feat(xgroup): add support for xgroup createconsumer by @Abhra303 in #1382
• Fix crash on shutdown from #1388 by @royjacobson in #1399
• fix: remove NotifyPending from UnwatchShardCb by @andydunstall in #1402
• fix(lua): Fix build on arm by @chakaz in #1404
• feat: Basic KNN (core only) by @dranikpg in #1397
• feat(cluster): Use shared_ptr for thread-local cluster config by @chakaz in #1395
• fix(test): Fix a race condition in test_flushall_in_full_sync by @royjacobson in #1409
• Put more keys, make it non-failure if the test fails because of bad params by @royjacobson in #1411
• test(cluster): Add a Cluster Management script by @chakaz in #1390
• bug(helm) Fix tolerations and affinity mishandling in Helm template by @kbelokon in #1373
• fix: remove bad check-fail in the transaction code by @romange in #1420
• fix: drain the connection socket on a protocol error by @andydunstall in #1422
• fix: Don't remove non-concluding tx from queue on ooo runs by @dranikpg in #1427

New Contributors

• @hermanbanken made their first contribution in #1293
• @kostasrim made their first contribution in #1333
• @kbelokon made their first contribution in #1373

Huge thanks to all the contributors! ❤️

Full Changelog: v1.3.0...v1.4.0