Dragonfly v1.16.0

Our spring release. We are getting closer to 2.0 with some very exciting features ahead. Stay tuned!

Some prominent changes include:

• Improved memory accounting of client connections (#2710 #2755 and #2692 )
• FT.AGGREGATE call (#2413)
• Properly handle and replicate Memcache flags (#2787 #2807)
• Intoduce BF.AGGREGATE BD.(M)ADD and BF.(M)EXISTS methods (#2801). Note, that it does not work with snapshots and replication yet.
• Dragonfly builds natively on MacOS. We would love some help with extending the release pipeline to create a proper macos binary.
• Following the requests from the Edge developers community, we added a basic HTTP API support! Try running Dragonfly with:
  --expose_http_api flag and then call curl -X POST -d '["ping"]' localhost:6379/api. We will follow up with more extensive docs later this month.
• Lots of stability fixes, especially around Sidekiq and BullMQ workloads.

What's Changed

• chore: make usan asan optional and enable them on CI by @kostasrim in #2631
• fix: missing manual trigger for daily sanitizers by @kostasrim in #2682
• bug(tiering): fix overflow in page offset calculation and wrong hash offset calculation by @theyueli in #2683
• Chore: Fixed Docker Health Check by @manojks1999 in #2659
• chore: Increase disk space in the coverage runs by @romange in #2684
• fix(flushall): Decommit memory after releasing tables. by @chakaz in #2691
• feat(server): Account for serializer's temporary buffer size by @chakaz in #2689
• refactor: remove FULL-SYNC-CUT cmd #2687 by @BorysTheDev in #2688
• chore: add malloc-based stats and decommit by @romange in #2692
• feat(cluster): automatic slot migration finalization #2697 by @BorysTheDev in #2698
• Basic FT.AGGREGATE by @dranikpg in #2413
• chore: little transaction cleanup by @dranikpg in #2608
• fix(channel store): add acquire/release pair in fast update path by @dranikpg in #2704
• chore: add ubuntu22 devcontainer by @romange in #2700
• feat(cluster): Add --cluster_id flag by @chakaz in #2695
• feat(server): Use mimalloc in SSL calls by @chakaz in #2710
• chore: Pull helio with new BlockingCounter by @dranikpg in #2711
• chore(transaction): Simplify PollExecution by @dranikpg in #2712
• chore(transaction): Don't call GetLocalMask from blocking controller by @dranikpg in #2715
• chore: improve compatibility of EXPIRE functions with Redis by @romange in #2696
• chore: disable flaky fuzzy migration test by @kostasrim in #2716
• chore: update sanitizers workflow by @kostasrim in #2686
• chore: Use c-ares for resolving hosts in ProtocolClient by @chakaz in #2719
• Remove check-fail in ExpireIfNeeded and introduce DFLY LOAD by @romange in #2699
• chore: Record cmd stat from invoke by @dranikpg in #2720
• fix(transaction): nullptr access on non transactional commands by @kostasrim in #2724
• fix(BgSave): async from sync by @kostasrim in #2702
• chore: remove core/fibers by @romange in #2723
• fix(transaction): Replace with armed sync point by @dranikpg in #2708
• feat(json): Deserialize ReJSON format by @chakaz in #2725
• feat: add flag masteruser by @kostasrim in #2693
• refactor: block list refactoring #2580 by @BorysTheDev in #2732
• chore: fix DeduceExecMode by @dranikpg in #2733
• fix(cluster): Reply with correct \n / \r\n from CLUSTER sub cmd by @chakaz in #2731
• chore: Introduce fiber stack allocator by @romange in #2730
• fix(cluster): Save replica ID per replica by @chakaz in #2735
• fix(ssl): Proper cleanup by @chakaz in #2742
• chore: add skeleton files for flat_dfs code by @romange in #2738
• chore: better error reporting when connecting to tls with plain socket by @romange in #2740
• chore: Support json paths without root selector by @dranikpg in #2747
• chore: journal cleanup by @romange in #2749
• refactor: remove start-slot-migration cmd #2727 by @BorysTheDev in #2728
• feat(server): Add TLS usage to /metrics and INFO MEMORY by @chakaz in #2755
• chore: preparations for adding flat json support by @romange in #2752
• chore(transaction): Introduce RunCallback by @dranikpg in #2760
• feat(replication): Do not auto replicate different master by @chakaz in #2753
• Improve Helm chart to be rendered locally and on machines where is not the application target by @fafg in #2706
• chore: preparation for basic http api by @romange in #2764
• feat(server): Add metric for RDB save duration. by @chakaz in #2768
• chore: fix flat_dfs read tests. by @romange in #2772
• fix(ci): do not overwrite last_log_file among tests by @kostasrim in #2759
• feat(server): support cluster replication by @adiholden in #2748
• fix: fiber preempts on read path and OnCbFinish() clears fetched_items_ by @kostasrim in #2763
• chore(ci): open last_log_file in append mode by @kostasrim in #2776
• doc(README): fix outdated expiry ranges description by @enjoy-binbin in #2779
• Benchmark runner by @adiholden in #2780
• chore(replication-tests): add cache_mode on test replication all by @kostasrim in #2685
• feat(tiering): DiskStorage by @dranikpg in #2770
• chore: add a boilerplate for bloom filter family by @romange in #2782
• chore: introduce conversion routines between JsonType and FlatJson by @romange in #2785
• chore: Fix memcached flags not updated by @dranikpg in #2787
• chore: remove duplicate code from dash and simplify by @kostasrim in #2765
• chore: disable test_cluster_slot_migration by @kostasrim in #2788
• fix: new[] delete[] missmatch in disk_storage by @kostasrim in #2792
• fix: sanitizers clang build and clean up some warnings by @kostasrim in #2793
• chore: add bloom filter class by @romange in #2791
• chore: add SBF data structure by @romange in #2795
• chore(tiering): Disable compilation for MacOs by @dranikpg in #2799
• chore: fix daily build by @romange in #2798
• chore: expose SBF via compact_object by @romange in #2797
• fix(ci): malloc trim on sanitizers workflow by @kostasrim in #2794
• fix(cluster): Don't miss updates in FLUSHSLOTS by @chakaz in #2783
• feat: add bf.(m)add and bf.(m)exists commands by @romange in #2801
• fix: SBF memory leaks by @kostasrim in #2803
• chore: refactor StringFamily::Set to use CmdArgParser by @romange in #2800
• fix: propagate memcached flags to replica by @romange in #2807
• DFLYMIGRATE ACK refactoring by @BorysTheDev in #2790
• feat: add master lsn and journal_executed dcheck in replica via ping by @kostasrim in #2778
• fix: correct json response for errors by @romange in #2813
• chore: bloom test - cover corner cases by @romange in #2806
• bug(server): do not write lsn opcode to journal by @adiholden in #2814
• chore: Fix build by disabling the tests. by @romange in #2821
• fix(replication): replication with multi shard sync enabled lagging by @adiholden in #2823
• fix: io_uring/fibers bug in DnsResolve by @romange in #2825

New Contributors

• @manojks1999 made their first contribution in #2659
• @fafg made their first contribution in #2706
• @enjoy-binbin made their first contribution in #2779

Huge thanks to all the contributors! ❤️

🇮🇱 🇺🇦

Full Changelog: v1.15.0...v1.16.0