Dragonfly v1.9.0

This release includes a number of important stability and security fixes, as well as some features that have been requested by users for a long time:

1. ACLs : starting from this release, you can now create multiple users in Dragonfly and control their access via acl categories. The following ACL sub-commands are now supported:
   ACL SETUSER/DELUSER/WHOAMI/LIST. Please note that ACLS are not persisted yet through process restarts.
2. Improved sorted-sets. We rolled out our experimental support for sorted sets (aka ZSET). They require 35% less memory and are more CPU efficient, especially for large sets. To try them now with --use_zset_tree.
3. Support for CONFIG GET/SET - now some runtime options can be changed without restarting the server (#1700).
4. The maxclients flag now prevents the number of client connections from growing indefinitely (#1496).
5. New flags max_multi_bulk_len and max_client_iobuf_len control the limits for large requests. The former controls how many elements can be sent via arrays (for commands like MSET, SADD etc). The latter controls maximum buffer size when accepting string blobs (#1771) .
6. HTTP endpoint authorization: now our HTTP endpoint is protected with the same token that is passed with --requirepass=token (#1792)
   Once basic auth is set, one can access the http port via curl -u user:token localhost:6379/

What's Changed

• feat(server): add oom guard by @adiholden in #1650
• fix(server): update denyoom flag for all commands by @adiholden in #1651
• test(server): Run all unit tests with --force_epoll by @chakaz in #1672
• chore: Refactor SortedMap by @romange in #1666
• fix(VersionMonitor): request to update on older version by @kostasrim in #1674
• feat(server): support config set for some flags by @adiholden in #1624
• opt(server): Execute lua on target shard, if it's 1 by @chakaz in #1639
• introduce --replicaof flag by @talbii in #1583
• test(server): test transaction locked keys by @adiholden in #1669
• chore: Connection fixes by @dranikpg in #1663
• fix: fix dchecks by @dranikpg in #1681
• chore: refactor a common code into a function by @romange in #1685
• fix(daily-builds): temporarily comment out bullmq tests by @Abhra303 in #1679
• opt(server): Run Lua on remote thread only if it's a different thread by @chakaz in #1684
• chore: get rid of cmdstats_map by @romange in #1687
• chore: move RdImpl functions together by @romange in #1694
• fix(ci): use auth mechanism with tls helm values by @Pothulapati in #1696
• feat(helm) - Enable configuring service type LoadBalancer with a Static IP by @mutiadavid in #1689
• chore: implemente User and UserRegistry classes by @kostasrim in #1693
• chore: several improvements around sorted map by @romange in #1699
• security: Fix the password hashing in UserRegistry. by @royjacobson in #1702
• fix: allow integer tokens in search queries by @dranikpg in #1701
• feat: Update search index on expiry by @dranikpg in #1697
• fix: MONITOR now works for multi transactions by @dranikpg in #1675
• feat: Pipeline squashing by @dranikpg in #1619
• fix: Fix squashing, pytest arg formatting by @dranikpg in #1712
• fix: fix batching during dispatch mode changes by @dranikpg in #1713
• feat: Support unicode strings in search by @dranikpg in #1698
• feat: json paths in search by @dranikpg in #1695
• feat: introduce partial DfImpl implementation for sorted_map by @romange in #1714
• feat: implement GetRange/GetLexRange for bptree. by @romange in #1715
• chore: add acl categories to all commands by @kostasrim in #1711
• fix: Add small timeout to monitor by @dranikpg in #1718
• replication: fix misleading log message about full sync by @royjacobson in #1717
• chore: clean up save by @dranikpg in #1709
• feat: implement DeleteRange functions in DfImpl by @romange in #1716
• tools: Hash defrag script by @dranikpg in #1723
• fix: Make restore accept ttl in ms by @dranikpg in #1724
• feat(AclFamilly): add acl list command by @kostasrim in #1722
• chore: Add a daily coverage run by @royjacobson in #1437
• chore: Mark slow pytests as 'slow' so they can be excluded. by @royjacobson in #1720
• fix: Correctly read container limits when running under k8s by @talbii in #1692
• fix(AclFamily): commands parsing and error handling by @kostasrim in #1726
• feat(AclFamily): add acl setuser command by @kostasrim in #1725
• feat: search index persistence by @dranikpg in #1721
• fix: add proper printing of sha256 passwords by @kostasrim in #1728
• chore(gh-workflows): add regression tests to release job by @kostasrim in #1665
• fix(ACL): Fix the iteration over categories loop in AclToString by @royjacobson in #1731
• chore: Split the build & test into 2 stages by @royjacobson in #1733
• fix: Fix inconsistency when deleting dash entries by @romange in #1734
• feat: Defragmentation for hash values by @dranikpg in #1727
• chore: add clear metric for the fragmentation waste by @romange in #1735
• fix: fix search test by @dranikpg in #1739
• chore: integrate sccache by @romange in #1738
• chore: cover zset over bptree implementation in the CI. by @romange in #1736
• feat(AclFamily): add AUTH for acl members by @kostasrim in #1732
• fix: fix index loading by @dranikpg in #1742
• fix: the bug where iobuf should be enlarged during the replication by @romange in #1744
• feat(server): support multi eval in lock ahead mode by @adiholden in #1662
• fix: extend CI running time by @romange in #1749
• fix: fix defrag stats by @dranikpg in #1740
• fix: fix memcache bugs by @romange in #1745
• feat(server): Support limiting the number of open connections. by @royjacobson in #1670
• fix: Run defrag on dbs > 0 as well by @dranikpg in #1737
• fix(server): rdb loader catch bad alloc by @adiholden in #1748
• docs: add snapshot_cron flag in README (English and zh-CN) by @Niennienzz in #1729
• fix: broken memcached error reporting by @kostasrim in #1741
• feat(search): named return values by @dranikpg in #1746
• bug(server): command stats show origin command name by @adiholden in #1761
• fix(replication): Handle errors instead of crashing. by @royjacobson in #1757
• fix: make Dragonfly compatible with older systems by @romange in #1755
• fix(server): service exit on rdb load fail by @adiholden in #1750
• feat: add validation of acl users by @kostasrim in #1743
• fix(zset): correct the wrong calculation of range.maxex by @theyueli in #1759
• chore: move rdb save related logic into auxillary file by @romange in #1769
• fix: Add db id to client list attributes by @romange in #1770
• fix: failing memcached regression test by @kostasrim in #1778
• fix: run container as dfly user by @sorenhansendk in #1775
• chore: remove locks from acl validation in the hot path by @kostasrim in #1765
• feat (facade): simple argument parser by @dranikpg in #1747
• feat: introduce configurable flags for I/O limits by @romange in #1776
• fix: use cd instead of working-directory which does not work by @romange in #1781
• fix(search): return whole json documents by @dranikpg in #1782
• fix(AclFamily): stream acl updates via dispatch queue in connection by @kostasrim in #1786
• feat(AclFamily): add acl deluser by @kostasrim in #1773
• feat(AclFamily): add acl whoami command by @kostasrim in #1774
• fix: account for huge pages when displaying RSS usage by @romange in #1785
• fix(server): Dont apply memory limit when loading/replicating by @adiholden in #1760
• fix(AclFamily): setuser signed interleaved categories updates by @kostasrim in #1790
• chore(AclFamily): disable flaky tests by @kostasrim in #1791
• feat(search): FT.PROFILE by @dranikpg in #1787
• chore: dragonfly compiles on FreeBSD by @romange in #1789
• chore: Update double-conversion library by @romange in #1793
• feat(search): Query parameters by @dranikpg in #1768
• Enable authentication for the HTTP interface. by @royjacobson in #1792
• fix(server): Adjust batching behavior to reduce network latency on MULTI blocks by @royjacobson in #1777

New Contributors

• @mutiadavid made their first contribution in #1689
• @Niennienzz made their first contribution in #1729
• @theyueli made their first contribution in #1759
• @sorenhansendk made their first contribution in #1775

Huge thanks to all the contributors! ❤️

Slava Ukraini 🇺🇦

Full Changelog: v1.8.0...v1.9.0