EchoVault/SugarDB v0.3.0

on GitHub

Changelog

• 2a7d472 Add coverage file to tracked files
• 4bc1f44 Added Apache 2.0 license header to all go files in src folder
• 2d2e872 Added CodeCov coverage report generation
• 1bd7147 Added Configuration and Eviction Sections in README.md
• 0b90c93 Added DEL command for deleting keys from the store
• 169fa39 Added GetLatestSnapshotTime to EchoVault interface as it's used in the admin module for the LASTSAVE command.
• 8000cef Added RWMutex to keysWithExpiry in order to improve thread safety
• f41e2f6 Added RWMutexes in PubSub Subscribe, Unsubscribe, and Channels receiver functions. Implemented test for PUBSUB CHANNELS command handler
• 82be1f6 Added addedTime to EntryLFU object to control which entry will be removed if access count is the same. If two entries have the same access count, the older entry should be removed first. SetKeyExpiry and GetValue keyspace receiver functions now require context object to be passed. Created adjustMemoryUsage function for key eviction for LFU, LRU, and Random eviction policies. Updated all modules to pass context to SetKeyExpirty and GetValue functions.
• d1d613d Added api test files
• e569bf6 Added config flags max-memory and eviction-policy to manage memory usage. Created ParseMemory utility function to parse max-memory value into bytes. Created LFU cache to be used with heap data structure for managing LFU cache.
• 79d5230 Added directory to codecov github action workflow
• 08d178b Added error checking on SetValue in module commands
• 1eebc86 Added godoc commands for keyspace.go
• 501879d Added godoc commends for api_generic.go functions
• 35d71a1 Added godoc commends for api_hash.go functions
• 4055432 Added godoc comment for ReadPubSubMessage function
• dca58e0 Added godoc comments for api_admin.go functions
• 4fbabbd Added godoc comments for api_list.go functions
• 62792a4 Added godoc comments for api_pubsub.go functions
• 155a1b8 Added godoc comments for api_set.go functions
• e42582e Added godoc comments for echovault.go exported functions
• 4001424 Added godoc comments to the api_string.go file
• 9696827 Added godoc documentation for api_acl.go function
• 53115c0 Added godocs for sorted set api
• 04b6144 Added license header to api test files
• 1016a1b Added license header to api_pubsub.go file
• 0b9fbb0 Added mutexes for lfu and lru caches on server struct
• cfe4ae8 Added new code coverage badge from codecov
• 8d4d6f8 Added punctuation to api_string.go godoc comments
• 59a03ae Added test for PUBLISH command handler
• 77300f0 Changed 'value' in list commands' descriptions to 'element' for more clarity.
• ad6b3b8 Changed StartTCP to startTCP to make it a private function
• 5c347cb Changed max-memory to uint64. Added TODO commends on how to handle memory checking for key eviction.
• ea0092e Check for nil rw object in LogStore before attemting a sync
• 2af4db5 Check if connection in nil before closing in acl module tests
• ed683ca Commend formatting
• a1789ad Comment update in config package in pkg folder
• d7c3509 Created 'constants' package for const.go file and 'types' package for shared and exported types
• 88a8e2a Created DefaultConfig to be used when embedding echovaule. Moved ACL and PubSub to internal packages with only the associated commands in the modules folder. Initialise ACL and PubSub when creating new EchoVault instance which removed the need to pass WithACL and WithPubSub options.
• 118e155 Created IsMaxMemoryExceeded utility function that is used to check whether the server has exceeded the max memory. Check whether max memory is exceeded before setting value or creating new key. Return error from SetValue function when max memory is exceeded. This change has been propagated to the aof and snapshot engines as well. Snapshot engine now only accepts SetValue option. No longer does it neet to use CreateKeyAndLock & KeyUnlock functions itself.
• 74c08a5 Created KeyData type to hold key data value and expireAt in the same map object. Created scaffolding for PERSIST, EXPIRETIME, PEXPIRETIME, TTL, PTTL, EXPIRE, PEXPIRE, EXPIREAT and PEXPIREAT commands
• 0a70104 Created api_ files in echovault subfolder that will allow us to export easyily usable APIs for embedding Go code
• b49a0e2 Created command test file in pubsub module
• 7c39e5f Created commands and config packages in pkg folder for easy import
• f4d4271 Created compareSlices and compareUsers functions as helpers for comparing slices and users in tests. Implemented tests for creating enabled and disabled users with ACL SETUSER command
• c611dd6 Created flow for deleting key consistently within cluster.
• 0e5f8ff Created init function for acl module test suite
• 1d15b89 Created outward-facing API for hash module commands
• ede32b4 Created outward-facing API or generic module commands
• 28dcfd2 Created test file for acl module commands
• cb13e82 Created test for LLEN API function
• a644742 Created testing skeleton for ACL module
• bc42dce Disabled all auth module tests
• 3dbd356 EchoVault context is now private
• 57cb8b3 Fixed WithContext option function
• 9113998 Fixed test error with regards to passing context to mockServer Start method
• faef3b0 Generate coverage report in go.yml workflow
• e34ed74 Implemented DeleteKey function to delete keys from data store
• 97a10e4 Implemented EXPIRTE and PEXPIRE command handler
• b7b8ecf Implemented PERSIST, EXPIRETIME, PEXPIRETIME, TTL, and PTTL command handlers. Added keyspace function GetExpiry to get expiry time of a given key.
• d51dfd5 Implemented RWMutex for users list when handling user mutations and auth. Fixed bug where user delete command would delete all the users except the specified one. Implemented test for ACL DELUSER command
• 8cc8f6c Implemented default KeyLock and KeyRLock timeout of 250 milliseconds when timeout is not provided by the caller.
• f27a0dd Implemented eviction algorithm that samples a configurable number of keys with TTLs (default 20) at a configurable interval (default 100ms) and deletes the keys that are expired.
• 10f1aea Implemented handler for EXPIREAT and PEXPIREAT commands
• 0cd8a4a Implemented locking mechanism for ACL LOAD and ACL SAVE commands
• b1ddcd1 Implemented outward-facing API for ADMIN module commands. Added Module field to Command and SubCommand structs. Added MODULE filter options to COMMAND LIST command. Added Module field to all existing commands.
• 6fb6862 Implemented outward-facing API for LIST module commands
• c56e749 Implemented outward-facing API for Pub/Sub module
• 7656828 Implemented outward-facing API for SET module commands
• 6b9db4e Implemented outward-facing API for STRING module commands
• 682261b Implemented outward-facing API for commands in the ACL module. Excluded the AUTH command as this is only relevant for TCP clients
• d7fd761 Implemented outward-facing API for sorted set module commands
• a19bfa8 Implemented rlocking/runlocking of rwmutex in Channel IsActive and NumSubs receiver functions. Implemented PUBSUB NUMPAT handler unit test
• 1cca650 Implemented test for ACL GETUSER command
• 78225cd Implemented test for ACL LIST command handler
• a48b98a Implemented test for ACL WHOAMI command handler
• 9191d16 Implemented test for AUTH command handler
• 8fc68a9 Implemented test for new SET command format
• 5bb2bd1 Implemented tests for ACL SETUSER command handler
• dbfa398 Implemented tests for SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE, and PUNSUBSCRIBE command handlers
• f68a369 Implemented tests for all commands handlers in the generic module
• 8af0937 Implemented tests for list module API
• f061af6 Implemented tests for sorted set API
• d08231f Implemented unit test for PUBSUB NUMSUB command handler
• 42a8ad2 Implemented unit tests for HASH API
• 0b66936 Implemented unit tests for SET module API
• ede342a Implemented unit tests for generic module API
• dea2ef2 Implemented unit tests for string module API
• 1aa6f87 In generic module test suite, add minites, seconds and milliseconds to the mock time
• 018aea7 Initialise default IncludedCategories to allCategories("*"). Implemented test for ACL USERS command.
• c414da1 KeyExists keyspace function now removes the key if the key expiry is in the past. KeyData type moved to utils package to allow sharing between multiple packages. Updated all commands and command tests to pass context object to KeyExists, KeyLock, keyUnlock, KeyRLock, and KeyRUnlock. Create context object for each test in all test suites instead of just passing context.Background() to all functions that accept a context.
• 1e50cc1 Made ACL and PubSub fields in EchoVault struct private
• 8017589 Merge branch 'main' into chore/pakage-docs
• 88a3d8b Merge main
• 348bae3 Merge pull request #11 from EchoVault/feature/eviction-policy
• a94ad78 Merge pull request #12 from EchoVault/chore/module-tests
• b4188a2 Merge pull request #13 from EchoVault/chore/module-tests
• 38450f1 Merge pull request #14 from EchoVault/chore/license-update
• 636b335 Merge pull request #15 from EchoVault/feature/embeddable-echovault
• e3a97c0 Merge pull request #18 from EchoVault/chore/pakage-docs
• 93bf70f Merge pull request #19 from EchoVault/chore/pakage-docs
• 97b41da Moved aof and snapshot packages from server folder to src folder. Created new package for evictions
• 7cec2da Moved aof and snapshot packages to internal folder
• 5c86fb6 Moved main.go file to cmd subfolder. Renamed src folder to pkg folder as it will contain all the importable package code. Moved config.go to new internals folder
• e4f4c89 Moved memberlist and raft packages to internal folder
• bc188cf Moved some types into internal folder as they will do not need to be exported. Changed GetState method to getState to make it private to the echovault package
• 7fac414 Moved utils.go file to internals folder
• 78b2e33 Pass config to IsMaxMemoryExceeded
• 52b39d5 Re-enabled cluster nodes in docker-compose.yml. Created flow to forward key deletion command from non-leader node to leader node. Created flow for propagating key deletion accross the entire cluster to maintain consistency of the deleted keys.
• 2ea30d2 Re-enabled the acl module tests
• 6db02f9 Re-implemented MSET, GET and MGET command handler tests
• d38f937 Refactor in NewEchoVault function
• c72e982 Refactored EchoVault struct fields to make them private
• a8ad7e3 Removed Commands filter helpers as there's no planned usage for them
• 90ed2d4 Removed KEY and CERT environment variables for all cluster nodes in docker compose file. broadcastRaftAddress function in memberlist package no longer requires a context to be passed to it. Moved FilterExpiredKeys function to utils package as it's shared by aof engine, snapshot engine, and raft snapshot store.
• 54d0ae7 Removed Opts from echovault. NewEchoVault now entirely uses the option pattern
• e70d101 Removed Plugin type from types.go
• 328165d Removed Score and Member types in api_sorted_set.go in favour of simpler built-in types
• 3b0493e Removed StartSnapshot, FinishSnapshot, SetLatestSnapshot, and GetLatestSnapshot funcs from EchoVault interface as these are now private functions passed to the raft module using inversion of control
• 7fff89a Removed context argumebt from server start method in cmd/main.go
• 852f23b Removed context parameter and implemented locking of RWMutex in NumPat and NumSub receiver functions in PubSub module
• d9f1b22 Removed context parameter from EchoVault, raft, and memberlist shutdown functions
• a9e183c Removed context parameter from Start, startTCP and handleConnection functions. These functions now directly pull the context from the EchoVault struct instance
• 28f6590 Removed discord label from README.md. Updated SUBSCRIBE command handler to send psubscribe event to connection when subscribing to a pattern.
• 28f9765 Removed etc and get modules and replaced them with generic module. Implemented functions to set and remove the expiry of a key. Implemented LRU and LFU caches using heap.
• bc66150 Removed unused context in GetAllCommands server receiver function
• 4362519 Renamed 's' server variable to 'server' in main function
• ba2ebaa Renamed 'server' package to 'echovault'
• 7bbae29 Renamed ReadMessage in api_pubsub.go to ReadPubSubMessage
• a3fbbd1 Renamed eviction2 import alias to eviction
• fa42da5 Renamed server.go to echovault.go
• 69df623 Renames Server object to EchoVault in raft package
• 93cfb9c Renames Server struct and utils.Server interface to EchoVault and utils.EchoVault respectively
• c7c38cf Return array when PUBSUB CHANNELS command is called with a pattern
• 4228952 Set sync to false for all keyspace read commands in the generic module
• e40e49e Skip AOF command logging if ReadWriter is nil. Pass config to api test files to prevent aof data directory from being created everytime tests are executed.
• 389c871 Start raft and memberlist layers in NewEchoVault. Return error fron NewEchoVault function when TLS config does not match certificate options provided
• 52646d1 Take expiry into account when creating snashot and AOF preamble. If the key is already expired when snapshot is taken, it will not be persisted. If the key is expired when loading a snapshot/preamble, it will not be restored.
• f23bbd4 Track src/aof folder but ignore aof subfolders in modules folder
• 51ac577 Update README.md
• 12b55e1 Update coverage report
• 19b6b42 Updated .goreleaser.yml file
• 1751196 Updated LICENCE.md to APACHE 2.0 License
• 6b58598 Updated README.md
• b337dcd Updated README.md badge to Apache 2.0 License
• d10eb19 Updated README.md to include imported library example
• 4010306 Updated build and test command in go.yml workflow file
• f86ea5d Updated coverage report
• f4b17f4 Updated coverage.out
• aea3e06 Updated snapshot engine to user options pattern. Updated SetValue and SetExpiry functions in snapshot and aof engine to SetKeyData function that handles setting both value and expiry of a key upon restore.
• fbc866f Updated test coverage
• 3c81ceb Updated test coverage
• 9d99aea Updated test coverage
• cf22a48 Updated test coverage report
• e685d50 Updated test suites to set mock server only once in each suite instead of instantiating it in every test.
• 8a8a357 ZSCORE API now returns nil if there are no scores to return
• 0bfb9e4 ignore context paramenter in Subscribe receiver function in PubSub struct
• 37a6958 initialiseCache is not a private function
• 893d577 isInCluster is not a private function