hazelcast/hazelcast v4.1 on GitHub

This document lists the new features, enhancements, fixed issues and, removed or deprecated features for **Hazelcast IMDG v4.1 ** release. The numbers in the square brackets refer to the issues in Hazelcast's GitHub repositories.

New Features

Hazelcast IMDG Open Source New Features:

SQL Implementation: Implemented the SQL engine that supports map scans, index scans, projections and filters. It also introduces the hazelcast-sql module to handle the SQL engine's dependencies on Apache Calcite. See the SQL section.
Automatic Discovery Strategy Detection: Hazelcast now automatically detects the cloud environment it's running on (Kubernetes, AWS, GCP or Azure) and applies the necessary discovery strategy for the environment. This allows automatically forming the Hazelcast clusters in cloud environments without any configuration. See the Auto Detection section.
Overriding Configuration: You can now override the configuration entries of your cluster without changing the declarative configuration files (XML/YAML). See the Overriding Configuration section.
Instance Tracking: Hazelcast now writes a file on a Hazelcast member startup at the configured location. This file contains metadata about the member such as version, product name and process ID, and it is not deleted on the member shutdown. See the Instance Tracking section.

Hazelcast IMDG Enterprise New Features:

Security Audit Logging: This feature allows observing some important cluster events. Auditable events have a unique type ID, and contain a timestamp and importance level. See the Logging Auditable Events section.

Breaking Changes

Starting a standalone Hazelcast instance requires disabling Auto Detection joiner (before it required disabling Multicast joiner). #17112
In a CP subsystem session, a generic IllegalStateException was being thrown when Hazelcast is shutdown. Now the same situation throws HazelcastInstanceNotActiveException. #17120
Implemented and/or overridden the default methods in Java 8 collections, such as computeIfAbsent(), forEach() compute() and replaceAll() for maps. #14913

Enhancements

Hazelcast IMDG Open Source Enhancements:

Improvements in Partial Network Disconnections: Introduced properties to be configured to detect and resolve the partial network issues among the Hazelcast IMDG members. See #16680 and the Partial Network Partitions section.
Parallel Independent Migrations: Parallelized the partition replica migrations so that the time needed for rebalancing the partitions after adding or removing a member to/from a cluster is reduced. See the hazelcast.partition.max.parallel.migrations property explanation in the System Properties appendix.
CP Subsystem Listeners and Metrics: Added membership and group availability listeners for CP Subsystem. Also, added support for publishing CP Subsystem and CP data structure statistics via Metrics. See the CP Subsystem Listeners section.
GenericRecord: Added support for accessing domain objects without domain classes. See the GenericRecord section.
Thread Affinity: Introduced the CPU thread affinity; threads can have affinity for particular CPUs and using this you have a better control on the latency and a better throughput. See the CPU Thread Affinity section.
Trusted Interfaces: It is now possible to restrict the source IP addresses from which the Management Center operations are allowed. See the Limiting Source Addresses section.
Priority Queue: Added support for the queues with comparators. Using Priority Queue, you can define items with higher priority to be polled first. See the Priority Queue section.
Statistics for Durable and Scheduled Executor Services: Added the statistics for scheduled and durable executor services, and for the executor of offloaded entry processors to be monitored by Management Center.

Hazelcast IMDG Enterprise Enhancements:

Out-of-the-Box Kerberos Support: Introduced support for Kerberos authentication protocol which is one of the standard solutions for single-sign-on. It also adds the GSSAPI authentication support in LDAP configuration. See the Kerberos Authentication Type section.
Multiple Persistent Directories: Added support for multiple mounting directories in the persistent memory configuration and also thread affinity for NVMs (Non-Volatile Main Memories). See the Using Persistent Memory section.
WAN Throttling Mechanism: This prevents the WAN consumers from getting overloaded by the WAN producers if transferring the WAN events takes less time than processing them in the target cluster. This can be the case with WAN synchronization if the network latency is low enough. It can be configured using the following properties:
- hazelcast.wan.consumer.invocation.threshold
- hazelcast.wan.consumer.ack.delay.backoff.init
- hazelcast.wan.consumer.ack.delay.backoff.max
- hazelcast.wan.consumer.ack.delay.backoff.multiplier
  #17113
Implemented a new concurrent High-Density Memory Store index for the queries, based on B+ tree that is enabled by default; brings better performance for lookup queries and enables SQL queries.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

Added the missing replicatedmap-permission support to the XML and YAML configuration handlers. #17812
Replaced the word "should" with "must" in the exception messages to improve the user focus and indicate obligation rather than advice. #17710
Added the getAndDecrement() operation support for the IAtomicLong data structure. #17699
Introduced the mode configuration attribute for the persistent memory. It defines operational mode of it which can be either MOUNTED or SYSTEM_MEMORY. #17696
Added the support for checking duplicate fields when building class definitions. #17682
Added the support for IList with nullable items to the client protocol. #17690
Improved the robustness of the diagnostics log writer and its plugins. #17665
Added the client name tag to the client related metrics. #17654
Added a check for IMap READ permissions for the maps involved in the SQL query executions. #17612
Added the support for accessing the Hazelcast instance in the login modules and credential factories. #17605
Introduced a simplified configuration of Kerberos authentication for simple scenarios. #17573
Added statistics support for IBM, OpenJ9, ZGC and Shenandoah. #17561
Introduced a state-tracking mechanism for the external configuration parser; all mismatched/unapplied configuration entries coming from environment variables or system properties are now logged. #17559
Updated the jackson.version to 2.11.2 and SnakeYAML to 2.1. #17484, #17446
Added support for the merge operation in IMap. #17419
Added public classes to expose the member- and client-side caching provider implementations (in the JCache implementation) without referring to internal classes. #17320
Improved the entry processing mechanism so that the read-only processors on backups are not executed anymore. #17318
Improved CacheSimpleConfig so that it now accepts cache name in its constructors. #17287
Cleaned up the noisy initial log messages on Hazelcast startups. #17243
Introduced auto-disposable tasks for the scheduled executor service. #17215
Added client labels to ClientEndpointImpl.toString() to simplify the client identification (in addition to client uuid) on the Hazelcast member side. #17178
Added exception handling when creating caches with tenant control. #17122
Added support for IMap.compute(). #17030
Added support to allow submitting and executing operations while a member is shutting down to increase the availability of a cluster more that keeps large data. #17028
Added ExtendedMapEntry in order to create a mechanism for setting TTLs in entry processors. #16991
Added hazelcast-azure discovery plugin to the hazelcast-all module. #16982
Introduced the ConfigRecognition API that determines if a provided declarative configuration is recognized by the rules defined in a given implementation. Along with the API the following three implementations are added:
- MemberConfigRecognizer for recognizing member XML and YAML configurations
- ClientConfigRecognizer for recognizing client XML and YAML configurations
- ClientFailoverConfigRecognizer for recognizing failover client XML and YAML configurations
  #16958
Added the publishAll(), publishAllAsync() and publishAsync() methods to ITopic. #16946
Made the diagnostics logs cloud-friendly so that they can also be sent to stdout in addition to sending to local files. #16941
Improved the mechanism of partition table updates to eliminate the latencies when these updates are sent to the clients by a member. #16939
Improved the client configuration such that when the client failover configuration is provided, the reconnect mode cannot be set as off; it now fails fast in this case. #16886
Introduced the forEach() loop support for IMap. #16877
Added the load() method to Config, ClientConfig and ClientFailoverConfig classes. This method loads the configuration with the known locations. If not found, the default configuration is returned. #16864
Improved the deleteAll() (MapStore) and ITopic Javadocs. #16862, #16861,
Added support for IMap.computeIfAbsent(). #16808
Added the setAll() and setAllAsync() methods for IMap. #16787
Added the creation time metric for the executor service. #16775
Improved an unclear exception message for credentials. #16756
Updated the related aspects of Hazelcast IMDG after the changes done on the client protocol side to add the ability to add new parameters, methods, services, events and custom types to codecs. #16718
Introduced the putAllAsync() method for MultiMap. #16698
Defined metrics for ISet and IList collections. #16665
Upgraded log4j2 version to 2.13.0. #16654
Improved the computeIfPresent() implementation so that now it keeps a clone of the old/existing value and uses that for replace/delete operations. #16636
Introduced the hazelcast.logging.details.enabled property to make the logging of cluster version, name and IP optional to decrease the noise in the logs when, for example, you have a single-member cluster. #16622
Upgraded the Hazelcast Kubernetes plugin version to 2.0.1. #16590
Added the support for automatically detecting the classloader of a type for the user code deployment feature. #16585
Made IMap.putAllAsync() and IMap.submitToKeys() methods public API. #16449
Clarified the exception message when you connect to a cluster with an invalid cluster name. #15099
Refactored the check and retry initialization logic of ReplicatedMapProxy so that they are performed in parallel for different partitions. #14331
Improved the behavior of ConcurrentMap.computeIfPresent: combined single client-server round trips instead of two (for get and replace methods). #11958

Fixes

Fixed an exception in the /node-state REST calls when the member is not fully activated. #17798
Fixed an issue where Hazelcast was not releasing the acquired lock sessions that fail for the reasons other than session expiration and wait key cancellation. #17697
Fixed an issue where Hazelcast was returning false if a client is successfully deregistered from any member, but events are still delivered for the non-deregistered ones. #17646
Fixed a data loss issue that was happening while promoting a lite member to a data member. #17621
Fixed an issue where the configuration was not updating entries' time-to-live values if the entry processor implements the Offloadable interface. #17606
Fixed an issue where the caller stacktrace was missing on the rethrown async runtime exceptions. #17546
Fixed the rendering of diagnostics in case there is an exception inside a diagnostics plugin. #17501
Fixed an exception that is thrown when using the entry store API and issuing a put into the IMap for an item which is not in the map but exists in the backing store. #17441
Fixed an issue where the custom load balancers could not be configured declaratively. #17415
Fixed a race issue when creating a cache (JCache) using multiple methods with the same cache name but having different keys. #17286
Fixed an issue where the repeated calls of executeOnKeys() in Hazelcast clients for NATIVE maps was causing a continuous increase in the used memory and exhaustion of the memory pool after a while. #17276
Fixed an issue where ReliableTopicMessageListener was firing a warning when the client is shutting down. #17153
Fixed an issue where the client was stuck with an outdated member list after a split-brain scenario. #17147
Fixed the member side user code deployment; it was throwing an exception when inner classes are used to be loaded. #17044
Fixed the broken interoperability between the CompletableFuture methods. #17020
Fixed an issue where touching a map entry having an entry processor working on it was modifying its time-to-live. #16987
Fixed an issue in the cache service where its pre-join operation was considering CacheConfig as resolved: it was assuming that key/value types, user customizations and other cache configurations have been loaded. This was an issue when the cache is not touched yet. #16917
Fixed an issue where Management Center was not working as expected when the cluster is set up using advanced network configuration. #16910
Fixed an issue where ServiceLoader was round-tripping between URL and URI, and consequently loses the associated URLStreamHandler when trying to load Hazelcast from a custom class loader. #16846
Fixed an issue where the class definitions, that are registered explicitly in the serialization configuration and have the same class ID in different factories, were not handled properly. #16831
Fixed the NullPointerException in IndexCopyBehavior.NEVER mode. #16784
Fixed an issue where the client XML configuration could not properly handle the empty Near Cache name. #16768
Fixed an issue where the client permissions for Reliable Topic and Ringbuffer were missing. #16755
Fixed an issue where the type information was missing the Metrics MBeans. #16747
Fixed an issue where the REST API was always requiring the call URLs to end with a slash character. #16688
Fixed an issue where the service URL for Eureka could not be set using the declarative configuration. #16679
Fixed an issue where the wait key of a blocking call within a Raft invocation was still being reported as a live operation, when the key times out. #16614
Fixed an issue where the upload of classes using the client user code deployment were not successful when they are retrieved not in their created order. #16612
Fixed an issue where the size() method was returning a negative value when map, cache and multimap contain more than Integer.MAX_VALUE entries. #16594
Fixed an invalidation issue when using a transactional map from a cache with a Near Cache: the cache invalidation event occurs when the transactionalMap.put method is called. As a result, the entry was getting invalidated before the change is committed to the map. #16579
Fixed an issue where InPredicate was not invoking value comparison when the read attribute is null. #15100
Fixed an issue where Map, Cache, MultiMap data structures were returning negative values (size()) when the size is more than Integer.MAX_VALUE. #14935
Fixed an issue when a Hazelcast client calls the distributed executor service and the callable throws an exception with a custom type, then the exception was not being transported to the client. #9753

Contributors

We would like to thank the contributors from our open source community who worked on this release: