Hazelcast IMDG 3.10 is Released
At long last, we are pleased to announce the production-ready release of Hazelcast IMDG 3.10. A lot of effort has been put into making this release the most feature complete and the most stable release so far. Many of these features were driven by the needs of the community. So let me start by introducing some features worth mentioning:
Flake ID Generator
Many systems require the generation of unique IDs. Although unique ID generation is a relatively easy problem in single-node systems, many problems pop-up in distributed implementations, such as preserving uniqueness, eliminating contention, maintaining safety during failures, etc.
In Hazelcast 3.10, we introduced a new mechanism for generating IDs – the FlakeIdGenerator. This implementation supersedes the existing IdGenerator. It is capable of generating IDs without coordination between Hazelcast members which means that it can generate unique IDs even during split-brain and network partitioning failures. Another consequence of no coordination is that the generator is highly available and can provide a higher throughput than the existing IdGenerator.
CRDT PN Counter
In Hazelcast 3.10, we have introduced our first conflict-free replicated data type (CRDT) implementation – the PN counter. As you might know, the property of a CRDT is that it is eventually consistent and updates are local. In our implementation, updates are local on a Hazelcast member. The PN counter is a simple CRDT which allows counting (increment/decrement/query) operations and guarantees that, once all members are connected, the value on all counters will be the same. The counter also enables low latency and high throughput – since operations are local (on a Hazelcast member), your application achieves high performance during high-volume traffic, such as counting likes, page views or connected user count. You can also configure the replication mechanism if you have a big cluster and would like to restrain the network or data usage by limiting the CRDT replica count and the replication system.
Now that we have our first CRDT, additional CRDTs will be easier to implement so we are eager to hear what you think and which CRDTs you would like to see next.
Split-Brain Merge Policies for Additional Data Structures
In Hazelcast 3.9 only Map, ICache and ReplicatedMap supported merging after a network partition had occurred. We have pushed this further and provided split-brain merging policies for other structures: ISet, IList, IQueue, Ringbuffer, MultiMap, IAtomicLong, IAtomicReference, IScheduledExecutorService and CardinalityEstimator. In addition to providing out-of-the box policies and split-brain merging functionality for these structures, we have worked on improving the split-brain merge configuration and SPI which means configuring and implementing split-brain merging for your services is now easier.
Extended Hazelcast Split-Brain Protection
Our existing split-brain protection functionality has worked well to minimize the window in which operations can continue to be processed on different parts of the cluster after a split-brain has occurred. Unfortunately, this protection was only available on IMap, ICache, IQueue and ILock. We have now expanded this to other structures: ISet, IList, ISemaphore, ICountDownLatch, IReplicatedMap, IAtomicLong, IExecutorService, IDurableExecutorService, IScheduledExecutorServicem, IMultiMap, IRingBuffer, CardinalityEstimator (HyperLogLog), IAtomicReference and CRDT PN-Counter. In conjunction with the “Shorten Split-Brain Detection Window” (see below), this brings value to protecting your structures from concurrent updates during split-brain. This limits the amount of time inconsistent reads or writes can be made.
Shorten Split-Brain Detection Window
Regardless of how long it takes to detect a down member, it may be that for a given data structure, we want to be more conservative than that, depending on the consequences of an inconsistent write or a stale read. In addition to expanding our split-brain protection to all structures, we have made split-brain protection more flexible. Now you can apply different levels of strictness of split-brain protection for some structures. Previously split-brain protection only acted on the information on the number of members currently observed. To improve this and provide stricter protection, split-brain protection can now act on additional information such as member heartbeats (ProbabilisticQuorumFunction or RecentlyActiveQuorumFunction), membership changes and lost ICMP pings as well as a successful ping after a lost ping.
JCache 1.1 Changes
We are always keen on keeping up-to-date and providing users with the latest updates. Because of this, we are the first product to be compliant with JCache 1.1. For more information on JCache 1.1 JSR, see: https://www.jcp.org/en/jsr/detail?id=107
Pluggable Replacer Interface for Protecting Sensitive Information in the XML Configuration
The Hazelcast XML can contain some sensitive information which might be a problem for your application. To work around this issue, we have provided a ConfigReplacer interface which allows you to encrypt property values in the XML configuration. Since it is an SPI interface, you can easily adapt this to your environment and deployment model and make your information more secure. You can have multiple replacers and decrypt your different configuration values in different ways.
Split-brain Merge Improvements for HD Structures (Enterprise only)
We have improved split-brain merging for HD structures and introduced resource usage optimizations for merging huge amounts of data which will reduce the time for your HD structures to be merged after the cluster has healed. Closing Words
In addition to the above-mentioned functionalities, countless hours have gone into small improvements and optimizations so please take some time to download Hazelcast IMDG 3.10 and try it out! We are eager to hear what you think so please stop by our Gitter channel, Google groups. If you would like to introduce some changes or contribute, please take a look at our or Github repository.