Hazelcast IMDG 3.10 BETA is released

Matko Medenjak | Feb 22, 2018

Good news! Hazelcast IMDG 3.10 BETA (previously known as EA) is now available and it packs an exciting range of features and improvements. So here we go…

Flake ID Generator

Many systems require the generation of unique IDs. Although unique ID generation is a relatively easy problem in single-node systems, many problems pop-up in distributed implementations, such as preserving uniqueness, eliminating contention, maintaining safety during failures, etc…

In Hazelcast 3.10, we introduce a new mechanism for generating IDs – the FlakeIdGenerator. This implementation supersedes the existing IdGenerator. It is capable of generating IDs without coordination between Hazelcast members which means that it can generate unique IDs even during split-brain and network partitioning failures.

Another consequence of no coordination is that the generator is highly available and can provide a higher throughput than the existing IdGenerator.

CRDT PN Counter

In Hazelcast 3.10, we have introduced a first CRDT implementation – the PN counter. As you might know, the property of a CRDT is that it is eventually consistent and updates are local. In our implementation, updates are local on a Hazelcast member. The PN counter is a simple CRDT which allows counting (increment/decrement/query) operations and guarantees that, once all members are connected, the value on all counters will be the same. The counter also enables low latency and high throughput – since operations are local (on a Hazelcast member), your application achieves high performance during high-volume traffic, such as counting likes, page views or connected user count.

Now that we have our first CRDT, additional CRDTs will be easier to implement so we are eager to hear what you think and which CRDTs you would like to see.

Merge Policies for Additional Data Structures

In Hazelcast 3.9 only Map, ICache and ReplicatedMap supported merging after a network partition had occurred. We have pushed this further and provided split-brain merging policies for other structures: ISet, IList, IQueue, Ringbuffer, MultiMap, IAtomicLong, IAtomicReference, IScheduledExecutorService and CardinalityEstimator. In addition to providing out-of-the box policies and split-brain merging functionality for these structures, we have worked on improving the split-brain merge configuration and SPI which means configuring and implementing split-brain merging for your services is now easier.

Extend Hazelcast Split-Brain Protection

Our existing split-brain protection functionality has worked well to minimize the window in which operations can continue to be processed on different parts of the cluster after a split-brain has occurred. Unfortunately, this protection was only available on IMap, ICache, IQueue and ILock. We have now expanded this to other structures: ISet, IList, ISemaphore, ICountDownLatch, IReplicatedMap, IAtomicLong, IExecutorService, IDurableExecutorService, IScheduledExecutorServicem, IMultiMap, IRingBuffer, CardinalityEstimator (HyperLogLog), IAtomicReference and CRDT PN-Counter. In conjunction with the “Shorten Split-Brain Detection Window” (see below), this brings value to protecting your structures from concurrent updates during split-brain. This limits the amount of time inconsistent reads or writes can be made.

Shorten Split-Brain Detection Window

Regardless of how long it takes to detect a down member, it may be that for a given data structure, we want to be more conservative than that, depending on the consequences of an inconsistent write or a stale read. In addition to expanding our split-brain protection to all structures, we have made split-brain protection more flexible. Now you can apply different levels of strictness of split-brain protection for some structures. Previously split-brain protection only acted on the information on the number of members currently observed. To improve this and provide stricter protection, split-brain protection can now act on additional information such as member heartbeats (ProbabilisticQuorumFunction or RecentlyActiveQuorumFunction), membership changes and lost ICMP pings as well as a successful ping after a lost ping.

JCache 1.1 Changes

We are always keen on keeping up-to-date and providing users with the latest updates. Because of this, we are the first product to be compliant with JCache 1.1. For more information on JCache 1.1 JSR, see: https://www.jcp.org/en/jsr/detail?id=107

Split-brain Merge Improvements for HD Structures (Enterprise only)

We have improved split-brain merging for HD structures and introduced optimizations for merging huge amounts of data which will reduce the time for your HD structures to be merged after the cluster has healed.

Disclaimer and Closing Words

In addition to the above-mentioned functionalities, countless hours have gone into small improvements and optimizations. Please download Hazelcast IMDG 3.10-BETA-1 and try it out! We are eager to hear what you think. We do not recommend running Beta versions in production, but we encourage everyone to grab it and play with the new features. Lets us know what you think. We love to hear your feedback, our developers tend to hang out at Gitter so feel free to stop by and contribute!