Everything you need to know about Hazelcast 3.6

I am excited to announce the general availability of Hazelcast 3.6. This release is the outcome of months of work by both the Hazelcast team and the Hazelcast community. The Hazelcast community has given precious support by sending pull requests as well as opening issues and feature requests. In this blog post, I’ll give an overview of the main new features introduced in 3.6.

Open Sourcing Native Clients

The biggest news around 3.6 is the decision to make the C++ and .NET clients open source. We recently released our Open Binary Client Client Protocol in order to allow community members to build their own clients. Java has always been open source, and changing C++ and .NET clients to open source allows them to be used as building blocks for new clients and to act as further examples. We are looking forward to community contributors developing clients for different languages. There is plenty of interest in NodeJS, Python, Ruby, Go and so on. Specifically, you can expect to see releases of Python and NodeJS clients in the near future. There is ongoing development of all clients with support of the community. The new Open Binary Client Protocol that was rolled out with Hazelcast 3.5 and 3.6 versions will help users to develop their own clients. We would love to see Hazelcast become popular and widely used in many other platforms in addition to the Java community.

Hazelcast 3.6 Enterprise, New Features

Hot Restart Store: 3.6 is an important feature for Hazelcast, because it is a major step forward into distributed persistence: Hot Restart Store persistence. With Hot Restart Store, you will be able to restart your cluster rapidly from disk, as each node copies its state to disk. It is SSD-optimized delivering incredible restart performance. For example, a Hazelcast cluster will restart in a maximum of four minutes with 500GB data per node. There are two strong use cases of Hot Restart Store: one is that it provides a safety back-up in the case that something crashes all of the nodes in your cluster (datacenter failures), and the second is that you can update your software without having to reload data. Hot Restart Store will initially support our two most widely used data structures: IMap and JCache.

High-Density Memory Store for IMap: The second most exciting new item is High-Density Memory Store support for IMap. You will recall High-Density Memory Store was introduced in version 3.4 supporting just JCache. With Hazelcast 3.6, High-Density Memory Store will be available for IMap – one of Hazelcast’s most popular data structure. High-Density Memory Store enables each node to hold large amounts of data without incurring garbage collection overhead.

WAN Replication Improvements: We continue to enhance our WAN Replication in Hazelcast Enterprise with advanced features. In 3.6, we made the following improvements:

• Starting with 3.6, we replicate the queue that keeps WAN replication events. This provides high availability for WAN replication queues.
• We introduced event filtering, so you will be able to filter events that will be replicated to the target cluster.
• In 3.6, we also allow the user to choose their consistency requirements on the receiving side. Before 3.6, the caller returns on completion of receiver operation. This introduces latencies into the communication. With 3.6, there will be an option where the caller sends the packet and returns without waiting for the operation to be executed.
• And we’ve added a dedicated management console in Management Center for managing replication across clusters.

Hazelcast 3.6 Open Source, New Features

Hot Restart and High-Density Memory Store are features available in Hazelcast Enterprise HD. Besides Hazelcast Enterprise HD, we are striving to make Hazelcast Open Source more powerful and faster. We are improving performance of our current modules but also adding new features requested by community users.

Lite Members: If you ever used the Hazelcast 2.x versions, you will remember Hazelcast had a instance type –  “Lite member”. Lite members are active members of the cluster in all ways with the exception that they do not hold data. We had removed lite members because we introduced smart clients that had access to all members, which removed the performance gap. But recently, we saw community requests for lite members; because they enable processing using Executor Service. This allows you to have pure compute nodes and keep this computation separated from data holding nodes.

Cloud Discovery Service Provider Interface (SPI): As we see more users deploy Hazelcast into clouds of all types, we wanted to make life easier for cloud users. Auto discovery is a critical feature for cloud that enables nodes to discover each-other. We have had auto discovery for AWS since the early versions of Hazelcast. In 3.6, our community (working on a HEP) added a Cloud Discovery SPI which will allow the community to add new cloud discovery providers. We redid our AWS discovery mechanism as a provider to the SPI and also added JClouds. You can see the current list of providers supported by Apache jclouds, and we encourage the community to build any new providers they need.

Query and Index Improvements: Query is one of the popular features of Hazelcast on accessing and filtering data. We know we had so many things to improve queries. In 3.6, we worked to reduce memory consumption of indexes which was being complained out by community. Also we were not happy with the performance of queries and we developed a Rule Based Query Optimizer to determine the most efficient way to execute the given query. Although there are still things to improve in this area, our tests showed a notable performance boost. What’s more, we added query support to multi-value attributes. It will be possible to add indexes to multi-value (collection) attributes to query them efficiently. This is important particularly for the financial services industry.

Docker Integration: Starting in 3.6, we are releasing Hazelcast docker images in Docker Hub. We have three images: Hazelcast, Hazelcast Enterprise and Management Center. We are planning more developments to make Hazelcast more Docker friendly. With the help of Discovery SPI, we expect more Docker integrations to be developed by the community.

Enhancing JCache: There are some features that already exist in Map and our JCache users wanted to see them in our JCache implementation too. These are:

• Handling split brain (network partitioning) cases for JCache.
• Listening partition events, being alerted when a partition is lost (partition lost listener)
• Specifying minimum number of nodes in cluster to let operations go or suspend. (Quorum for JCache)

Ringbuffer For .NET Client: We had introduced Ringbuffer and Reliable Topic in 3.5. In 3.6 we also developed Ringbuffer also for the .NET Client. Ringbuffer is a high performance data structure popularised by the LMAX Exchange Disruptor framework. **Replicated Map Improvements: **Thanks to you (Hazelcast community) we detected problems regarding our replicated map implementation. In 3.6, we worked on a design that makes replicated map proof against any missing updates due to intermittent connection problems.

Try Lock with Lease: This is an extension to our current Map API. We are adding the method with following signature: boolean tryLock(K key, long time, TimeUnit timeunit, long leaseTime, TimeUnit leaseTimeunit)

Expiry Event For Map: Starting in 3.6, expiry events will be differentiated from eviction events. Expiry is the event only fired in case of expiration due to ttl (time-to-live seconds) where entries can be evicted because of size constraints (eviction event).

Paging Predicate Seek: In 3.6, the constant time access to a specific page is provided. We introduce the setPage(int PageNumber) method that goes to a page in O(1) time. **Better OSGI Bundling: **We made improvements to make Hazelcast more OSGI friendly.

Better Performance

Besides developing new features, we have dedicated a team to focus on the performance of Hazelcast. In 3.6, performance was especially on our radar. We have made improvements and refactorings on our threading model, clients, queries and transactions. Happily we see that 3.6 is performing much, much faster than the 3.5 release in all these areas. Besides our own tests, we tested Hazelcast with different performance suites like Yardstick and Radargun. Hazelcast is fastest on both of these suites. In a future blog post, we will share our performance benchmarks and explain what we did in Hazelcast 3.6 to drive further performance increases.

And Bugs…

Other than features and performance improvements, we fixed many bugs (more than 130). You can see the list of fixed bugs for 3.6 here: https://github.com/hazelcast/hazelcast/issues?q=is%3Aissue+milestone%3A3.6+is%3Aclosed Note that, till the final release we will continue to test the product and fix issues.

Downloading Hazelcast

You can download 3.6 from our downloads page

Also from maven:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast</artifactId>
    <version>3.6</version>
</dependency>

Final Words

The Hazelcast community guides us on our way. We want to thank everyone who contributed to Hazelcast by either sending pull requests or creating issues in GitHub. We have big plans about Hazelcast but certainly we need your support. Please review and test 3.6 release and create issue on GitHub when you encounter any problem. We are open to all sort of criticism and comments. You can use our Google group, Twitter @hazelcast or StackOverflow for your questions. And of course, GitHub for your bug reports and contributions. Thank you all for your support!

• Try Hazelcast Enterprise HD »