Hazelcast 3.6 Early Access is out!

I’m excited to announce the Early Access (EA) release of Hazelcast 3.6. The EA release provides the opportunity for you to review and contribute to the finalization of new features in Hazelcast 3.6. The easiest way to contribute is try Hazelcast 3.6 EA and report any issues you encounter. The Hazelcast development team is now working on finalizing the new features and preparing 3.6 for production environments.

Open Sourcing Native Clients

The biggest news in 3.6 EA is the decision to make the C++ and .NET clients open source. We recently released our Open Binary Client Client Protocol in order to allow community members to build their own clients. Java was always open source. Changing C++ and .NET clients to open source allows them to be used as building blocks for new clients and to act as further examples. We are looking forward to community contributors developing clients for different languages. There is plenty of interest in NodeJS, Python, Ruby, Go and so on. The new Open Binary Client Protocol that we rolled out in the Hazelcast 3.5 and 3.6 versions will help users develop their own clients. We would love to see Hazelcast popular and widely used in many other platforms in addition to the Java community.

Hazelcast 3.6 Enterprise, New Features

Hot Restart Store: 3.6 EA is an important version for Hazelcast because it contains a major step forward into distributed persistence: Hot Restart Store persistence. With hot restart, you will be able to restart your cluster rapidly from disk, as each node copies its state to disk. It is SSD-optimized delivering incredible reboot performance. For example, a Hazelcast cluster will restart in a maximum of 4 minutes with 500GB data per node. There are two strong use cases of Hot Restart Store: one is that it provides a safety back-up in the case that something crashes all of the nodes in your cluster (datacenter failures), and the second one is that you can update your software without having to reload data. Hot restart will initially support our two large data holding structures: IMap and JCache.

High-Density Memory Store for IMap

The second biggest new item is High-Density Memory Store support for IMAP. You will recall High-Density Memory Store was introduced in version 3.4 supporting just JCache. With Hazelcast 3.6, High-Density Memory Store will be available for IMAP. It is probably Hazelcast’s most popular data structure. High-Density Memory Store enables each node to hold large amounts of data without incurring garbage collection overhead.

WAN Replication Improvements

We continue to enhance our WAN Replication in Hazelcast Enterprise with advanced features. In 3.6, we made the following improvements: * Starting with 3.6, we replicate the queue that keeps WAN replication events. This provides high availability for WAN replication queues. * We introduced event filtering, so you will be able to filter events that will be replicated to the target cluster. * In 3.6, we also allow the user to choose their consistency requirements on the receiving side. Before 3.6, the caller returns on completion of receiver operation. This introduces latencies into communication. With 3.6, there will be an option where the caller sends the packet and returns without waiting for the operation to be executed.

Hazelcast 3.6 Open Source, New Features Hot Restart and HD memory are features available in Hazelcast Enterprise. Besides enterprise, we are striving to make Hazelcast open source more powerful and faster. We are improving the performance of our current modules but also adding new features most requested by community users.

Lite Members

If you ever used the Hazelcast 2.x versions, you will remember Hazelcast had an instance type; “Lite member”. Lite members are active members of the cluster in all ways with the exception that they do not hold data. We had removed lite members because we introduced smart clients that had access to all members, which removed the performance gap. But recently, we saw community requests for lite members; because they enable processing using Executor Service. This allows you to have pure compute nodes and keep this computation separated from data holding nodes.

Cloud Discovery SPI

As we see more users deploy Hazelcast into clouds of all types, we wanted to make life easier for cloud users. Auto-discovery is a critical feature for cloud that enables nodes to discover each other. We have had auto-discovery for AWS since the early versions of Hazelcast. In 3.6, our community (working on a HEP) added a Cloud Discovery SPI which will allow the community to add new cloud discovery providers. We redid our AWS discovery mechanism as a provider to the SPI and also added JClouds. You can see the list of providers supported by JClouds here: https://jclouds.apache.org/reference/providers/. We encourage the community to build any new providers they need.

Query and Index Improvements

Query is one of the popular features of Hazelcast on accessing and filtering data. We know we had so many things to improve queries. In 3.6, we worked to reduce memory consumption of indexes which was being complained out by the community. Also we were not happy with the performance of queries and we developed a Rule Based Query Optimizer to determine the most efficient way to execute the given query. Although there are still things to improve in this area, our tests showed a notable performance boost. What’s more, we added query support to multi-value attributes. It will be possible to add indexes to multi-value (collection) attributes to query them efficiently. This is important particularly for the financial services industry.

Docker Integration

Starting in 3.6, we are releasing Hazelcast docker images in Docker Hub. We have three images: Hazelcast, Hazelcast Enterprise and Management Center. We are planning more developments to make Hazelcast more Docker friendly. With the help of Discovery SPI, we expect more Docker integrations to be developed by the community. https://hub.docker.com/u/hazelcast/

Enhancing JCache

There are some features that already exist in Map and our JCache users wanted to see them in our JCache implementation too. These are: * Handling split brain (network partitioning) cases for JCache. * Listening partition events, being alerted when a partition is lost (partition lost listener) * Specifying minimum number of nodes in cluster to let operations go or suspend. (Quorum for JCache)

Ring Buffer For .NET Client

We had introduced Ring Buffer and Reliable Topic in 3.5. In 3.6 we also developed Ring Buffer also for the .Net Client. Ring Buffer is a high performance data structure popularized by the LMAX Exchange Disruptor framework.

Replicated Map Improvements

Thanks to the Hazelcast community we’ve detected problems regarding our replicated map implementation. In 3.6, we worked on a design that makes replicated map proof against any missing updates due to intermittent connection problems.

Try Lock with Lease: This is an extension to our current Map API. We are adding the method with following signature: boolean tryLock(K key, long time, TimeUnit timeunit, long leaseTime, TimeUnit leaseTimeunit)

Expiry Event For Map

Starting in 3.6, expiry events will be differentiated from eviction events. Expiry is the event only fired in case of expiration due to ttl (time-to-live seconds) where entries can be evicted because of size constraints (eviction event). Paging Predicate Seek: In 3.6, the constant time access to a specific page is provided. We introduce the setPage(int PageNumber) method that goes to a page in O(1) time.

Better OSGI Bundling

We made improvements to make Hazelcast more OSGI friendly.

Better Performance

Besides developing new features, we have dedicated a team just to focusing on the performance of Hazelcast. In 3.6, performance was especially on our radar. We have made improvements and refactorings on our threading model, clients, queries and transactions.

Happily we see that 3.6 is performing much, much faster than the 3.5 release in all these areas. Besides our own tests, we tested Hazelcast with different performance suites like Yardstick and Radargun. Hazelcast is fastest on both of these suites. In a future blog post, we will share our performance benchmarks and explain what we did in Hazelcast 3.6 to drive further performance increases.

And Bugs…

Other than features and performance improvements, we fixed many bugs (more than 130). You can see the list of fixed bugs for 3.6 here: https://github.com/hazelcast/hazelcast/issues?q=is%3Aissue+milestone%3A3.6+is%3Aclosed

Note that, till the final release we will continue to test the product and fix issues.

Downloading Hazelcast

You can download 3.6-EA from our downloads page (see development releases):

http://hazelcast.org/download/

Also from maven: Hazelcast 3.6 Maven Repository

Final Words

The Hazelcast community guides us on our way. We want to thank everyone who contributed to Hazelcast by either sending pull requests or creating issues in github. We have big plans about hazelcast but certainly we need your support. Please review and test this early access release. We are open to all sort of criticism and comments.

You can use our forum , Twitter (@hazelcast) or Stackoverflow for your questions; and Github for your bug reports and contributions.

Thank you all for your support.