Hazelcast Platform 5.0 GA Release – the Speed of Insight

About two months ago, we announced the first BETA release of the Hazelcast Platform 5.0. Since then we’ve spent much effort in making the product adhere to the stability and production-ready standards which are expected from any Hazelcast release.

For those who haven’t yet heard of Hazelcast Platform, we will first explain what the Hazelcast Platform is, what led us up to this point and how it can bring value to you, your application and your company. We’ll summarise some features of the Platform, as well as talk about the future of Hazelcast IMDG and Jet.

What is Hazelcast Platform 5.0?

Many users might be familiar with our existing products, Hazelcast IMDG and Hazelcast Jet. Both of these products were battle-tested and established, but something was missing. The combination of these two products can bring much more value than the sum of the values of each individual product. Hazelcast IMDG provided fast ways how to store, retrieve and modify data and Hazelcast Jet provided fast processing of data. But combining these two together brings something more – fast insight and action.

With Hazelcast Platform, we merged the Hazelcast IMDG and Hazelcast Jet products into a single product which not only simplifies the development of applications using Hazelcast, but also puts more focus on developing real-time applications that will engage with your end-users in new ways.

Chances are, your environment and system are full of data that is being stored, but not used to provide value at all or requires some long-running (e.g. nightly) process to provide information on which users, applications and systems can act:

• Clickstreams, contact center call records, customer data, transactions, orders, insurance claims, credit card applications
• Traffic, weather and vehicle telemetry streams
• IoT data streams from control systems and sensors
• Databases, application logs

Sometimes, even, the data is stored in different systems written by different people with different behaviour. With Hazelcast Platform 5.0, you can connect to these systems, unlock and combine the data and start providing value instantly:

• Enable fraud detection
• Provide recommendations
• Enable alerting
• Helping e-commerce, contact center and other CRM applications determine next-best offers and next-best actions for customer service
• Helping transport companies to monitor and manage fleet movements
• Enable predictive, condition-based maintenance and performance optimization of machines, production lines and networks

This is just a taste of what you can come up with. In short, Hazelcast 5.0 can be thought of as a successor of Hazelcast IMDG 4.2 with the addition of the streaming and pipeline capabilities of Hazelcast Jet 4.5.

As a side note, you might come across the term “Hazelcast Platform” on our website and marketing materials. It’s the official name for the new Hazelcast 5.0+ product and is used in places where we need to distinguish between Hazelcast “the product” and Hazelcast “the company”.

Compatibility with IMDG and Jet

We focused on keeping the compatibility with the existing IMDG users. Therefore, Hazelcast Platform 5.0 is fully compatible with Hazelcast IMDG 4.x:

• There are no incompatible API changes. You don’t have to change your application code when upgrading from Hazelcast IMDG 4.x
• The 4.x clients can connect to Hazelcast 5.0
• You can perform a zero-downtime rolling upgrade from 4.x to 5.0

Additionally, we also provide a zero-downtime upgrade path for the Hazelcast IMDG 3.x users. However, in this case, some application code changes are required. To learn more about the upgrade path from Hazelcast IMDG to Hazelcast 5.0, refer to the Migrating Between IMDG and Platform 5.0 documentation section.

For the existing Jet users, the upgrade will require stopping and resuming the jobs just as was the case while upgrading between Jet releases. For details, see Migrating Between Jet and Platform 5.0 documentation section.

New Features

Hazelcast 5.0 is not just about the merging of IMDG and Jet, but also about major feature updates. In this section, we highlight the most notable ones but we urge you to visit the release notes.

Expanded SQL Support and Production Readiness

As stated during our recent, previous IMDG and Jet releases, our investment in SQL support continues. Hazelcast 5.0 takes a big leap forward and brings a lot of new expressions:

• INSERT, UPDATE and DELETE with IMap
• Sorting (ORDER BY) and aggregations (GROUP BY) without any limitations
• Basic JOIN support
• New operators such as IN, BETWEEN or CASE
• New functions, e.g. REPLACE, POSITION, EXTRACT, POWER, SQRT and more.

To see the full list of features, see the SQL reference. Additionally, we are lifting the beta status of the whole SQL engine and present it as production-ready. For the list of all SQL capabilities, refer to the SQL reference.

Persistence (Enterprise Edition)

For years, we provided persistence capabilities through MapLoader/MapStore SPI. However, the user had to provide the actual implementation. Another option was to leverage Hot Restart capabilities but in the practical application it turned out to be quite limiting in certain scenarios.

We introduce new persistence functionality, simply referred to as Persistence. This feature completely supersedes the Hot Restart subsystem which in turn will be deprecated. With Persistence, the content of your IMap and ICache gets written to disk making it even more resilient towards failures. At this point, Persistence is a mirror of the in-memory content, it’s not a tiered storage solution for going beyond memory limits.

Using Persistence, you can achieve the following scenarios with simple configuration changes:

• Safely restore the member data after an unexpected crash.
• You can control when the cluster should start rebalancing the data. For example, if you’re willingly shutting down one member just for maintenance with the intention of bringing it up soon, you likely want to avoid migrations and data reshuffling. Now you can control such behavior and shorten your maintenance window to an absolute minimum.
• Transfer a member to another hardware with reduced migration times. You can take the persistence files, put them on a new machine and start the member there. The member will load the data and do only the necessary (delta) migrations drastically reducing the maintenance window needed.

For the full documentation of the Persistence feature, visit the respective Persistence documentation section.

Streaming

Although the current Hazelcast Jet users might not get that excited, it’s worth mentioning that Hazelcast 5.0 effectively brings all of the battle-proven streaming capabilities to Hazelcast IMDG.

Former IMDG users can upgrade to Hazelcast 5.0 with no risk and zero downtime enabling use cases like:

• analyzing data using querying or streaming,
• add context to the real-time events,
• combine Hazelcast data with data from remote systems, and
• ingest data using SQL or Java through a rich suite of connectors for out-of-the-box connectivity to databases and message brokers.

Not sure where to start? Get started with stream processing in the client/server mode using Java or via SQL.

Increased HD Capacity (Enterprise Edition)

Up until now, the maximum supported amount of high-density memory configured per member was 100 GBs. We’re increasing this limit to 200 GBs per member, effectively doubling the maximum storage capacity on the same hardware. For the end-users, this means reducing the hardware cost tremendously and making the life of your DevOps team easier.

Compact Serialization Format (Preview Feature)

In addition to the current serialization formats, we’re releasing a preview of the new serialization format named Compact Format. which aims to not only improve the non-functional aspect of the serialization but also the user experience.

In particular, Compact Format’s goals are:

• Require much less storage space overhead
• Provide better latency/throughput when using in conjunction with other Hazelcast features
• Support much more extensive typeset
• Require much less configuration than Portable (zero configuration on common use-cases for Java only)
  • This format supports the schema evolution of a class and does not require any version to be specified explicitly.
  • Java DTO classes are automatically serialized without requiring a Serializer to be implemented on common use cases where the field types of an object are natively supported via Compact Format and the class has a default constructor.
  • User classes do not need to implement an interface like Portable. Once compact serialization is enabled, the classes which do not match any other serialization methods will be serialized in the new format.

To learn about the format, see Compact Serialization documentation.

Management Center UI Redesign

In addition to many new features, our web-based monitoring tool, Management Center, has gone through a complete UI redesign. We’re bringing a consistent, modern look focused on improved user experience with the release of Management Center 5.0.

Find out more about the Management Center in the Management Center Getting started guide.

The Future of IMDG and Jet

We do not plan to release any new minor versions of Hazelcast IMDG or Hazelcast Jet. All new feature development will continue in Hazelcast Platform. However, we’ll continue to support IMDG and Jet for the standard version support windows, as well as provide the patch releases of the supported versions.

Therefore, if you’re an existing user, you don’t have to be worried about your favorite in-memory data grid and streaming engine disappearing. It’s more of an evolution and setting up a new direction.

Closing Words

We would love to hear from you what you think. You can reach us directly via Hazelcast Community Slack or Google Groups for discussions, or report issues on the GitHub repository.

Happy Hazelcasting!

Useful Links:

• Download Open source or Enterprise Edition
• Documentation
• Release notes