Introducing Hazelcast Platform 5.1!

We are very happy to announce that Hazelcast Platform 5.1 is now generally available, via our usual download channels and to trial in Hazelcast Cloud.

Starting with the release of Hazelcast Platform 5.0 last September we merged our bulletproof in-memory data grid with our low-latency stream processing engine, and since then the 5.x line of Hazelcast is becoming our most rapidly adopted and most popular offering ever! 

Our goal is for the Hazelcast Platform to become the last-mile data storage and processing layer for real-time applications, be they transactional, batch, or streaming. Incorporating data processing, data storage and data integration in one distributed, easy to scale, highly available cluster. That is capable of handling terabytes of data and is deployable anywhere.

Hazelcast as a System of Record

With the release of Hazelcast Platform 5.1 we are pushing towards this goal, and perhaps the most notable feature in this release is Tiered Storage.  This will radically change the way Hazelcast is used within real-time system architectures.  Now Hazelcast can not only act as an ultra-low-latency data store but also now act as a durable system of record itself.

Tiered Storage will offer the opportunity to store datasets in Hazelcast that are much larger than the available memory.  For example, in the past, you would need to configure an eviction policy to remove least frequently used (LFU) or least recently used (LRU) data from the memory store when limits approach.  Now with Tiered Storage, Hazelcast can move data automatically between tiers, between memory and disk.

In 5.1 there will be two tiers available, memory and disk.  Soon we will also offer the capability to add extra tiers.  For example, the ability to have a fast in-memory tier, then a second SSD disk tier and a third tier such as Amazon S3, which could be used for less frequently accessed data.  In this way system architects can choose a varying range of storage options based on requirements of access speed, data set size, and cost.

Tiered Storage will be available as an Enterprise BETA feature with an expected production-ready release for 5.2 in the middle of this year.

SQL, Streaming SQL & JSON

We continue to advance our capabilities in the world of SQL, in 5.0 we extended our basic SQL DML support and we’re going further today with 5.1 as we add the ability to create views, create indexes and run troubleshooting explain plans.

One of the key challenges facing real-time application developers today is how they handle the new reality of data-in-motion together with the well-known world of data-at-rest. How do they build applications that combine streaming data with data stored in traditional databases and NoSQL stores?

The Hazelcast SQL Engineering team are at the forefront of shaping this future and are a participant in an industry standards body working group for Streaming SQL, which includes the likes of Confluent, Amazon, Microsoft, Google, and Oracle.

Combining data-at-rest and data-in-motion is an area in which Hazelcast Platform really shines and our work in the world of streaming data continues apace as we extend our SQL capabilities in this area.

In this release, we’ve added more comprehensive support for joining data, not only between maps but also between maps and other forms of streaming data. 

Imagine the ability to join a stream of changes from a relational database via CDC, to a stream from an event broker such as Apache Kafka or Apache Pulsar, and then enriching this with data from a Hazelcast Map, and finally creating a new view of this data in another Hazelcast Map. Which then of course is available at low latency to your application.

Starting with 5.1 there are additional keywords; union, union all, right join, exists and not exists.  We’ve also added SQL support for aggregating streams into tumbling and hopping windows, sometimes known as sliding windows.  This then gives you the ability to run functions and aggregations such as sum or count that is snapped to a time window.

Finally, a word about JSON.  In 5.1 you can now insert JSON into Hazelcast with a SQL statement and Hazelcast will automatically index it. This then allows for hierarchical queries of your JSON objects using SQL select statements together with JSON Path syntax.  

For example given a JSON dataset of countries.

"countries": [
  {
  "country":"United Kingdom",
  "cities": [{"city": "London"}]
  },
  {
  "country":"France",
  "cities": [{"city": "Lyon"}]
  },
  {
  "country":"Turkey",
  "cities": [{"city": "Istanbul"}]
  }
]

To return just the cities we might run a JSON Path SQL query such as

SELECT
   JSON_QUERY(this, '$.countries..cities WITH WRAPPER)
   FROM countries;

That would return the following

["London","Lyon","Istanbul"]

In future versions, we will continue to expand our JSON capabilities so that it becomes a much more viable option as a JSON document repository.

Dynamic Configuration Persistence

Hazelcast has had the ability to dynamically re-configure data structures for some time now; this is a well-used open-source feature. 

In 5.1 we’ve improved this situation, and now changes made to configuration are automatically persisted to the file-based configuration at the same time.  In addition, we’ve also added a new REST endpoint that forces a reload of configuration, so now you can edit your XML or YAML configuration file first and then ask Hazelcast to reload it.  This is available as an Enterprise feature.

Management Center

Our Management and Monitoring Engineering team have continued to make great improvements to the Hazelcast Management Center.  In 5.1 they’ve greatly improved the experience around SQL, with the ability to not only run standard DML but also to watch the results of streaming queries as they run live, with new data automatically appearing in the results window as it arrives at the cluster.

Also in Management Center 5.1, we’ve added the ability to view metrics on data structures within the CP Subsystem, such as AtomicLongs.  It also introduces management facilities for members within the CP subsystem.

Java 17 support

Hazelcast 5.1 has also been tested against Java 17, so whilst this is not a compile time requirement for your code, you can now feel safe that Hazelcast members will run as expected against a Java 17 JVM.

Wrapping Up

The improvements and new features we’ve outlined above are only the tip of the iceberg for this release.  You can read more about what’s new and what’s been fixed in our release notes. In summary, we’ve closed over 670 GitHub issues, making Hazelcast Platform 5.1 the most stable release of Hazelcast you can get.

As always we’d be nothing without our community and a special thanks this time around goes to Lenny Primak, Chelsea31 and Thomas Gaweda for their contributions to the open source codebase.

If you’d like to become more involved in our community or just ask some questions about Hazelcast please join us on our Slack channel and also please check out our new Developers homepage.

Again, don’t forget to try this new version and let us know what you think. Our engineers love hearing your feedback and we’re always looking for ways to improve.

 Just head over to our getting started page at https://hazelcast.com/get-started/