Seamless Integration with Vert.x: Boosting Performance and Scalability in Java Applications

This project was developed in collaboration with the Vert.x team, including project lead Julien Viet and core developer Thomas Segismont. Hazelcast software engineer Frantisek Hartman also contributed to this blog.

Introduction: A quick overview of Vert.x and Hazelcast Platform

Eclipse Vert.x is a toolkit for building reactive applications on the JVM. Reactive applications are designed to scale efficiently as workloads increase and remain resilient in the face of failures. These applications maintain low latency by optimally using system resources and safeguarding against errors. Vert.x is supported by a robust ecosystem of reactive modules, providing everything needed to develop modern services, including a full web stack, reactive database drivers, messaging, event streams, clustering, metrics, distributed tracing, and more.

Hazelcast Platform provides distributed low-latency querying, aggregation, and stateful computation on event streams and traditional data sources. It uniquely integrates a distributed compute engine with a fast data store in a single runtime, delivering high performance, resilience, and scalability for real-time and AI-driven applications. Hazelcast Platform enables rapid development of resource-efficient, real-time applications that are scalable from small edge devices to large cloud clusters.

It processes data across networked, clustered computers that combine their RAM, allowing applications to share data within the cluster. Storing data in RAM significantly speeds up applications by eliminating the need to retrieve it from disk. With Hazelcast Platform, you can store and process data in RAM, distribute it across a cluster, and replicate it to ensure resilience against member failures.

Clustering in Vert.x: Leveraging Hazelcast Platform for distributed applications

Clustering in Vert.x enables distributed applications to run multiple Vert.x instances across different nodes while functioning as a single system. This setup allows seamless communication between instances, the sharing of resources, and the distribution of tasks. Clustering is essential for building scalable, highly available, and fault-tolerant applications. It ensures that if one instance or node fails, other nodes in the cluster can take over, preventing downtime and improving reliability. However, Vert.x does not natively include a cluster manager responsible for managing node discovery and membership, subscription management, and shared data storage which creates a need for an external solution to handle these tasks efficiently.

The Hazelcast Platform implementation of the cluster manager addresses the need by providing a powerful and easy-to-integrate solution for Vert.x clustering. Hazelcast Platform is a popular solution allowing for distributed data storage and synchronization between nodes. As a cluster manager for Vert.x, it facilitates critical operations, such as node discovery, communication between nodes, and maintaining a consistent state across the cluster. With Hazelcast Platform, Vert.x instances can automatically discover each other, distribute workloads, and share session data, ensuring the cluster operates cohesively. This integration simplifies the setup of distributed Vert.x systems, providing robust and scalable clustering capabilities for applications.

Hazelcast Platform integration with Vert.x

There are two available modules for integrating Hazelcast Platform with Vert.x, each catering to different needs and offering varying levels of functionality:

• io.vertx:vertx-hazelcast This module is an integral part of the Vert.x ecosystem, maintained directly by the Vert.x team in collaboration with Hazelcast developers. Being open-source, it is licensed under the Apache 2 license. As a cluster manager, it is designed to work seamlessly within the Vert.x environment, providing essential clustering features such as distributed data, event bus clustering, and shared data functionality.

• com.hazelcast:vertx-hazelcast-enterprise: This module is built on top of the core `vertx-hazelcast` module, but it explicitly leverages Hazelcast Platform Enterprise Edition’s advanced capabilities. In addition to the standard clustering features, this module implements some advanced cluster manager functionalities. For instance, it uses the CP Subsystem, part of Hazelcast Platform Enterprise offering, to provide strong consistency guarantees for features like `io.vertx.core.shareddata.Lock` and `io.vertx.core.shareddata.Counter`. This makes it an ideal choice for applications with higher levels of consistency, fault tolerance, and advanced distributed data structures.

  By choosing between these modules, developers can opt for the flexibility and simplicity of the open-source `vertx-hazelcast`, or the enhanced consistency and feature set of Hazelcast Platform Enterprise Edition, depending on the specific requirements of their application.

Vert.x with Hazelcast Platform deployment

In a typical scenario, the user application integrates the Vert.x and Hazelcast Platform libraries into the same JVM (Java Virtual Machine) process. Instead of directly interacting with Hazelcast Platform, the application accesses its features through the APIs provided by Vert.x. This approach simplifies the integration of Hazelcast with Vert.x by exposing some Hazelcast Platform features through the Vert.x API, making the application portable and functional, even when running standalone (e.g., on a developer machine), while still requiring the user to understand and manage Hazelcast Platform as part of a distributed system when deployed, allowing developers to leverage its distributed data structures and clustering capabilities seamlessly within the Vert.x ecosystem. The integration ensures Vert.x manages the communication with Hazelcast Platform, streamlining the development of scalable and reactive applications without requiring a user to manage an independent Hazelcast Platform cluster.

Hazelcast Platform instances operate as full-fledged cluster members but are not designed to function as clients. However, these instances can be configured as “Lite members,” which do not hold data. In this configuration, the actual data is stored and managed by other full members of the Hazelcast Platform cluster, which may exist externally. 

This architecture is particularly beneficial in scenarios where a fast restart of certain members is required. For example, during an application code update, it may be necessary to restart specific instances without disrupting the entire cluster. In such cases, configuring Hazelcast Platform instances as Lite members ensures they can be restarted quickly without triggering data migration, as they do not store or manage any data. By avoiding the time-consuming process of migrating data across cluster members, this setup reduces downtime and improves the overall efficiency of the application, especially during routine maintenance or updates.

Using Lite members is especially recommended in dynamic environments where application code may need frequent updates or where high availability and minimal downtime are critical, as it ensures that restarts occur smoothly without affecting the integrity or performance of the data cluster.

Utilizing Vert.x Hazelcast Enterprise Cluster Manager

To integrate Vert.x Hazelcast Platform Enterprise Cluster Manager into your Vert.x application, follow these steps to set up and configure both the necessary dependencies and clustering functionality.

1. Add the dependency

First, you must add the `vertx-hazelcast-enterprise` dependency to your `pom.xml` (if you’re using Maven) or the equivalent configuration for your build tool (e.g., Gradle).

For Maven, include the following in your `dependencies` section:

```xml
    <dependency>
      <groupid>com.hazelcast</groupid> <artifactid>vertx-hazelcast-enterprise</artifactid>
      <version>5.0.0-SNAPSHOT</version>
    </dependency>
```

This will pull in the necessary Hazelcast Platform enterprise cluster manager module, which allows Vert.x to handle distributed session management, data sharing, and more advanced clustering features using Hazelcast Platform.

2. Add the Hazelcast Platform private repository

Since this artifact is hosted in Hazelcast Platform’s private repository, you will also need to configure your `repositories` section in `pom.xml` to include the Hazelcast Platform repository URL:

```xml
    <repositories>
        <repository>
        <id>private-repository</id>
        <name>Hazelcast Private Repository</name>
        <url>https://repository.hazelcast.com/release/</url>
    </repository> </repositories>
```

NOTE: Before the GA release of the `vert.x-hazelcast-enterprise` module you need to use the following repository:

```xml
<repositories>
    <repository>
      <id>private-snapshot-repository</id>
      <name>Hazelcast Private Snapshot Repository</name>
      <url>https://repository.hazelcast.com/snapshot/</url>
    </repository>
</repositories>
```

This ensures Maven can locate and download the `vertx-hazelcast-enterprise` dependency from Hazelcast’s private repository. Make sure you have the appropriate access to this repository if it’s a protected resource. To enable clustering in your Vert.x application, you need to run the application in clustered mode. This is achieved by adding the `-cluster` flag when starting your Vert.x application.

Vert.x and Hazelcast Platform-supported versions

Step-by-step tutorial

Refer to the Vert.x and Hazelcast Platform Cluster Manager site for the official documentation of the `vertx-hazelcast` module.

Additionally, you can follow our step-by-step guide, which explains how to integrate Vert.x with Hazelcast Platform for distributed session management and other distributed data structures.

In this guide, we will:

• Begin with a basic Vert.x “Hello World” application.
• Add the `vertx-hazelcast` module and enable distributed session management.
• Use the `io.vertx.core.shareddata.Counter` data structure to implement a unique ID generator.

Follow the following link for the step-by-step guide: https://docs.hazelcast.com/hazelcast/latest/integrate/vertx-step-by-step-guide 

Summary

Integrating Hazelcast Platform with Vert.x provides a powerful solution for building scalable, high-performance, resilient Java applications. The seamless combination of Vert. x’s reactive, event-driven architecture and Hazelcast Platform’s distributed data processing capabilities enable developers to create modern, real-time applications that can handle scalable workloads while maintaining low latency. 

By choosing between the open-source `vertx-hazelcast` module for simplicity or the `vertx-hazelcast-enterprise` module for advanced features like strong consistency and enhanced fault tolerance, developers can tailor their solutions to meet specific needs. This integration reduces complexity, abstracts low-level management tasks, and ensures optimal resource efficiency, making it ideal for dynamic environments requiring high availability and minimal downtime. Together, Vert.x and Hazelcast Platform provide a robust platform for scaling Java applications to meet the demands of modern, distributed systems.

References

• Vert.x Docs https://vertx.io/docs/vertx-hazelcast/java/
• Hazelcast Platform Docs: https://docs.hazelcast.com/