Hazelcast Platform Feature Comparison
Compare features across the Hazelcast editions.
Hazelcast 5.x In-Memory Computing Features
Distributed Caching | Hazelcast Platform - Open Source Edition | Hazelcast Platform - Enterprise Edition |
---|---|---|
CP Subsystem
The CP subsystem is a component of a Hazelcast cluster that builds an in-memory strongly consistent layer. It is accessed via HazelcastInstance.getCPSubsystem(). Its data structures are CP with respect to the CAP principle, i.e., they always maintain linearizability and prefer consistency over availability during network partitions. | ||
CP Subsystem Persistence
CP Subsystem Persistence writes in-memory data for the CP Subsystem to disk to enable fast recovery without sacrificing data safety guarantees. | ||
JSON Support
Hazelcast recognizes JSON data structures when saved as a value to a Map using the HazelcastJsonValue type. Once saved, all standard operations can be carried out, such as Predicate Queries and Aggregations. Hazelcast support for In-Memory JSON storage provides a 400% increase in throughput when compared to popular NoSQL Document Stores. JSON support is available for all Hazelcast clients. | ||
Pipelining
A new convenience API for rapid populations of the cluster is now available. The Pipelining API manages multiple async ingests. You can send multiple requests in parallel using a single thread and therefore can increase throughput. | ||
AtomicLong
IAtomicLong, Hazelcast distributed implementation of java.util.concurrent.atomic.AtomicLong, offers most of AtomicLong’s operations such as get, set, getAndSet, compareAndSet and incrementAndGet. Since IAtomicLong is a distributed implementation, these operations involve remote calls and hence their performances differ from AtomicLong. You can send functions to an IAtomicLong. The reason for using a function instead of a simple code line like atomicLong.set(atomicLong.get() + 2)); is that the IAtomicLong read and write operations are not atomic. Since IAtomicLong is a distributed implementation, those operations can be remote ones, which may lead to race problems. By using functions, the data is not pulled into the code, but the code is sent to the data. This makes it more scalable. | ||
AtomicReference
IAtomicReference, the Hazelcast distributed implementation of java.util.concurrent.atomic.AtomicReference, offers compare-and-set and get-and-set operations on object references that are guaranteed atomic across application instances in a cluster. | ||
CountDownLatch
ICountDownLatch, the Hazelcast distributed implementation of java.util.concurrent.CountDownLatch, is a synchronization aid that allows one or more threads––in one or more application instances––to wait until a set of operations being performed in other threads across the cluster completes. ICountDownLatch is initialized with a given count. The countDown() method is a non-blocking operation that decrements the count. When the count reaches zero, all threads blocking on the await() method are allowed to proceed. | ||
Cardinality Estimator Service (HyperLogLog)
The Hazelcast cardinality estimator service is a data structure which implements Flajolet’s HyperLogLog algorithm for estimating cardinalities of unique objects in theoretically huge data sets. The implementation offered by Hazelcast includes improvements from Google’s version of the algorithm, i.e., HyperLogLog++. Some common use cases includes:
| ||
CRDT PN Counter
The PN Counter is a lightweight Positive-Negative Counter implementation, which is a CRDT (Conflict-free Replicated Data Type). Each cluster member can increment and decrement the counter value and these updates are propagated to all members. Since operations are local (on a Hazelcast member), your application can achieve great performance for high-volume traffic, such as counting likes, page views or connected user count. | ||
FencedLock
FencedLock is a linearizable & distributed & reentrant implementation of j.u.c.locks.Lock. FencedLock is accessed via CPSubsystem.getLock(String). It is CP with respect to the CAP principle. It works on top of the Raft consensus algorithm. It offers linearizability during crash-stop failures and network partitions. If a network partition occurs, it remains available on at most one side of the partition. FencedLock works on top of CP sessions. Please see CP Sessions section for more information about CP sessions. | ||
Flake ID Generator
Hazelcast Flake ID Generator is used to generate cluster-wide unique identifiers. Generated identifiers are long primitive values and are k-ordered (roughly ordered). IDs are in the range from 0 to Long.MAX_VALUE. | ||
IdGenerator
IdGenerator is a distributed id generator that facilitates creating ids that are unique across application instances in a cluster. | ||
List
List is similar to Set, but List also allows duplicate elements. List also preserves the order of elements. List is also a non-partitioned data structure where values and each backup are represented by their own single partition. List cannot be scaled beyond the capacity of a single machine. All items are copied to local and iteration occurs locally. | ||
Lock
ILock is the distributed implementation of java.util.concurrent.locks.Lock. If you lock using an ILock, the critical section that it guards is guaranteed to be executed by only one thread in the entire cluster. Even though locks are great for synchronization, they can lead to problems if not used properly. Also note that Lock does not support fairness. Locks can be configured to check for a minimum number of available members before applying lock operations (Split-Brain Protection). This is a check to avoid performing successful lock operations on all parts of a cluster during a network partition. | ||
Map
Map (IMap) extends the interface java.util.concurrent.ConcurrentMap and hence java.util.Map. It is the distributed implementation of Java map. You can perform operations like reading and writing from/to a Hazelcast map with the well known get and put methods. In addition Search and Map/Reduce can be run on Maps. Finally, maps may be integrated with a database using MapStore. | ||
MultiMap
MultiMap is a specialized map where you can store multiple values under a single key. Just like any other distributed data structure implementation in Hazelcast, MultiMap is distributed and thread-safe. | ||
Queues
Hazelcast distributed queue is an implementation of java.util.concurrent.BlockingQueue. Being distributed, it enables all cluster members to interact with it. Using Hazelcast distributed queue, you can add an item in one machine and remove it from another one. Queues can be configured to check for a minimum number of available members before applying queue operations (Split-Brain Protection). This is a check to avoid performing successful queue operations on all parts of a cluster during a network partition. | ||
Replicated Map
Unlike IMap, which is partitioned to balance data across the cluster, ReplicatedMap is fully replicated such that all members have the full map in memory. It replication is weakly consistent––rather than eventually consistent––and done on a best-effort basis. ReplicatedMaps have faster read-write characteristics, since all data is present in local members and writes happen locally and eventually replicated. Replication messages are also batched to minimize network operations. ReplicatedMaps are useful for immutable objects, catalog data, or idempotent calculable data. | ||
Ringbuffer
Ringbuffer is a lock-free distributed data structure that stores its data in a ring-like structure. Think of it as a circular array with a given capacity. Each Ringbuffer has a tail, where the items are added, and a head, where the items are overwritten or expired. You can reach each element in a Ringbuffer using a sequence ID, which is mapped to the elements between the head and tail (inclusive) of the Ringbuffer. It supports single and batch operations and is very high-performance. Hazelcast also allows you to load and store the Ringbuffer items from/to a persistent datastore using the interface RingbufferStore. If a Ringbuffer store is enabled, each item added to the Ringbuffer will also be stored at the configured Ringbuffer store. | ||
Semaphores
ISemaphore is the distributed implementation of java.util.concurrent.Semaphore. Semaphores offer permits to control the thread counts in the case of performing concurrent activities. To execute a concurrent activity, a thread grants a permit or waits until a permit becomes available. When the execution is completed, the permit is released. | ||
Set
A Set is a collection where every element only occurs once and where the order of the elements doesn’t matter. The Hazelcast com.hazelcast.core.ISet implements the java.util.Set. Set is a distributed and concurrent implementation of java.util.Set. In Hazelcast, the ISet (and the IList) is implemented as a collection within MultiMap, where the id of the set is the key in the MultiMap and the value is the collection. | ||
Topic and ReliableTopic
Hazelcast provides a distribution mechanism for publishing messages that are delivered to multiple subscribers. This is also known as a publish/subscribe (pub/sub) messaging model. Publishing and subscribing operations are cluster wide. When a member subscribes to a topic, it is actually registering for messages published by any member in the cluster, including the new members that joined after you add the listener. ReliableTopic is backed by a Ringbuffer with a backup to avoid message loss and to provide isolation between fast producers and slow consumers. | ||
DISTRIBUTED COMPUTE | ||
Entry Processor
An entry processor enables fast in-memory operations on a map without having to worry about locks or concurrency issues. It can be applied to a single map entry or to all map entries. It supports choosing target entries using predicates. You do not need any explicit lock on entry: Hazelcast locks the entry, runs the EntryProcessor, and then unlocks the entry. Hazelcast sends the entry processor to each cluster member and these members apply it to map entries. Therefore, if you add more members, your processing is completed faster. | ||
Executor Service
One of the coolest features of Java 1.5 is the Executor framework, which allows you to asynchronously execute your tasks (logical units of work), such as database query, complex calculation, and image rendering. The default implementation of this framework (ThreadPoolExecutor) is designed to run within a single JVM. In distributed systems, this implementation is not desired since you may want a task submitted in one JVM and processed in another one. Hazelcast offers IExecutorService for you to use in distributed environments: it implements java.util.concurrent.ExecutorService to serve the applications requiring computational and data processing power. With IExecutorService, you can execute tasks asynchronously and perform other useful tasks. If your task execution takes longer than expected, you can cancel the task execution. In the Java Executor framework, tasks are implemented as java.util.concurrent.Callable and java.util.Runnable. If you need to return a value and submit to Executor, use Callable. Otherwise, use Runnable (if you do not need to return a value). Tasks should be Serializable since they will be distributed. | ||
Scheduled Executor
Scheduled Executor is the distributed implementation of Java’s ScheduledExecutorService API. You can now schedule tasks at a given moment in time, or repetitive scheduling at fixed intervals in your cluster. | ||
Partition Predicate
Partition Predicate is a control mechanism for QueryEngine to schedule a Query. Once scheduled, Partition Predicate strips away from actual Predicate, and then actual Predicate is executed on appropriate partition. This benefits applications making use of smart partitioning. Off-heap memory management is enabled in Hazelcast Enterprise via High-Density Memory Store | ||
User Defined Services
In the case of special/custom needs, Hazelcast SPI (Service Provider Interface) module allows users to develop their own distributed data structures and services. The SPI makes it possible to write first class distributed services/data-structures yourself. With the SPI, you can write your own data-structures if you are unhappy with the ones provides by Hazelcast. You also could write more complex services, such as an Actor library. | ||
DISTRIBUTED QUERY | ||
Fast Aggregations
Prior to Hazelcast 3.8, aggregations were based on our Map-Reduce engine. Fast Aggregations functionality is the successor of the Aggregators. Instead of running on the MapReduce engine they run on the Query infrastructure. Their performance is tens to hundreds times better since they run in parallel for each partition and are highly optimized for speed and low memory consumption. Much faster and a simpler API. R aggregate(Aggregator, R> aggregator); | ||
Continuous Query
ContinuousQueryCache ensures that all update messages are available in local memory for fast access. This is beneficial when queries on distributed IMap data are very frequent and local, in-memory performance is required. | ||
Listener with Predicate
Listener with Predicate enables you to listen to the modifications performed on specific map entries. It is an entry listener that is registered using a predicate. This makes it possible to listen to the changes made to specific map entries. | ||
Query
Hazelcast partitions your data and spreads it across cluster of servers. You can iterate over the map entries and look for certain entries (specified by predicates) you are interested in. However, this is not very efficient because you will have to bring the entire entry set and iterate locally. Instead, Hazelcast allows you to run distributed queries on your distributed map. If you add new members to the cluster, the partition count for each member is reduced and hence the time spent by each member on iterating its entries is reduced. Therefore, the Hazelcast querying approach is highly scalable. Another reason it is highly scalable is the pool of partition threads that evaluates the entries concurrently in each member. The network traffic is also reduced since only filtered data is sent to the requester. | ||
CLIENTS | ||
C#/.NET Client
You can use the native .NET client to connect to Hazelcast nodes. All you need is to add HazelcastClient3x.dll into your .NET project references. The API is very so the Java native client. | ||
Near Cache for C#/.NET Client
Near Cache allows a subset of data to be cached locally in memory on the C#/.NET Client. | ||
C++ Client
You can use Native C++ Client to connect to Hazelcast nodes and perform almost all operations that a node can perform. Clients differ from nodes in that clients do not hold data. The C++ Client is by default a smart client, i.e. it knows where the data is and asks directly for the correct node. The features of C++ Clients are:
| ||
C++ Client
You can use Native C++ Client to connect to Hazelcast nodes and perform almost all operations that a node can perform. Clients differ from nodes in that clients do not hold data. The C++ Client is by default a smart client, i.e. it knows where the data is and asks directly for the correct node. The features of C++ Clients are:
| ||
Java Client
Native Clients (Java, C#, C++) enable you to perform almost all Hazelcast operations without being a member of the cluster. It connects to one of the cluster members and delegates all cluster wide operations to it (dummy client), or it connects to all of them and delegates operations smartly (smart client). When the relied cluster member dies, the client will transparently switch to another live member. The Java client is the most full featured client. It is offered both with Hazelcast Open Source and Hazelcast Enterprise editions. The main idea behind the Java client is to provide the same Hazelcast functionality by proxying each operation through a Hazelcast node. It can access and change distributed data, and it can listen to distributed events of an already established Hazelcast cluster from another Java application. | ||
Near Cache for Java Client
Near Cache allows a subset of data to be cached locally in memory on the Java Client. Off-heap memory management is enabled in Hazelcast Enterprise via High-Density Memory Store | ||
Node.js Client
Node.js Client for Hazelcast, using Hazelcast Open Client Protocol 1.0 for Hazelcast 3.6 and higher. | ||
Near Cache for Node.js Client
Near Cache allows a subset of data to be cached locally in memory on the Node.js Client. | ||
Python Client
The Python client is the reference implementation of the new Hazelcast Client Binary Protocol. Hazelcast is a robust in-memory computing platform now available to Python applications. | ||
Near Cache for Python Client
Near Cache allows a subset of data to be cached locally in memory on the Python Client. | ||
Scala Client
The Hazelcast Scala API is based on Scala 2.11 and Hazelcast 3.6, but does not define them as hard dependencies, so make sure to also include the relevant Hazelcast dependencies. | ||
Near Cache for Scala Client
Near Cache allows a subset of data to be cached locally in memory on the Scala Client. Off-heap memory management is enabled in Hazelcast Enterprise via High-Density Memory Store | ||
Go Client
With the Go Client for Hazelcast, the fastest open source caching solution is now available to Go applications. Elastically scale go caching. Hazelcast is an excellent Redis alternative when scaling and speed are of importance. Go Client for Hazelcast uses the Hazelcast Open Client Protocol 1.6 for Hazelcast 3.6 and higher and supports Go 1.6+. | ||
Portable Serialization
As an alternative to the existing serialization methods, Hazelcast offers a language/platform independent Portable serialization that has the following advantages:
Portable serialization is totally language independent and is used as the binary protocol between Hazelcast server and clients. | ||
Pluggable Serialization
You need to serialize the Java objects that you put into Hazelcast because Hazelcast is a distributed system. The data and its replicas are stored in different partitions on multiple nodes. The data you need may not be present on the local machine, and in that case, Hazelcast retrieves that data from another machine. This requires serialization. Hazelcast serializes all your objects into an instance of com.hazelcast.nio.serialization.Data. Data is the binary representation of an object. Serialization is used when:
| ||
CLOUD AND VIRTUALIZATION SUPPORT | ||
Amazon Web Services
Hazelcast AWS cloud module helps Hazelcast cluster members discover each other and form the cluster on AWS. It also supports tagging, IAM Role, and connecting clusters from clients outside the cloud. | ||
Apache jclouds
Hazelcast supports the Apache jclouds API, allowing applications to be deployed in multiple different cloud infrastructure ecosystems in an infrastructure-agnostic way. | ||
Apache Zookeeper Discovery
The Hazelcast Zookeeper Discovery Plugin provides a service based discovery strategy by using Apache Curator for communicating your Zookeeper server for Hazelcast 3.6.1+ Discovery SPI enabled applications. | ||
Azure Cloud Discovery
Azure Discovery is for Hazelcast 3.6.1 and above. It provides all Hazelcast instances in a cluster by returning VMs within your Azure resource group that are tagged with a specified value. | ||
Consul
Consul is a tool for discovering and configuring services in your infrastructure. It provides: Service Discovery, Health Checking, Key/Value Store, and Multi Datacenter support out-of-the-box. | ||
etcd
The etcd plugin provides an etcd-based discovery strategy for Hazelcast 3.6-EA+ enabled applications. This is an easy to configure plug-and-play Hazelcast DiscoveryStrategy that will optionally register each of your Hazelcast instances with etcd and enable Hazelcast nodes to dynamically discover one another via etcd. | ||
Discovery Service Provider Interface (SPI)
The Discovery plugin is an extension SPI to attach external cloud discovery mechanisms. Discovery finds other Hazelcast instances based on filters and provides their corresponding IP addresses. The SPI ships with support for Apache jclouds and Google’s Kubernetes as reference implementations. | ||
Docker
Docker containers wrap up Hazelcast in a complete filesystem that contains everything it needs to run – code, runtime, system tools, system libraries – guaranteeing that it will always run the same, regardless of the environment it is running in. | ||
Eureka Cloud Discovery
Hazelcast Discovery SPI plugin for Netflix’ Eureka Service Discovery v1. | ||
Kubernetes
Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. | ||
Pivotal Cloud Foundry
Hazelcast for Pivotal Cloud Foundry is an on-demand service broker that dynamically creates and scales Hazelcast clusters without pre-provisioning blocks of Virtual Machines. | ||
OpenShift Container Platform
The Hazelcast Docker Image is an extension of official Hazelcast Docker image packed with Kubernetes discovery plugin which enables deployment of Hazelcast on your Openshift platform as managed cache service. | ||
INTEGRATED CLUSTERING | ||
Hibernate Second Level Cache
Hazelcast provides a distributed second level cache for your Hibernate entities, collections and queries. This cache associates with the Session Factory object. This cache is not restricted to a single session, but is shared across sessions, so data is available to the entire application, not just the current user. This can greatly improve application performance as commonly used data can be held in memory in the application tier. Implied by the name, Hibernate will go to the first level cache first, and if the entity is not there, it will go to the second level. See the docs for this feature | ||
Generic Web Sessions
Filter based Web Session Replication for JEE Web Applications without requiring changes to the application. Filter Session Replication is a feature where the state of each created HttpSessionObject is kept in a distributed Map. If one of the servers goes down, users will be routed to other servers without their noticing that a server went down. Also, you can scale your web servers under heavy load, and easily add new servers to your cluster. We use delta updates for efficient operation even with the largest session objects. | ||
Jetty Clustered Web Sessions
Session Replication is a container specific module that enables session replication for JEE Web Applications without requiring changes to the application. Jetty Session Replication is a Hazelcast module where the state of each created HttpSessionObject is kept in a distributed Map. If one of the servers goes down, users will be routed to other servers without their noticing that a server went down. Also, you can scale your web servers under heavy load, and easily add new servers to your cluster. We use delta updates for efficient operation even with the largest session objects. | ||
Tomcat Clustered Web Sessions
Session Replication is a container specific module that enables session replication for JEE Web Applications without requiring changes to the application. Tomcat Session Replication is a Hazelcast module where the state of each created HttpSessionObject is kept in a distributed Map. If one of the servers goes down, users will be routed to other servers without their noticing that a server went down. Also, you can scale your web servers under heavy load, and easily add new servers to your cluster. We use delta updates for efficient operation even with the largest session objects. | ||
CLIENT-SERVER PROTOCOLS | ||
Memcached Client
A Memcached client written in any language can talk directly to a Hazelcast cluster. No additional configuration is required. (Hazelcast Memcached Client only supports ASCII protocol. Binary Protocol is not supported.) | ||
Open Binary Client Protocol and Client Implementation Guide
Hazelcast’s new client-server protocol now supports versioning and easy client implementation. This provides enterprises deployment and upgrade flexibility by allowing clients to be upgraded independently of servers. Caching services may be deployed and upgraded enterprise-wide, without forcing clients across business units to upgrade in lock step. The accompanying protocol documentation and client implementation guide also allows clients to be easily implemented in any platform. The implementation guide ships with a Python reference implementation. | ||
REST
The Clustered REST API is exposed from Management Center to allow you to monitor clustered statistics of distributed objects. | ||
STANDARDS | ||
JCache
JCache is the standardized Java caching layer API. The JCache caching API is specified by the Java Community Process (JCP) as Java Specification Request (JSR) 107. Starting with release 3.3.1, Hazelcast offers a specification compliant JCache implementation. It is not just a simple wrapper around the existing APIs; it implements a caching structure from ground up to optimize the behavior to the needs of JCache. The Hazelcast JCache implementation is 100% TCK (Technology Compatibility Kit) compliant and therefore passes all specification requirements. It has asynchronous versions of almost all operations to give the user extra power. | ||
Apache jclouds Support
Hazelcast supports the Apache jclouds API, allowing applications to be deployed in multiple different cloud infrastructure ecosystems in an infrastructure-agnostic way. | ||
CLUSTER MANAGEMENT | ||
Management Center
Management Center enables you to monitor and manage your nodes running Hazelcast. In addition to monitoring overall state of your clusters, you can also analyze and browse your data structures in detail, update map configurations and take thread dump from nodes. With its scripting and console module, you can run scripts (JavaScript, Groovy, etc.) and commands on your nodes. | ||
User Code Deployment
User Code Deployment enables you to load new classes to Hazelcast nodes dynamically without restarting all servers. You can call it “distributed classloading”. If it is enabled in your cluster, a new class will be copied and loaded by other servers transparently. | ||
Advanced Network Configuration
Prior to 3.12, all network communications to a cluster member were handled via the same listener configuration. Now, unique network configurations can be applied to client-member, member-member, WAN-WAN, REST & Memcache endpoints. Practically this now means that different public addresses and ports can be used, as well as different TLS certificates and TCP/IP configurations per service. | ||
Unlimited node count for Management Center (Open Source Edition limited to 3-node clusters) | ||
Clustered JMX
Clustered JMX via Management Center allows you to monitor clustered statistics of distributed objects from a JMX interface. You can use jconsole or any other JMX client to monitor your Hazelcast Cluster. Use the Clustered JMX interface to integrate Hazelcast Management Center with New Relic and AppDynamics. | ||
Clustered REST
For Hazelcast Enterprise, the Clustered REST API is exposed from Management Center to allow you to monitor clustered statistics of distributed objects. To enable Clustered REST on your Management Center, you need only pass a system property at startup. | ||
Rolling Upgrades
Rolling Upgrade is the ability to upgrade cluster nodes’ versions without service interruption. Currently this is only supported between patch versions, e.g., from 3.7.1 to 3.7.3. Starting from Hazelcast 3.8, we support Rolling Upgrade among minor versions too. You will be able to upgrade your cluster nodes from 3.8 to 3.9 without any service interruption. You have to restart one node at a time in order to not lose any data. | ||
WAN Replication
There are cases where you need to synchronize multiple clusters to the same state. Synchronization of clusters, also known as WAN (Wide Area Network) Replication, is mainly used for replicating stats of different clusters over WAN environments like the Internet. | ||
Delta WAN Synchronization (Enterprise Only)
Separate Clusters connected by WAN can become out-of-sync for various reasons. Currently, Maps in different clusters may be synced only by transferring all entries from the source to the target cluster. This may be inefficient as some entries could be equal on both clusters and do not require to be transferred. With Hazelcast 3.11, Delta WAN Synchronisation can be enabled rather than full synchronization. This will detect if two clusters become out-of-sync by comparing a tracking data structure called a Merkle Tree. Differences can be detected and sent to the opposite cluster. Enterprise WAN Replication now contains additional diagnostics making it easier to analyze and monitor replication flows. | ||
Enhanced WAN Replication Configuration and Monitoring
The Management Center now has a dedicated interface for configuration and monitoring of WAN replication. | ||
CLIENT FAILOVER AND MIGRATION MANAGEMENT | ||
Blue/Green Deployments
Blue/Green Deployments reduce downtime and risk by running two identical Hazelcast® Enterprise clusters called Blue and Green. One of the clusters provides production services to clients whilst the other cluster can be upgraded with the new application code. Hazelcast Blue/Green Deployment functionality allows clients to be migrated from one cluster to another without client interaction or cluster downtime. All clients of a cluster may be migrated or groups of clients can be moved with the use of label filtering and black/white lists via Management Center. | ||
Automatic Disaster Recovery Failover
Automatic disaster recovery failover provides clients connected to Hazelcast® Enterprise clusters with the ability to automatically failover to a Disaster Recovery cluster should there be an unplanned outage of the primary production Hazelcast cluster. | ||
STORAGE | ||
On-Heap
The on-heap store refers to objects that will be present in the Java heap (and also subject to GC). Java heap is the space that Java can reserve and use in memory for dynamic memory allocation. All runtime objects created by a Java application are stored in heap. By default, the heap size is 128 MB, but this limit is reached easily for business applications. Once the heap is full, new objects cannot be created and the Java application shows errors. | ||
Hot Restart Store
Provides fast restart regardless of data size by persisting node state to disk so nodes can load data into memory without blocking on an overtaxed backing store. Hot Restart Store allows a partial start which basically means the cluster can be restarted with some missing members. Servers can have different IP addresses before and after the restart. In other words, you can use Hot Restart Store in many use cases like moving data to other servers, replacing a server, having different cloud instances. During Hot Restart operations you can take a snapshot of the Hot Restart Store at a certain point in time. This is useful when you wish to bring up a new cluster with the same data or parts of the data. The new cluster can then be used to share load with the original cluster, to perform testing, QA or reproduce an issue on production data. | ||
High-Density Memory Store (HDMS)
High-Density Memory Store is an add-on product to Hazelcast Enterprise that allows Java applications to near cache 100s of GBs of data from off-heap memory without garbage collection overhead and nanosecond access times. In Hazelcast Enterprise, the High-Density Memory Store is built around a pluggable memory manager which enables multiple memory stores. These memory stores are all accessible using a common access layer that scales up to Terabytes of main memory on a single JVM. At the same time, by further minimizing the garbage collection pressure, High-Density Memory Store enables predictable application scaling and boosts performance and latency while minimizing pauses for Java garbage collection. | ||
High-Density Memory Store for Intel Optane PMem
High-Density Memory Store is certified for use on Intel Optane PMem as an alternative to RAM with comparable speeds at higher densities and lower costs. | ||
High-Density Memory Store for Maps
| ||
High-Density Memory Store for JCache and JCache Near Cache
| ||
High-Density Memory Store Near Cache for Java Client | ||
High-Density Memory Store Near Cache for Scala Client | ||
High-Density Memory Store for Web Sessions | ||
High-Density Memory Store for Hibernate 2nd Level Cache | ||
High-Density Memory Store for Predicate & Partition Predicate | ||
SECURITY SUITE | ||
SSL/TLS 1.3 Asymmetric Encryption
Provides encryption based on TLS Certificates between members, between clients and members, and between members and Management Center. | ||
SSL/TLS 1.3 Asymmetric Encryption Java SSLEngine
Uses the SSLEngine built-in to the JDK with some performance enhancements. | ||
SSL/TLS 1.3 Asymmetric Encryption with OpenSSL
Uses the industry standard OpenSSL native library packaged as an additional module in Hazelcast. | ||
Allowed Connection IP Ranges
| ||
Authentication
The authentication mechanism for Hazelcast client security works the same as cluster member authentication. To implement client authentication, configure a Credential and one or more LoginModules. The client side does not have and does not need a factory object to create Credentials objects like ICredentialsFactory. Credentials must be created at the client side and sent to the connected node during the connection process. | ||
Authorization
Hazelcast client authorization is configured by a client permission policy. Hazelcast has a default permission policy implementation that uses permission configurations defined in the Hazelcast security configuration. Default policy permission checks are done against instance types (map, queue, etc.), instance names (map, queue, name, etc.), instance actions (put, read, remove, add, etc.), client endpoint addresses, and client principal defined by the Credentials object. Instance and principal names and endpoint addresses can be defined as wildcards(*). | ||
Encryption at Rest
Data in the Hot Restart Store is encrypted to prevent unauthorized access to the data-at-rest. Key rotation is supported to protect the master key to add an additional safeguard against data breaches. | ||
JAAS Module
Hazelcast has an extensible, JAAS based security feature you can use to authenticate both cluster members and clients, and to perform access control checks on client operations. Access control can be done according to endpoint principal and/or endpoint address. | ||
LDAP Integration
Hazelcast Enterprise includes JAAS login modules to authenticate against LDAP servers either in plaintext or TLS-protected connections. | ||
Pluggable Socket Interceptor
Hazelcast allows you to intercept socket connections before a node joins to cluster or a client connects to a node. This provides the ability to add custom hooks to join and perform connection procedures (like identity checking using Kerberos, etc.). | ||
Security Interceptor
Hazelcast allows you to intercept every remote operation executed by the client. This lets you add a very flexible custom security logic. | ||
FIPS 140-2 Validation
The Federal Information Processing Standard (FIPS) 140-2 is a US government computer security standard published by National Institute of Standards and Technology (NIST). It specifies the security requirements for cryptographic modules. FIPS 140-2 compliance is often a requirement of the software systems used by the US government agencies. Hazelcast can now run in the FIPS compliant mode. |
Hazelcast Stream/Batch Processing Features
DISTRIBUTED COMPUTATION | Hazelcast Platform - Open Source Edition | Hazelcast Platform - Enterprise Edition |
---|---|---|
Pipeline API
Pipeline API is the primary API of Hazelcast for processing both bounded and unbounded data. Use it to declare the data processing pipelines by composing high-level operations (such as map, filter, group, join, window) on a stream of records. The Pipeline API is a Java 8 API with static type safety. | ||
Core API
Core API is a low-level API that directly exposes the computation engine’s raw features (DAGs, partitioning schemes, vertex parallelism, distributed vs. local edges, etc.). | ||
STREAM PROCESSING | ||
Streaming Core
Hazelcast is built on top of a low latency streaming core. This refers to processing the incoming records as soon as possible as opposed to accumulating the records into micro-batches before processing. | ||
Windowing
As data streams are unbounded and there is the possibility of infinite sequences of records, a tool is required to group individual records to finite frames in order to run the computation. Hazelcast windows provides a tool to define the grouping. Types of windows supported by Hazelcast:
| ||
Event Time Processing
Hazelcast allows you to classify records in a data stream based on the timestamp embedded in each record — the event time. Event time processing is a natural requirement as users are mostly interested in handling the data based on the time that the event originated (the event time). Event time processing is a first-class citizen in Hazelcast. For handling late events, there is a set of policies to determine whether the event is “still on time” or “late”, which results in the discarding of the latter. | ||
Handling Back Pressure
In the streaming system, it is necessary to control the flow of messages. The consumer cannot be flooded by more messages than it can process in a fixed amount of time. On the other hand, the processors should not be left idle wasting resources. Hazelcast comes with a mechanism to handle back pressure. Every part of the Hazelcast job keeps signaling to all the upstream producers how much free capacity it has. This information is naturally propagated upstream to keep the system balanced. | ||
GUARANTEES | ||
Exactly-Once or At-Least Once
Hazelcast supports distributed state snapshots. Snapshots are periodically created to back up the running state. Periodic snapshots are used as a consistent point of recovery for failures. Snapshots are also taken and used for up-scaling. For snapshot creation, exactly-once or at-least-once semantics can be used. This is a trade-off between correctness and performance. It is configured per job. | ||
Exactly-Once
Hazelcast ensures exactly-once semantics when a replayable source (e.g., Kafka) is used with an idempotent sink (e.g., any store with upsert functionality). | ||
Two-Phase Commit for Exactly-Once
Hazelcast supports the two-phase commit protocol to enable exactly-once guarantees on more types of sources and sinks to participate in transaction-based streaming. This capability currently adds JMS as a source, and both Kafka and files as sinks, with more planned. | ||
RELIABILITY | ||
Fault-Tolerance
Hazelcast is able to tolerate faults such as network failure, split or node failure, with its redundancy in the cluster. This is just one of the business continuity capabilities in Hazelcast that let customers achieve 99.999% uptime, with many customers reporting zero downtime in production. When there is a fault, Hazelcast uses the latest state snapshot and automatically restarts all jobs that contain the failed member as a job participant from this snapshot. | ||
Resilient Snapshot Storage
Hazelcast uses the distributed in-memory storage to store the snapshots. This storage is an integral component of the Hazelcast cluster, no further infrastructure is necessary. Data is stored in multiple replicas distributed across the cluster to increase the resiliency. | ||
IN-MEMORY COMPUTING | ||
In-Memory Data Store Integration
Hazelcast has an integrated, elastic in-memory data store, to provide a highly optimized read and write to distributed, in-memory implementations of java.util.Map, java.util.Cache and java.util.List. The in-memory store is to be used for:
| ||
Embedded In-Memory Storage
The Hazelcast in-memory data store is embedded. So, all the services of the in-memory store are available to your Hazelcast jobs without any additional deployment effort. To isolate the processing from the storage, you can still make use of Hazelcast processing in a separate cluster reading from or writing to remote Hazelcast in-memory-only clusters. | ||
Streaming from In-Memory Data Store
In Hazelcast, a connector is included which allows the user to process streams of changes (Event Journal) of an IMap and ICache, enabling developers to stream process IMap/ICache data or to use the Hazelcast in-memory data store as storage for data ingestion. | ||
CONNECTORS | ||
Hazelcast In-Memory Data Structures
High-performance readers and writers for Hazelcast IMap, ICache and IList. The IMap and ICache are partitioned and distributed. Hazelcast makes use of data locality reading the data from the same node to prevent network transit penalty. The streaming connectors for IMap and ICache allow the user to treat the Hazelcast distributed map itself as a streaming source, where an event is created for every change that happens on the map. This allows the map to be used as a source of events during a streaming job. | ||
Kafka Connector
Hazelcast utilizes message brokers for ingesting data streams and it is able to work as a data processor connected to a message broker in the data pipeline. Hazelcast comes with a Kafka connector for reading from and writing to the Kafka topics. | ||
JMS Connector
Java Messaging Services is a traditional means for implementing an enterprise integration. Hazelcast JMS connector allows you to stream messages from/to a JMS queue or a JMS topic using a JMS Client on a classpath (such as ActiveMQ or RabbitMQ). Reading from the queue can be parallelized for higher throughput. | ||
JDBC
Hazelcast JDBC connector can be used to read or write the data from/to relational databases or another source that supports the standard JDBC API. It’s a batch connector that executes a SQL query and sends the result to the Hazelcast pipeline. It supports parallel reading for partitioned sources. | ||
HDFS
Hadoop Distributed File System (HDFS) is a common file system used for building large, low cost data warehouses and data lakes. Hazelcast can use HDFS as either a data source or data sink. If Hazelcast and HDFS clusters are co-located, then Hazelcast benefits from the data locality and processes the data from the same node without incurring network transit latency penalty. | ||
Avro
Hazelcast can read and write Avro-serialized data from the self-contained files (Avro Object Container format), HDFS and Kafka. A Kafka connector can be configured to use the schema registry. | ||
Local Data Files
Hazelcast Hazelcast comes with batch and streaming file readers to process local data (e.g. CSVs or logs). The batch reader processes lines from a file or directory. The streamer watches the file or directory for changes, streaming the new lines to Hazelcast. | ||
Sockets
The socket connector allows Hazelcast jobs to read text data streams from the socket. Every line is processed as one record. | ||
Custom Sources and Sinks
Hazelcast provides a flexible API that makes it easy to implement your own custom sources and sinks. Here are the code samples to be used as a template. | ||
Kafka Connect Modules
Hazelcast supports the use of any Kafka Connect module without the presence of a Kafka cluster. This adds more sources and sinks to the Hazelcast ecosystem. This feature includes full support for fault-tolerance and replaying. | ||
Change Data Capture Integration with Databases
Hazelcast supports change data capture through integration with Debezium and Striim. Debezium provides CDC integration with SQL Server, MySQL, PostgreSQL, MongoDB, and Cassandra. Striim provides CDC integration with Oracle. | ||
CLUSTER MANAGEMENT | ||
Management Center
Management Center enables you to monitor and manage your Hazelcast cluster. In addition to monitoring the overall health of your cluster, you can also analyze the data flow of the distributed pipelines. Management Center provides visual tools to inspect running jobs and detect potential bottlenecks. Crucially, developers can observe clusters in real-time and gain far more insight into what is occurring “under the hood”. | ||
Elasticity
Hazelcast is elastic — it is able to dynamically re-scale to adapt to workload changes. When the cluster extends or shrinks, running jobs can be automatically replanned to make use of all available resources. | ||
CONTINUOUS OPERATIONS | ||
Lossless Cluster Restart
Hazelcast uses persistence to back up its state snapshots regularly. Jobs, Job State, Job Configuration is configured to be persistent with the Hazelcast Persistence capability. Computations are restarted from where they left off after the cluster is online. | ||
Job Upgrade
Allow jobs to be upgraded without data loss or interruption and makes use of state snapshots to switch to new job version in milliseconds. | ||
SECURITY SUITE | ||
SSL/TLS 1.2 Asymmetric Encryption
Provides encryption based on TLS Certificates between members, between clients and members, and between members and Management Center. | ||
SSL/TLS 1.2 Asymmetric Encryption with OpenSSL
Uses the SSLEngine built in to the JDK with some performance enhancements. | ||
Secure Connectors
Connectors are used to connect the Hazelcast job with data sources and sinks. Secure connections to external systems combined with security within the Hazelcast cluster make the data pipeline secure end-to-end. The following connectors do have security features: Hazelcast in-memory data store, Kafka, JDBC, JMS. | ||
Allowed Connection IP Ranges | ||
Authentication
The authentication mechanism for Hazelcast client security works the same as cluster member authentication. To implement client authentication, configure a Credential and one or more LoginModules. The client side does not have and does not need a factory object to create Credentials objects like ICredentialsFactory. Credentials must be created at the client side and sent to the connected node during the connection process. | ||
Authorization
Hazelcast client authorization is configured by a client permission policy. Hazelcast has a default permission policy implementation that uses permission configurations defined in the Hazelcast security configuration. Default policy permission checks are done against instance types (map, queue, etc.), instance names (map, queue, name, etc.), instance actions (put, read, remove, add, etc.), client endpoint addresses, and client principal defined by the Credentials object. Instance and principal names and endpoint addresses can be defined as wildcards(*). | ||
Symmetric Encryption
In symmetric encryption, each node uses the same key, so the key is shared. | ||
JAAS Module
Hazelcast has an extensible, JAAS-based security feature you can use to authenticate both cluster members and clients, and to perform access control checks on client operations. Access control can be done according to endpoint principal and/or endpoint address. | ||
Pluggable Socket Interceptor
Hazelcast allows you to intercept socket connections before a node joins to a cluster or a client connects to a node. This provides the ability to add custom hooks to join and perform connection procedures (like identity checking using Kerberos, etc.). | ||
Security Interceptor
Hazelcast allows you to intercept every remote operation executed by the client. This lets you add a flexible custom security logic. | ||
CLOUD AND VIRTUALIZATION SUPPORT | ||
Cloud Provider
Hazelcast can be extended by cloud plugins, allowing applications to be deployed in different cloud infrastructure ecosystems. See Hazelcast Cloud Plugins: Amazon Web Services, Microsoft Azure, Docker, Apache JClouds, Consul Discovery, Apache ZooKeeper Discovery | ||
OpenShift Container Platform
Hazelcast Docker image is an extension of the official Hazelcast Docker image with a Kubernetes discovery plugin which enables deployment of Hazelcast on your OpenShift platform as a data processing service. |