Dynamic Configuration: No Restarts Required

Hazelcast Platform is known for its high-performance data caching and integration capabilities and provides a robust and scalable framework for distributed applications. The platform offers numerous ways to configure clusters—from XML or YAML files to environment variables, system properties, and programmatic APIs.

But what if your configuration needs to evolve while your Hazelcast Platform cluster is live? Enter dynamic configuration. It enables you to add new data structure configurations at runtime without restarting the cluster, ensuring uninterrupted service even in production environments.

Why dynamic configuration?

Traditional methods of reconfiguring a cluster often necessitate downtime—a costly and disruptive process, especially in production. Dynamic configuration solves this by enabling the following:

• Runtime updates to configuration, such as adding new data structures
• Persistence of changes, ensuring they survive cluster restarts
• A flexible, programmatic API for seamless updates

Dynamic configuration* is similar to a static Hazelcast Platform configuration and can be called over similar interfaces. You can append your new data structure configuration to the config file or call programmatic APIs to set the new configuration. Dynamically added configuration can also be persisted, even when the cluster is restarted or shut down.

Hazelcast Platform documentation has more information on dynamic configuration.

Use case: Configure a new distributed map using the API

In the following scenario, you’re running a Hazelcast Platform cluster and must introduce a new map data structure with custom configurations requiring disk persistence. Restarting the cluster to apply these changes isn’t an option in production, so dynamic configuration is employed.

Connect to the cluster

Start by initializing a Java client to connect to your cluster.

If you use a client other than Java, check the feature list to ensure your client supports dynamic configuration. 

ClientConfig clientConfig = new ClientConfig();
clientConfig.setClusterName("dev");
clientConfig.getNetworkConfig().addAddress("127.0.0.1:5701");
HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig );

So far, the config is similar to a static one. You can interact with the cluster using the client.

IMap<String, String> myFirstMap = client.getMap("myFirstMap");
myFirstMap.put("myKey", "myValue");

“myFirstMap” is initialized with default map configuration values since no configuration was applied when starting the cluster, as per the initial requirements.

However, we now need a map structure that persists data to the disk. This will require a new map configuration because data persistence is not enabled by default. Since this is a production environment, dynamic configuration can be used to apply the new configuration and avoid restarting the cluster.

Create a new configuration with persistence enabled

Next, you will create a new configuration with persistence enabled and set the new map configuration to the cluster via the client.

// Let's create the map config
MapConfig mapConfig = new MapConfig();
mapConfig.setName("my-persistent-map")
         .setDataPersistenceConfig(new DataPersistenceConfig().setEnabled(true));
// Then, set the new config over the client
client.getConfig().addMapConfig(mapConfig);

Now, the “my-persistent-map” map will persist to the disk and can be used immediately without restarting the cluster.

The programmatic APIs were used in the above scenario since there were no constraints on deploying Java code. However, if introducing new code is a constraint, we need another method of using dynamic configuration.

Use case: Configure a new map using a config file

If you can’t modify the cluster with the programmatic APIs, you can update your configuration file and dynamically reload it.

In this scenario, the new map configuration is appended to the static config file, typically `hazelcast.yaml` or hazelcast.xml. In our example, it’s the `hazelcast.xml` file.

<hazelcast>
...
<map name="my-persistent-map">
    <data-persistence enabled="true">
        <fsync>false</fsync>
    </data-persistence>
</map>
</hazelcast>

The cluster is now ready to consume the new map configuration.  

Reload the updated configuration via the Hazelcast Management Center or REST API, avoiding downtime entirely.

Tip! Management Center also accepts a configuration string to propagate to the cluster. If you can’t edit the config file, use this feature instead. 

Also, our documentation has more information on configuring dynamic configuration over REST. 

Handling split-brain scenarios

Split-brain syndrome, or network partitioning, occurs in distributed computing systems when a network failure causes the system to split into two or more isolated groups of nodes (or servers). Unaware of the other, each group may attempt to operate independently, often leading to inconsistencies, duplicate actions, or conflicts (more information).

If you add a new configuration during such an occurrence, the other parts of the cluster will be unaware of it. However, during recovery, the merging side adopts the dynamically added configuration from the initiating segment. The merging ensures consistency once the cluster is whole again.

Looking ahead

Dynamic configuration supports adding new data structures but not modifying existing ones at runtime. Hazelcast is constantly expanding its dynamic configuration capabilities and is considering developing the following new features to apply changes at runtime on an existing distributed map. In addition to persisting dynamic config changes, the new features would dynamically update:

• Map size limits — set size limits on maps to prevent them from consuming too much memory 
• Max size policy — determine how the maximum size of a map or Near Cache is measured and enforced
• Eviction policy — determine which entries should be removed from a map or cache when it reaches its maximum size limit
• Time to Live (TTL) — limit the lifetime of entries in a map
• Read from backup (read-backup-data) — allow local members to read map data from their local backups when using Hazelcast in embedded mode
• Max idle — limit the lifetime of entries in a map based on their last access time
• WAN replication references — enable WAN replication for specific data structures like maps and caches

You can explore Hazelcast Platform’s dynamic configuration capabilities in the Community Edition available here.

Join our Community Slack channel to shape future features and stay connected with Hazelcast.