How to Use Embedded Hazelcast on Kubernetes

Hazelcast IMDG is a perfect fit for your (micro)services running on Kubernetes since it can be used in the embedded mode and therefore scale in and out together with your service replicas. This blog post presents a step-by-step description of how to embed Hazelcast into a Spring Boot application and deploy it in the Kubernetes cluster. The source code for this example can be found here.

Step 1: Configure Hazelcast to work on Kubernetes

Add the following Maven dependencies:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast</artifactId>
    <version>3.11</version>
</dependency>
<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-kubernetes</artifactId>
    <version>1.3.1</version>
</dependency>

Then, configure the Kubernetes Discovery Strategy. You can do it in two different manners: Java-based configuration or XML configuration. In the code sample, we used the first approach:

public Config hazelcastConfig() {
    Config config = new Config();
    JoinConfig joinConfig = config.getNetworkConfig().getJoin();
    joinConfig.getMulticastConfig().setEnabled(false);
    joinConfig.getKubernetesConfig().setEnabled(true);
    return config;
}

The equivalent XML configuration would look as follows:

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.11.xsd">
  <network>
    <join>
      <multicast enabled="false"/>
      <kubernetes enabled="true"/>
    </join>
  </network>
</hazelcast>

Note that this configuration will form a Hazelcast with all Hazelcast instances assigned to services in the current namespace. If you want to filter the instances, use the properties as described here.

Step 2: Build an application and Docker image

Now, you need to build the application and publish it as a Docker image. You can use the standard “docker build” and “docker push” commands, however, if you follow the source code mentioned at the beginning, you may just execute the following command which compiles the project, builds the Docker image, and pushes it into your Docker Hub account.

mvn clean compile jib:build -Dimage=leszko/hazelcast-embedded-sample

Step 3: Grant access to Kubernetes API

Hazelcast uses Kubernetes API for the auto-discovery. That is why you need to grant certain roles to your service account. You can do it by creating the following “rbac.yaml” file.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-cluster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- kind: ServiceAccount
  name: default
  namespace: default

Then, apply it into your Kubernetes cluster.

kubectl apply -f rbac.yaml

Step 4: Deploy the application

Create “deployment.yaml” with Deployment and Service which will use the image you pushed to Docker Hub.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hazelcast-embedded
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hazelcast-embedded
  template:
    metadata:
      labels:
        app: hazelcast-embedded
    spec:
      containers:
        - name: hazelcast-embedded
          image: leszko/hazelcast-embedded-sample
          imagePullPolicy: Always
          ports:
          - containerPort: 5701
          - containerPort: 8080

---

apiVersion: v1
kind: Service
metadata:
  name: hazelcast-embedded
spec:
  type: LoadBalancer
  selector:
    app: hazelcast-embedded
  ports:
  - name: hazelcast
    port: 5701
  - name: app
    port: 8080

Then, to deploy an application, run the following command:

kubectl apply -f deployment.yaml

Step 5: Verify that the application works correctly

You can check that the Deployment and Service were created.

$ kubectl get all
NAME                                      READY     STATUS    RESTARTS   AGE
pod/hazelcast-embedded-57f84c545b-64tnk   1/1       Running   0          2m
pod/hazelcast-embedded-57f84c545b-jjhcs   1/1       Running   0          45s

NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                         AGE
service/hazelcast-embedded   LoadBalancer   10.19.251.145   104.154.43.142   5701:32302/TCP,8080:31613/TCP   2m

NAME                                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/hazelcast-embedded   2         2         2            2           2m

NAME                                                  DESIRED   CURRENT   READY     AGE
replicaset.extensions/hazelcast-embedded-57f84c545b   2         2         2         2m

NAME                                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/hazelcast-embedded   2         2         2            2           2m

NAME                                            DESIRED   CURRENT   READY     AGE
replicaset.apps/hazelcast-embedded-57f84c545b   2         2         2         2m

In the logs for PODs, you should see that the Hazelcast members formed a cluster.

$ kubectl logs pod/hazelcast-embedded-57f84c545b-jjhcs
 ...
 Members {size:2, ver:4} [
         Member [10.16.2.6]:5701 - 33076b61-e99d-46f2-b5c1-35e0e75f2311
         Member [10.16.2.8]:5701 - 9ba9bb61-6e34-460a-9208-c5a644490107 this
 ]
 ...

Then, you can access the application, by its EXTERNAL-IP.

That’s it! If you would like to learn more about scaling Hazelcast IMDG on Kubernetes, here is another post with step-by-step instructions.