How to Use Embedded Hazelcast on Kubernetes

Rafal Leszko | Feb 7, 2019

Hazelcast IMDG is a perfect fit for your (micro)services running on Kubernetes since it can be used in the embedded mode and therefore scale in and out together with your service replicas. This blog post presents a step-by-step description of how to embed Hazelcast into a Spring Boot application and deploy it in the Kubernetes cluster. The source code for this example can be found here.

Step 1: Configure Hazelcast to work on Kubernetes

Make sure you have the following Maven dependencies:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-all</artifactId>
    <version>4.1</version>
</dependency>

Then, configure the Kubernetes Discovery Strategy. You can do it in 3 different manners: YAML configuration, XML configuration, or Java-based configuration. Let’s use the first approach and create the file src/resources/hazelcast.yaml:

hazelcast:
  network:
    join:
      kubernetes:
        enabled: true

The equivalent XML configuration would look as follows:

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast  xmlns_xsi="http://www.w3.org/2001/XMLSchema-instance" xsi_schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-4.1.xsd">
  <network>
    <join>
      <kubernetes enabled="true"/>
    </join>
  </network>
</hazelcast>

Note that this configuration will form a Hazelcast with all Hazelcast instances assigned to services in the current namespace. If you want to filter the instances, use the properties described here.

Step 2: Build an application and Docker image

Now, you need to build the application and publish it as a Docker image. First, let’s build the project.

mvn package

Then, we can create the following Dockerfile.

FROM openjdk:8-jre-alpine
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]

Finally, we’re ready to build and push our Docker image.

docker build -t leszko/hazelcast-embedded-sample .
docker push leszko/hazelcast-embedded-sample

Note that you need to change leszko to your Docker Hub account name.

Step 3: Grant access to Kubernetes API

Hazelcast uses Kubernetes API for auto-discovery. That is why you need to grant certain roles to your service account. You can do it by creating the following “rbac.yaml” file.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-cluster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- kind: ServiceAccount
  name: default
  namespace: default

Then, apply it into your Kubernetes cluster.

kubectl apply -f rbac.yaml

Step 4: Deploy the application

Create “deployment.yaml” with Deployment and Service which will use the image you pushed to Docker Hub.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hazelcast-embedded
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hazelcast-embedded
  template:
    metadata:
      labels:
        app: hazelcast-embedded
    spec:
      containers:
        - name: hazelcast-embedded
          image: leszko/hazelcast-embedded-sample
          imagePullPolicy: Always
          ports:
          - containerPort: 5701
          - containerPort: 8080

---

apiVersion: v1
kind: Service
metadata:
  name: hazelcast-embedded
spec:
  type: LoadBalancer
  selector:
    app: hazelcast-embedded
  ports:
  - name: hazelcast
    port: 5701
  - name: app
    port: 8080

Then, to deploy an application, run the following command:

kubectl apply -f deployment.yaml

Step 5: Verify that the application works correctly

You can check that the Deployment and Service were created.

$ kubectl get all
NAME                                      READY     STATUS    RESTARTS   AGE
pod/hazelcast-embedded-57f84c545b-64tnk   1/1       Running   0          2m
pod/hazelcast-embedded-57f84c545b-jjhcs   1/1       Running   0          45s

NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                         AGE
service/hazelcast-embedded   LoadBalancer   10.19.251.145   104.154.43.142   5701:32302/TCP,8080:31613/TCP   2m

NAME                                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/hazelcast-embedded   2         2         2            2           2m

NAME                                                  DESIRED   CURRENT   READY     AGE
replicaset.extensions/hazelcast-embedded-57f84c545b   2         2         2         2m

NAME                                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/hazelcast-embedded   2         2         2            2           2m

NAME                                            DESIRED   CURRENT   READY     AGE
replicaset.apps/hazelcast-embedded-57f84c545b   2         2         2         2m

In the logs for PODs, you should see that the Hazelcast members formed a cluster.

$ kubectl logs pod/hazelcast-embedded-57f84c545b-jjhcs
 ...
 Members {size:2, ver:4} [
         Member [10.16.2.6]:5701 - 33076b61-e99d-46f2-b5c1-35e0e75f2311
         Member [10.16.2.8]:5701 - 9ba9bb61-6e34-460a-9208-c5a644490107 this
 ]
 ...

Then, you can access the application, by its EXTERNAL-IP.

That’s it! If you would like to learn more about scaling Hazelcast IMDG on Kubernetes, here is another post with step-by-step instructions.

Relevant Resources

Guide
| PDF
| 112 pages

Hazelcast IMDG Deployment and Operations Guide

The Hazelcast IMDG® Deployment and Operations Guide provides an introduction to the most important aspects of deploying and operating a successful Hazelcast® installation. This edition is current with Hazelcast IMDG version 4.0. In this guide, you’ll learn about different approaches to topologies, advantages and disadvantage of various types of architecture and how to configure Hazelcast for optimal success. You’ll learn how to plan for lifecycle events to ensure high uptime and smooth operation. This guide also covers Hazelcast-specific optimization considerations to keep in mind when preparing for a new Hazelcast deployment. You’ll learn how to determine cluster size and how to authenticate cluster members and clients, as well as how to deploy and scale a Hazelcast cluster while ensuring failure detection and recovery. Moreover, you’ll learn about license management and how to report issues to Hazelcast. Download the guide to get started now.
View All Resources
About the Author

Rafal Leszko

Cloud Software Engineer

Rafał is a passionate software engineer, trainer, conference speaker, and author of the book, Continuous Delivery with Docker and Jenkins. He specializes in Java development, cloud environments, and continuous delivery. Prior to joining Hazelcast, Rafał worked with a variety of companies and scientific organizations, including Google, CERN, and AGH University of Science and Technology.