Operator vs Helm: Why Do We Need an Operator?

Introduction

The popularity of Kubernetes makes it one of the most widely used platforms for deploying applications. And there are many tools to simplify Kubernetes deployments. But, two deployment solutions stand out among others: Helm and Operator.

Why do we need an Operator?

Hazelcast’s platform is cloud-native, and for this purpose, we have a well-supported Helm chart that allows for easy installation and configuration of the Hazelcast Platform and Management Center. The Hazelcast Platform is a powerful product, but with great features brings some complexity. Therefore, we aimed to make the users’ cloud-native experience with Hazelcast as smooth as possible. With the Helm chart, it is pretty straightforward to install the Hazelcast cluster, but more hands-on work may be required when we want to use some of the Platform’s advanced features. So, instead of trying to script Helm to do what it’s not supposed to, the decision was to write a Kubernetes Operator.

One of the decisions taken for the operator development was to give an abstraction over the Hazelcast Platform configuration. It means one can’t provide hazelcast.yaml to configure the Hazelcast cluster. Instead, we deliver a smooth API that focuses on features rather than the powerful yet complex Hazelcast configuration.

At the moment, Hazelcast Platform Operator gives us full control of the application and features lifecycle, which makes it a Level III operator. But besides that, having an operator makes it easier to grow and add additional features such as metrics or scaling in the future.

The problems that the operator solves

The table below represents a summary of the Operator and Helm Capabilities that will be described in detail further down.

Custom Resource Definition (CRD)

With the operator, we can make the Hazelcast Platform truly Kubernetes-native with the help of CRD. With this, we can create a custom resource called Hazelcast and manage it as any other Kubernetes resource.

apiVersion: hazelcast.com/v1alpha1 
kind: Hazelcast 
metadata:   
  name: my-hazelcast

Helm and Operator allow using opinionated configuration so that the user doesn’t need to configure anything to install a basic cluster. However, with the Operator, we can decide on the runtime and what configuration is better to apply. For example, we use a different SecurityContextconfiguration for the StatefulSet if the cluster runs on the OpenShift platform. Using the Helm chart, the user must manually configure the SecurityContext for a specific platform.

And, of course, Hazelcast CR would not be truly Kubernetes-native if you could not observe its status.

$ kubectl get hazelcast my-hazelcast 
NAME         STATUS  MEMBERS EXTERNAL-ADDRESSES 
my-hazelcast Running 3/3

Dynamic configuration

One might not use some Hazelcast features from the beginning but can decide to enable them in the future. Let’s use WAN Replication as an example of a feature we want to add. It can be configured and enabled with the Helm chat. However, such a change will need to restart the cluster. On the other hand, the Operator can update the configuration without the cluster restart simply by creating the WanReplication CR.

apiVersion: hazelcast.com/v1alpha1
kind: WanReplication 
metadata:   
  name: wanreplication-sample 
spec:   
  mapResourceName: my-map   
  targetClusterName: dev   
  endpoints: "203.0.113.61"

API

The crucial part of every product is its API. With the operator, we want to ease the usage of the Hazelcast Platform. And for a great experience, we need to provide a great API that will abstract all the complexity and be intuitive for the user. One example is connecting to Hazelcast running on Kubernetes from outside with a smart client. User needs to create the service per pod manually, making scaling more difficult, or use a third-party tool such as metacontroller. With the Operator, users can declaratively specify the exposeExternally configuration in the Hazelcast CRD.

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  licenseKeySecret: hazelcast-license-key
  exposeExternally:
    type: Smart
    discoveryServiceType: LoadBalancer
    memberAccess: NodePortExternalIP

And the operator will take care of creating the service per pod.

Feature lifecycle

Hazelcast Platform is rich in features. However, these features can be advanced and may require multiple manual steps. For example, let’s say the user wants to create a Backup of the cluster and copy it to external storage. Helm chart will allow you to mount the volume for the members, but since Helm does not manage the application lifecycle, you will have to trigger the Backup process manually using any available API. After, you must copy the Backup to the external storage of choice. The operator simplifies these operations so that all you will need is to enable persistence and apply the HotBackup CRD.

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  licenseKeySecret: hazelcast-license-key
  persistence:
    baseDir: "/data/hot-restart/"

apiVersion: hazelcast.com/v1alpha1
kind: HotBackup
metadata:
  name: hot-backup
spec:
  hazelcastResourceName: my-hazelcast
  bucketURI: "s3://operator-backup"
  secret: "br-secret-s3"

Moreover, the Backup process can be Scheduled, allowing it to run periodically without any interventions.

Conclusions

We have seen how users can benefit from the Operator and how it can simplify the usage of the Hazelcast Platform. However, this doesn’t mean that the Helm chart is no longer needed; it can serve a different purpose. The Hazelcast Helm chart is the preferred option among experienced Hazelcast options as it provides the user with complete control over the Hazelcast configurations. But, for the users that prefer simplicity, automation, and a Kubernetes-native experience, the Operator is the way to go. It might seem that the Hazelcast Operator supports fewer Hazelcast Platform features than the Helm chart; however, it is still in active development. In the near future, we plan to achieve feature parity so you can choose based on your needs, not on the supported features. Don’t hesitate to share your experience with us in our community Slack or Github repository.