Getting Started With the Hazelcast Python Client
The Hazelcast Python Client lets developers connect to Hazelcast clusters from Python apps and services. We can use the client to handle all of Hazelcast’s distributed data structures. We can also subscribe to cluster and data structure events.
In this tutorial, we’ll explore how to install the client and use some of its key features.
Installing the Hazelcast Python Client
To get started with the Hazelcast Python Client, we need to ensure there’s a Hazelcast cluster already running. Then, we download the open-source Python client from PyPI.
To quickly start a single-member cluster for this tutorial, we’ll use a Docker image. We run this command on the terminal:
docker run -p 5701:5701 hazelcast/hazelcast:5.0
This command quickly deploys a local Hazelcast instance. You can use the Python Client on any Hazelcast cluster whether it is running locally or remotely.
Alternatively, you can download a Hazelcast version that suits you (in ZIP or TAR format).
Next, we install the Hazelcast Python Client. If you’re a Python developer familiar with PyPI, just go to your terminal and use pip install to install the Hazelcast Python Client.
pip install hazelcast-python-client
Note: This command works for the latest installed Python version (2 or 3). If you are running Python version 3, use pip3 instead of pip.
Then, we create or open a Python file and import Hazelcast:
import hazelcast
Next, we enable logging for our Hazelcast Python Client:
import logging
# to configure the logging
logging.basicConfig(level=logging.INFO)
Now that we’ve finished enabling logging, our next steps are configuring our client and connecting it to a Hazelcast cluster.
# Create a client using the configuration below
client = hazelcast.HazelcastClient(
# For public IPs, add a colon and the port number after the address.
# Your Hazelcast cluster members should have at least one member address, so you should add its host:port pair to the configuration.
# If the port is not specified, by default 5701, 5702 and 5703 will be tried.
cluster_members=[
"127.0.0.1:5701",
"127.0.0.1:5702",
]
)
# Disconnect the client and release its resources
client.shutdown()
Exploring Hazelcast Python Client’s Features
Now, let’s explore some exciting features of the Hazelcast Python Client.
First, we can use the Python client to add username and password authentication to secure the connection to our Hazelcast cluster.
# Start a new Hazelcast client with the given credentials.
client = hazelcast.HazelcastClient()
When working with Hazelcast distributed data structures, by default the Hazelcast client returns an asynchronous Future. However, when working with a data structure you may want all of your calls to be blocking — that is, you want to wait until the Hazelcast operation on the data structure completes before executing the next line of code. You can accomplish this in a couple of ways.
You can call the blocking() method when creating a reference to the data structure:
# call blocking() to ensure all calls to hz_map will be synchronous
hz_map = client.get_map("auth-map").blocking()
hz_map.set("key", "value")
print(hz_map.get("key"))
Or, you can wait for the future to complete before you proceed:
hz_map = client.get_map("auth-map")
# wait until the value is added before proceeding
hz_map.set("key", "value").wait()
# If using Python 3.5 or above, instead of calling .wait() you can also use await:
# await hz_map.set("key", "value")
# if you don't need to wait for completion before proceeding, don't wait for future completion
hz_map.set("key2", "don't need to wait for this")
print(hz_map.get("key"))
Regardless of how you use Hazelcast distributed objects, it’s good practice to shut down the client when you’re done using it:
client.shutdown()
Monitoring Cluster Events
One of Hazelcast’s cluster listener’s, the membership listener, can generate the following events:
- member_added fires when a cluster gains a new member
- memberRemoved fires when an existing member leaves the cluster
To implement this using the Python client, we write the following code:
import hazelcast
import time
def member_added(member):
print("Member added: {}".format(member))
def member_removed(member):
print("Member removed: {}".format(member))
client = hazelcast.HazelcastClient()
client.cluster_service.add_listener(member_added, member_removed, True)
# Add/Remove member now to see the listeners in action
time.sleep(100)
client.shutdown()
Managing Supported Distributed Data Structures
Hazelcast provides plenty of distributed data structure implementations for the Python client. Let’s explore some of them. We’ll try out some data structures available to the Python client.
Using Map in the Python Client
The Hazelcast map is a distributed dictionary that accesses the map on the cluster. We can read from and write to it with methods such as get and set.
Note that we can also use the put method to add a method to a map. The difference between set and put is that if you’re overwriting a value already in the map, put sets the new value and returns the old value, while set just overwrites the old value and doesn’t return anything. set is faster, so use it unless you’re overwriting a key’s value and need the old value.
# Get the Distributed Map from cluster.
my_map = client.get_map("my-distributed-map").blocking()
# Standard Set and Get
my_map.set("key", "value")
get = my_map.get("key")
# Concurrent Map methods, optimistic updating lets us proceed without performing separate checks of current values in the map
my_map.put_if_absent("somekey", "somevalue")
my_map.replace_if_same("key", "value", "newvalue")
Using the Multimap Client
MultiMap is a special type of Hazelcast map. It’s a distributed data structure that allows us to store multiple values for a single key. Below is a MultiMap usage example.
# Get the Distributed MultiMap from cluster.
multi_map = client.get_multi_map("my-distributed-multimap").blocking()
# Put values in the map with the same key
multi_map.put("my-key", "value1")
multi_map.put("my-key", "value2")
multi_map.put("my-key", "value3")
# Print out all the values for associated with key called "my-key"
values = multi_map.get("my-key")
print(values)
# remove specific key/value pair
multi_map.remove("my-key", "value2")
Using Queue in the Python Client
The Hazelcast distributed queue is an observable, concurrent, and blocking queue. You can use the queue to add an item to one cluster member and remove it from another cluster member.
# Retrieve a blocking queue named distributed-queue from Hazelcast and assign it to the queue variable
queue = client.get_queue("distributed-queue").blocking()
# Appends a string to the distributed queue, if there's available space
queue.offer("item")
# Poll the distributed queue and return the string
item = queue.poll()
# Timed blocking operations
queue.offer("anotheritem", 0.5)
another_item = queue.poll(5)
# Indefinitely blocking operations
queue.put("yetanotheritem")
print(queue.take())
Using Set in the Python Client
A Hazelcast Set is concurrent and distributed. It is an unordered collection of items that contains no duplicates. The code sample below shows how to add and get items in a Set.
# Get the distributed Set from the cluster.
my_set = client.get_set("my-distributed-set").blocking()
# Add items to the set with duplicates
my_set.add("item1")
my_set.add("item1")
my_set.add("item2")
my_set.add("item2")
my_set.add("item2")
my_set.add("item3")
# Get the items. Note that there are no duplicates.
for item in my_set.get_all():
print(item)
Using List in the Python Client
A Hazelcast List permits duplicate elements while maintaining their order. It’s similar to a Hazelcast Set. Below is an example List implementation using the Python client.
# Get the distributed List from the cluster.
my_list = client.get_list("my-distributed-list").blocking()
# Add element to the list
my_list.add("item1")
my_list.add("item2")
# Remove the first element
print("Removed:", my_list.remove_at(0))
# There is only one element left
print("Current size is", my_list.size())
# Clear the list
my_list.clear()
Using the Ringbuffer in the Python Client
We can use RingBuffer to store and process a series of items or events reliably. Unlike a queue, taking an item from a RingBuffer is non-destructive. So, if your item processing code fails, you can retrieve it from the RingBuffer again. Note, however, that you can’t remove items from a RingBuffer once added, so you can’t use it to process a continuous stream of events like you might with a queue.
An example of this distributed data structure is below.
rb = client.get_ringbuffer("rb").blocking()
# add two items into ring buffer
rb.add(100)
rb.add(200)
# We start from the oldest item.
# if you want to start from the next item, call rb.tail_sequence()+1
sequence = rb.head_sequence()
print(rb.read_one(sequence))
sequence += 1
print(rb.read_one(sequence))
Creating a Flake ID Generator
We can also create a flake ID generator using the Python client. We use a flake ID when we need to generate an identifier that is unique across the entire Hazelcast cluster.
client = hazelcast.HazelcastClient(
flake_id_generators={
"id-generator": {
"prefetch_count": 50,
"prefetch_validity": 30,
}
}
)
generator = client.get_flake_id_generator("id-generator").blocking()
for _ in range(100):
print("Id:", generator.new_id())
client.shutdown()
Concurrency Primitives
Now let’s explore concurrency primitives. These primitives provide building blocks for running asynchronous and synchronous tasks on distributed data.
Using AtomicLong in the Python Client
AtomicLong helps individually update long values. Below is an example of how we use it.
# Get the AtomicLong counter from Cluster
counter = client.cp_subsystem.get_atomic_long("counter").blocking()
# Add and get the counter
value = counter.add_and_get(3)
print("Counter value is", value)
Using FencedLock in the Python Client
FencedLock helps prevent multiple processes from accessing an object at the same time and corrupting data. It sacrifices availability to gain consistency. It comes from the CP Subsystem, which is a cluster component that builds a consistent layer for data structures.
# Get the Distributed Lock from CP Subsystem
lock = client.cp_subsystem.get_lock("my-distributed-lock").blocking()
# Now acquire the lock and execute some guarded code
fence = lock.lock()
print("Fence token:", fence)
try:
# do something here
pass
finally:
lock.unlock()
Next Steps
These are just a few code samples showing how you can use the Hazelcast Python Client. We didn’t explore several other features here. Now that you know what the Hazelcast Python Client can do, try it out for yourself and share your work with the community. Check out the Hazelcast Python Client library homepage for more information about the features we discussed here and to learn what else you can do with Hazelcast.
