Companies need a data-processing solution that increases the speed of business agility, not one that is complicated by too many technology requirements. This requires a system that delivers continuous/real-time data-processing capabilities for the new business reality.
Stream processing is a hot topic right now, especially for any organization looking to provide insights faster. But what does it mean for users of Java applications, microservices, and in-memory computing?
In this webinar, we will cover the evolution of stream processing and in-memory related to big data technologies and why it is the logical next step for in-memory processing projects.
Setting up servers and configuring software can get in the way of the problems you are trying to solve. With Hazelcast Cloud we take all of those pain points away.
Watch this webinar to learn how you can instantly fire up and then work with Hazelcast Cloud from anywhere in the world. With our auto-generated client stubs for Java, Go, Node.js, Python and .NET, we can have you connected and coding in less than a minute!
Redis is good for simple data caching use cases, but in the real world, you will want more than simple caching from your in-memory computing platform. When you’ve hit a wall at scaling Redis or struggled to work around its security issues, it’s time to turn to Hazelcast. The Hazelcast In-Memory Computing Platform joins Hazelcast IMDG and Hazelcast Jet to enable not only a high-performance cache, but also stream and batch processing at microsecond speeds.
There are a list of reasons to switch from Redis to Hazelcast. Repeatable benchmarks show that Hazelcast is many times faster. Redis is single-threaded, so it does not efficiently scale for larger loads, while Hazelcast performance scales linearly with additional resources. Hazelcast is easy to use, and it can be embedded in apps or deployed in a client-server model. Hazelcast is extremely secure, and the security overhead only has a negligible effect on performance.
Key architectural differences between Hazelcast IMDG and Redis
Language. Hazelcast IMDG is written in Java, while Redis is implemented in C.
Threading. Redis is single-threaded. Hazelcast IMDG can benefit from all available CPU cores as it scales.
Design. Hazelcast IMDG is designed from the ground up for a distributed environment; Redis was initially intended to be used in standalone mode. Clustered Redis does not support SELECT commands, meaning it only supports a single database (namespace) per cluster. In contrast, Hazelcast IMDG supports an unlimited number of maps and caches per cluster.
High availability. More on this in the failover section.
To reduce DevOps efforts and simplify deployment, Hazelcast leverages multicast discovery, where members use multicast UDP transmission to get to know each other automatically without any additional configuration. You don’t need to install software on every resource you add or manually rebalance your cluster.
It is possible to manually specify member addresses through TCP. You simply need to provide just one working address, regardless of whether your cluster consists of three or thirty nodes.
For cloud deployments, Hazelcast IMDG supports automatic discovery of its member instances on Amazon EC2 and Google Compute Engine.
Hazelcast IMDG provides a Discovery Service Provider Interface (SPI), which allows users to implement custom member discovery mechanisms to deploy Hazelcast IMDG on any platform. Hazelcast Discovery SPI allows you to use third-party software like Zookeeper, Eureka, Consul, and more to implement a custom discovery mechanism.
A Redis client connects to a Redis cluster through a TCP connection. There is no provision for discovering Redis servers on a multicast UDP network.
Redis does not provide an automatic discovery mechanism for any cloud provider, which makes it difficult to use in custom cloud deployments.
Establishing a Redis cluster is not straightforward. A developer needs to launch a special utility script while specifying all member addresses. This utility script (“redis-trib.rb”) is written in Ruby and thus requires additional Ruby runtime dependencies. (The script is included in Redis distributions.)
Adding a node to a Hazelcast IMDG cluster is incredibly easy — just launching the node with the proper configuration does the job. Removing a node is also remarkably simple — just shutdown the node and the data is recovered on other nodes from backups.
In both cases, Hazelcast IMDG automatically rebalances the data across all available server nodes without affecting the availability of the data.
On the other hand, adding a node to a Redis cluster is not convenient. The user has to use the utility script to manually introduce a new member and then re-partition the cluster with the same script. Removal in Redis is similar, but the order of operations is different — repartition, make the cluster “forget” about the node, and only then shutdown.
Benchmarks show that the performance of Hazelcast IMDG outshines that of Redis. Hazelcast publishes benchmark test suites so that you can verify the results independently. Hazelcast aims to run fair benchmarks to make a valid comparison. In several cases, benchmarks run by other vendors have shown superior performance by their products because of unequal configurations that favor the other vendor.
One important point about Hazelcast performance is that superior speed is attained without needing to sacrifice other areas. For example, data safety features are kept intact in Hazelcast under high loads. On the other hand, Redis performs well at high loads because the system turns off replication to accommodate the load. This results in a system that is vulnerable to data loss should a master node fail, which is not a reasonable tradeoff for production systems. Details on this topic are covered in this blog post.
More benchmark information on Hazelcast vs. Redis is available here.
When it comes to providing fault tolerance, Redis uses a traditional master-slave approach, where each primary site for the data is backed up by one or more slaves that are promoted if the master fails.
In a Hazelcast IMDG cluster, all members are equal and hold primary data as well as a backup of the primary data from other members in the cluster. Thus, Hazelcast IMDG requires fewer physical machines to establish the same level of resilience. Also, Redis slave nodes do not process read requests, which reduces the throughput of the entire system.
High availability for Redis is provided by Redis Sentinel, which continuously monitors the Redis cluster and initiates a slave promotion if it detects a master failure. At least three instances of Sentinel are recommended for a reliable deployment, which unnecessarily uses resources.
With Hazelcast IMDG, member failures are detected by peers in the cluster, and recovery of lost data from backups is initiated automatically and immediately on remaining members in the cluster.
Redis supports two modes of file system persistence — append-only file (AOF) and snapshotting (RDB):
Hazelcast IMDG also supports file system persistence via the Hot Restart feature as well as abstractions for persisting to designated data stores such as SQL databases.
With Hazelcast IMDG, data can be backed up by a database in two distinct ways: database first (write-through), or cache first (write-behind).
If a user requests a non-existent entry from the map, Hazelcast IMDG is able to look up missing data from the database and, if found, automatically save it for later use in the map.
Hazelcast IMDG has officially supported clients for Java, Scala, C++, C#/.NET, Python, and Node.js. Redis has community-developed clients for most mainstream languages.
Hazelcast IMDG also includes official support for the memcached protocol.
Fetching data from the server to the client and then extracting metadata on the client-side takes a great deal of time and uses excessive memory in the client. It is better to send extraction logic to the servers that store the actual data and let them work in parallel, with less data exchanged over the wire. This can be done with both Hazelcast IMDG and Redis.
Redis supports the evaluation of Lua scripts that can invoke pretty much any Redis operation. In clustered mode, the Lua scripts have a limitation — if the script is going to use multiple keys, all those keys must belong to the same hash slot (partition).
Hazelcast IMDG has a distributed execution service, which is a special implementation of java.util.concurrent.ExecutorService that allows one to distribute computation tasks written in Java among several physical machines. Depending on the use case, Hazelcast can route the tasks to cluster nodes a few different ways: randomly, based on address, or based on key ownership.
Hazelcast IMDG also supports efficient bulk updates of map data with an abstraction called EntryProcessor. EntryProcessor can be executed on all keys, specific keys supplied by the user, or on keys that match specific criteria.
Hazelcast IMDG supports the predicate API for locating map entries by value attributes.
Redis does not support querying directly, but makeshift solutions can be done using sorted sets.
Hazelcast has performed a number of performance benchmarks against Redis over multiple software versions, showing that in fair, comparably configured setups, Hazelcast outperforms Redis, especially at scale.
This white paper provides a point by point comparison of Hazelcast IMDG and Redis across a number of dimensions.
Background This performance benchmark compares Redis Open Source 5.0.3 vs Hazelcast IMDG Enterprise 3.12. The purpose is to reproduce the low thread count benchmark conducted by Redis Labs in 2018, correcting for deficiencies in the Redis benchmark. The things we changed were: We used Java clients for both. The original benchmark used C++ with Redis. […]
Whether you're interested in learning the basics of in-memory systems, or you're looking for advanced, real-world production examples and best practices, we've got you covered.