Introduction to Hazelcast HD Memory

Background

One of the most common problems that organizations face today is dealing with their ever evolving data (often to referred to as the problem of the 4 ‘V’s. See graphic below). This data is exploding at lightning speed and comes from a variety of sources. The true challenge is to build an infrastructure that enables you to extract value from the enormous amount of data available to you. There is a time-value component for most data, especially business data – the longer it takes to distill value from the data, the less it is worth.

Hazelcast provides the means and tools to build an infrastructure that reduces the time-value lost in the processing of your of data, by addressing a number of the barriers that are introducing latency into the processing of your data. The largest barrier is the lack of high-speed access to the data in order to enable faster processing. Accessing data on a disk-bound data store is a painstakingly slow and sluggish process as compared to retrieving data from a memory-bound data store.

In this post, I’ll focus on how Hazelcast enables you to store TBs of data in the memory of Hazelcast servers–saving you from significant investments that are otherwise required to build monolithic caching layers in your system.

Standard impediments of Caching

Every caching solution that runs on a Java Virtual Machine (JVM) has to deal with the perilous problems of garbage collection (GC) caused by storing data in the memory. Modern hardware has much more available memory. If you want to make use of that hardware to scale up by specifying larger heap sizes, GC becomes an increasing problem: the application faces long GC pauses that make the application performance unresponsive and unpredictable. Simply put, the problem of GC pauses gets more serious as the size of heap increases: the larger the heap, the longer it takes JVM to finish stop-the-world full GC cycle.

This brings us to another very important problem i.e. scaling up and waste of resources. GC, which is an automatic process that manages application’s runtime memory, often forces you into configurations where multiple JVMs with smaller heaps run on lesser physical hardware devices to avoid long GC pauses. This approach forces you to maintain a much bigger cluster than you would ever want. It can be an operational nightmare to manage a cluster of a few hundred nodes.

Lets look at one example. Imagine you have 64 GB of data to cache. For the two approaches:

1. Small heap but many nodes: if you decide to run cache servers with more manageable heap size of 4GB – 5GB, you will end up with a cluster of ~16 nodes
2. Large heap: if you run cache servers with large heap, for example 64 GB, you will run into very long GC pauses and the cluster is likely to fail on performance

High Density Memory Store

High Density Memory Store is part of the Hazelcast Enterprise HD offering. It solves garbage collection limitations so that applications can exploit hardware memory more efficiently without the need of oversized clusters. It is designed as a pluggable memory manager which enables multiple memory stores for different data structures including IMap (Map) and JCache. And with this, it addresses all the challenges of caching large volumes of data in memory.

High-Density (HD) Memory allows you to store all the important data where it is used: in the memory where the application runs and brings full spectrum of in-memory data management capabilities that are essential to perform at scale. Capacity of in-memory data store is only limited by the hardware provisioned as Hazelcast can utilize TBs of RAM in modern day hardware to store your high-value data in memory.

One of the most important features of HD Memory is that it does not require any change in application code to enable or use HD Memory. All the settings are configurable and HD Memory can be activated with minimal changes on servers.

Use Case

Going back to the same example that we saw earlier, HD Memory Store will allow you to run your cache server with significantly less amount of heap to mitigate GC issues and utilize all the RAM available on the machine. As the picture below shows, you will need to run a Hazelcast HD server instance with standard heap size of 2 – 3GB and use rest of the RAM in the system as HD Memory Store to cache all your high value data.

The High-Density Memory Store is built around a pluggable memory manager which enables multiple memory stores. These memory stores are all accessible using a common access layer that scales up to very large amount of main memory on a single JVM. At the same time, by further minimizing the GC pressure, HD Memory Store enables predictable application scaling and boosts performance and latency while minimizing pauses for Java Garbage Collection.

Configuring HD Memory

HD Memory configuration is map specific. You may choose to activate HD Memory for some maps and leave the others to store data on normal heap.

Configuring HD Memory is a highly straightforward and simple task. Let’s look at an example:

To activate HD Memory, you need to configure settings in hazelcast.xml. Note that this is a cluster wide configuration, so it must remain outside of any particular Map’s configuration.

See below if you use Hazelcast Java APIs to configure the cluster programmatically:

We are now ready to configure a map to use HD memory and the only configuration change required is to set the in-memory-format to NATIVE.

For extremely low latency use cases in a client-server deployment model, you can configure HD Memory on Near Cache clients also. This design allows you to store large volumes of hot data close to the memory of application clients, eliminating the need of making round trip to servers for similar get requests.

Here is an example:

Let’s take a look at the properties of HD configuration:

• Size: Size of the total native memory to allocate. Default value is 512 MB
• Allocator type: Hazelcast has 2 types of mechanism for allocating memory to HD.
  • STANDARD: allocate memory using default OS memory manager. Standard allocation works fine with large objects as the number of allocations will not be high. But for smaller objects where the system would require to perform many allocations of memory blocks, performance would be affected due to the allocator being a singleton resource that is shared among all threads. It uses malloc()/free() from standard glibc library to allocate memory and these are slow operations when performed by a single but shared resource.
  • POOLED: is Hazelcast’s own pooling memory allocator and the default strategy. It allocates memory in fairly large blocks (4mb by default). These large blocks may further split into smaller blocks to form a pool when required. The allocator can also decide to merge these smaller blocks back to create a big block if/when required. The allocation is based on buddy allocation policy. Memory allocation happens on demand; as opposed to Standard allocation, local memory pools are allocated per-thread every time when memory is required to store data. The allocation is similar to tcmalloc but without passing blocks between threads. This makes the allocation faster and efficient.
• Page size: Size of the page in bytes to allocate memory as a block. It is used only by the POOLED memory allocator. Default value is 1 << 22 = 4194304 Bytes, about 4 MB.
• Minimum block size: is the size of smallest block that will be allocated. It is used only by the POOLED memory allocator. Blocks are configured in power-of-2 sizes. One memory block does not store 2 entries, resulting in possible fragmentation. For example, if you have a block configured to be of 16 KB and the entry size is 12 KB, there will be a wastage of 4 KB. If the entry happens to be 20 KB in size then one block of 32 KB will be used, leaving 12 KB of unused space in memory block, causing higher internal fragmentation. Therefore, tune this property carefully when you know the in-memory size of your object, to reduce internal memory fragmentation. Also, be careful in not setting this value to a very low number as smaller block leads to more internal fragmentation due to the overhead of block header, metadata structures, more heavyweight buddy-merging process and hence, much higher chance of external fragmentation. In most cases, you won’t need to go lower than the default of 16 bytes.
• Metadata space percentage: Defines the percentage of the allocated native memory that is used for the metadata of other map components such as index (for predicates), offset, etc. It is used only by the POOLED memory allocator. Default value is 12.5. Set this to a higher number if the number of entries is in the region of few millions and/or have indexes, otherwise an exception is thrown when metadata space fills up completely.

Good to know Forced Eviction: Hazelcast has a behind-the-scene forced eviction in HD Memory that prevents the system from going OOM due to fragmentation even if the fragmentation is very high. Since fragmentation is measured in bytes, this forced eviction kicks in if there are not enough bytes left to store data. This forced eviction policy is built inside Hazelcast Enterprise HD and is independent of standard Map or JCache eviction that can be controlled by user also. However, forced eviction in IMap is triggered when an external eviction of some sort is configured for this map whereas in JCache, forced eviction is always enabled by default.

Conclusion

Hazelcast HD Memory solves the problem of storing high volumes of data in the memory by maximizing utilization of physical memory in the system to store data for ultra fast access. Hazelcast HD Memory dramatically reduces the latency of access to your data, thereby increasing the time-value of your data. Use of HD Memory is highly recommended to address the problems of:

• GC and problems imposed by it
• Inability to scale up within the box or machine
• Inadequate utilization of resources
• Unpredictable performance
• High infrastructure and operational costs

The picture below demonstrates the capabilities and purpose of HD Memory with the impact on performance:

This was a simple test conducted to show the difference in performance due to GC. In the test with normal heap, the system recorded 9 major garbage collection cycles, amounting to 49 seconds of total pause time. Whereas with HD Memory, no major GC pause was recorded and the total duration of minor GC cycles was also significantly lower than on-heap memory test.

Important Tips

• Always use POOLED allocation
• When sizing the cluster, keep room for memory fragmentation which can be approximately 25% of your physical memory
• Typical size of metadata that gets created on heap for indexes, offsets etc is 16 bytes per index. If you configure multiple indexes and have large data set, tune the property of metadata space percentage accordingly.
• Always leave room for failures: if a node fails in the cluster, remaining nodes should have space to accommodate data that belonged to the failed node. I suggest to use no more than 60% of the available physical memory for storing data.

For more information on Hazelcast and HD documentation, visit the link: http://docs.hazelcast.org/docs/3.6/manual/html-single/index.html#high-density-memory-store

5: https://hazelcast.com/wp-content/uploads/blog-archive/2016/05/image04.png “Shot 5