Compact Serialization Added to Serialization Benchmark Suite

Hazelcast Platform continues adding new features to simplify developers’ writing of high-performance applications. Some of these features first appear as a ‘Beta’ or ‘Preview’ for one or more releases before the API becomes stable and released as a generally Available (GA) platform feature.

Compact Serialization is now one of “these” features. Hazelcast has long supported many different serialization mechanisms. There is no one-size-fits-all solution since different workloads have different requirements in areas like performance, interoperability across languages, and the need to support object versioning. Even though there is no best solution, there is at least one to avoid: Java Serialization. Because this has been the default option, it’s easy for a new user to go with the default setting, but this will result in suboptimal performance in nearly all cases.

So given the variety of choices already available, why introduce another serialization mechanism?  As Hazelcast has evolved from caching use cases to more real-time distributed compute, demands have shifted. Being able to partially deserialize an object, for example, is less important if the client always wants the full object returned. But as SQL becomes the dominant way of querying Hazelcast data, we see far more queries that select only specific columns rather than an entire object, so full object deserialization is wasteful. Similarly, we see more significant numbers of non-Java clients, so polyglot support has become more critical. Serialization schemes that supported object versioning, like the Portable format, did so by storing and sending schema information along with every object; Compact separates the schema, stores it only once, and avoids repeatedly sending the same schema.  

If your domain classes do not implement any other serialization mechanism (such as Java Serializable or Externalizable interfaces, or Hazelcast’s IdentifiedDataSerializable) then they will, by default, use Compact Serialization. To make Compact Serialization an explicit choice for classes that have enabled other serialization mechanisms, you can explicitly register them for Compact Serialization programmatically or declaratively, as shown in the documentation. 

Neil Stevenson posted a blog examining the performance of the most popular serialization options used at that time; this is still an excellent reference for understanding the performance characteristics of the different choices and is well-worth the read. But since Compact Serialization wasn’t available then, I wanted to provide a brief update to the blog that includes information about the performance of Compact Serialization, using the same benchmark from the earlier blog post. 

My original updated benchmark results were completed during the Beta period of Hazelcast Platform 5.2, and are presented in HazelVision Episode 10: Compact Serialization. You can watch that video for more explanation of the results or see the raw numbers below. Two sets of measurements were done – one to measure the size of the serialized objects (which in turn impacts memory requirements to store the data in the grid, and the network bandwidth needed to transfer items between members and clients). The second measurement is the CPU time needed to serialize and deserialize the data, which impacts the performance (both throughput and latency) of operations that get or set the data. 

Here are the object size results from those original tests:

 

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 

Sizes for 100,000 records.

                                        java.io.Serializable :   32,747,458 bytes :   first object is 409 bytes

                                      java.io.Externalizable :   15,896,489 bytes :   first object is 215 bytes

            com.hazelcast.nio.serialization.DataSerializable :   15,896,337 bytes :   first object is 211 bytes

  com.hazelcast.nio.serialization.IdentifiedDataSerializable :    6,045,064 bytes :   first object is  79 bytes

                    com.hazelcast.nio.serialization.Portable :   15,796,261 bytes :   first object is 208 bytes

           com.hazelcast.nio.serialization.VersionedPortable :   15,796,261 bytes :   first object is 208 bytes

                     com.hazelcast.nio.serialization.Compact :    6,495,159 bytes :   first object is  86 bytes

                       com.hazelcast.core.HazelcastJsonValue :    9,946,052 bytes :   first object is 144 bytes

                                     https://avro.apache.org :    3,394,741 bytes :   first object is  44 bytes

                    https://github.com/EsotericSoftware/kryo :    3,094,703 bytes :   first object is  40 bytes

              https://developers.google.com/protocol-buffers :    4,145,337 bytes :   first object is  56 bytes

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 

 

Here are the serialization/deserialization time results:

Benchmark                                                           (kindStr)  Mode  Cnt      Score       Error  Units

Timer.deserialize                                        java.io.Serializable  avgt    3  44215.793 ± 44170.530  ms/op

Timer.deserialize                                      java.io.Externalizable  avgt    3  11240.083 ±  3564.550  ms/op

Timer.deserialize            com.hazelcast.nio.serialization.DataSerializable  avgt    3   4416.772 ±  3000.143  ms/op

Timer.deserialize  com.hazelcast.nio.serialization.IdentifiedDataSerializable  avgt    3   2022.424 ±  2175.608  ms/op

Timer.deserialize                    com.hazelcast.nio.serialization.Portable  avgt    3   4247.027 ±  5164.019  ms/op

Timer.deserialize           com.hazelcast.nio.serialization.VersionedPortable  avgt    3   3955.259 ±  1542.450  ms/op

Timer.deserialize                     com.hazelcast.nio.serialization.Compact  avgt    3   5990.315 ±  2579.730  ms/op

Timer.deserialize                       com.hazelcast.core.HazelcastJsonValue  avgt    3   1561.570 ±   210.369  ms/op

Timer.deserialize                                     https://avro.apache.org  avgt    3   9467.304 ±  1923.199  ms/op

Timer.deserialize                    https://github.com/EsotericSoftware/kryo  avgt    3   4604.161 ±   310.732  ms/op

Timer.deserialize              https://developers.google.com/protocol-buffers  avgt    3   2156.267 ±   139.682  ms/op

 

Timer.serialize                                          java.io.Serializable  avgt    3  12543.481 ±  2410.757  ms/op

Timer.serialize                                        java.io.Externalizable  avgt    3   6157.792 ±   126.061  ms/op

Timer.serialize              com.hazelcast.nio.serialization.DataSerializable  avgt    3   4148.649 ±   116.265  ms/op

Timer.serialize    com.hazelcast.nio.serialization.IdentifiedDataSerializable  avgt    3   3039.590 ±    77.650  ms/op

Timer.serialize                      com.hazelcast.nio.serialization.Portable  avgt    3   5438.809 ±   350.096  ms/op

Timer.serialize             com.hazelcast.nio.serialization.VersionedPortable  avgt    3   5737.014 ±  1311.173  ms/op

Timer.serialize                       com.hazelcast.nio.serialization.Compact  avgt    3   4351.516 ±   237.803  ms/op

Timer.serialize                         com.hazelcast.core.HazelcastJsonValue  avgt    3   2846.657 ±  2002.508  ms/op

Timer.serialize                                       https://avro.apache.org  avgt    3   4558.247 ±   375.578  ms/op

Timer.serialize                      https://github.com/EsotericSoftware/kryo  avgt    3   5234.505 ±   372.863  ms/op

Timer.serialize                https://developers.google.com/protocol-buffers  avgt    3   2385.283 ±   198.084  ms/op

After running these benchmarks and publishing results internally, I received feedback from our engineering team suggesting the performance could be boosted considerably by providing an explicit Compact Serializer rather than falling back to the default Compact Serialization that uses Java Reflection to extract schema information from a class. Just as there are different trade-offs to be made between different serialization mechanisms, there are trade-offs to be made within the choice of Compact Serialization – there is a zero-code, zero configuration (in many cases) option where the schema for a class is determined reflectively. This approach is quick to get up and running with Compact Serialization and still provides significantly better performance than other low-code options, such as Java Serialization. However, an explicitly written serializer will bypass the overhead of the Reflection APIs and can be a better choice for performance-critical code.  The choice isn’t all-or-nothing; you can certainly use the default reflective Compact Serialization for objects that aren’t high-volume or performance-critical; and then write custom serializers for the high-use domain objects where real-time performance is needed.  

When the benchmark is updated to use hand-coded serializers, there is no difference in the object size calculations – a Compact Serialized object for the specific domain object used in the test is still 86 bytes as it contains exactly the same fields encoded the same way. The difference is in the time required to serialize and deserialize the objects, as shown below: 

Deserialization – using reflective serializer

Timer.deserialize                     com.hazelcast.nio.serialization.Compact  avgt    3   5990.315 ±  2579.730  ms/op

Deserialization – using hand-coded serializer

Timer.deserialize                     com.hazelcast.nio.serialization.Compact  avgt    3   3062.712 ±   111.624  ms/op

The time to deserialize 100K objects dropped from just under 6ms per object to just over 3; nearly a 50% reduction.  The serialization changes were not as dramatic but still impressive: 

Serialization – using reflective serializer

Timer.serialize                       com.hazelcast.nio.serialization.Compact  avgt    3   4351.516 ±   237.803  ms/op

Serialization – using hand-coded serializer

Timer.serialize                       com.hazelcast.nio.serialization.Compact  avgt    3   3266.894 ±    99.157  ms/op

Aside from the raw numbers, we can see that the Compact Deserializer moved up from 7th fastest (of 11 choices) to 3rd fastest. On the serialization task, it moved from 5th fastest to 2nd. These performance gains are not necessarily representative of what will be seen across the board but are significant enough that the investment of time will be worthwhile for workloads where low-latency, real-time performance is critical. 

With these results, it really seems like we’re getting a twofer with Compact Serialization – we have one option (reflective serializers) that is no-code (zero or minimal configuration) and gives us benefits like polyglot support, object versioning, and partial deserialization – with strong performance. And, we have a second option (hand-coded serializers) that give us the same advantages with even better performance if you’re willing to put in the extra work of manually coding the serializers.  

With Compact Serialization, we have an alternative that provides a good combination of performance, multi-language support, and ease of development. When used in conjunction with the GenericRecord capability, Compact Serialization also provides a powerful schema evolution capability that is especially useful in deployment architectures, such as microservices, where clients may not be updated at the same time changes are rolled out to the server.

See also:

• 5.2 Release Technical Blog covering Compact Serialization
• Compact Serialization documentation
• Benchmark code used to produce the results
• HazelVision Episode 10: Compact Serialization
• Introduction to Compact Serialization
• Compact Serialization: In-Depth