By Enes Akar
Co-founder & CTO, Hazelcast Cloud
Enes was one of the early developers of Hazelcast. He joined the company in 2011 with a passion for making distributed computing incredible simple. From 2012 to 2017, he headed the engineering team as VP of Engineering. Since 2017, he has led the development of Hazelcast Cloud, which is enterprise-grade Hazelcast software as a fully managed service in the cloud. Prior to joining Hazelcast, Enes worked in software development and software engineering at telecommunications companies, including Alcatel-Lucent and Ericsson. He holds a Bachelor of Science in Computer Engineering and an MBA from Bogazici University, Istanbul.
View all blogs by the authorDec 30, 2013
Out of Memory and Hazelcast
OOM is a real trouble for distributed environment. A node having trouble may pull down whole cluster’s performance.
Hazelcast’ general policy on OOM condition is trying to isolate the problem and shutdown instance as soon as possible.
Here as you can see below, it will first try close connections, stopping threads and shutting down JVM instances. Logging the error is the last thing; so you may not see the OOM log.
This is the default action. Closing connections is the first thing being tried. Otherwise other nodes can still consider the problematic node alive till operations’ timeout.
You can have your own action executed when OOM. You can do some cleanup, log error, call System.exit() etc.
Here an example to define a custom OOM handler:
Problem: Hazelcast can detect OOM, when hazelcast threads experience the problem. So if OOME is thrown by an external thread; it can not be handled by hazelcast.