Open Source Projects:
Pricing
Chat
Contact
Back to top

Hazelcast on AWS – Kernel Bug

September 07, 2016

The other day we discovered a bug in older Linux kernel versions and I thought you would like to know about it.

Summary

Use Linux kernel 3.19+ when running Hazelcast on AWS. TCP connections can get stuck with older kernel versions. They appear to be fine, but data are not flowing. This can result in hard-to-explain timeouts.

The Gory Details

Read this: http://johnjianfang.blogspot.bg/2015/08/xen-driver-bug-xennetfront-xennet-skb.html

tl;dr: There is a bug in Xen network driver and AWS happens to use Xen for virtualization. A workaround is to disable (sudo ethtool -K eth0 sg off) the buggy features, but it comes with a performance price and it’s better to use a kernel version with fix = 3.19+. I assume Linux distribution vendors back-ported the fix to older kernel versions, but I have not checked that.

Further Sources:

Kernel Patches:

About the Author

About the Author

Jaromir Hamala

Jaromir Hamala

Director of Engineering

Jaromir wrote his first line of code when he was seven years old. Since then, he has been interested in understanding how systems work ‘under the hood,’ and enjoys blogging about it. Jaromir’s areas of specialty include: performance and stability tuning of JVM, garbage collectors, data grids, application servers and architecture and design of applications. He enjoys pushing systems towards their limits, sharpening his skills by exploring HotSpot source code, contributing to open source projects and arguing about development over a pint of beer.

Follow me on

Latest Blogs

LockSupport.parkNanos() Under the Hood and the Curious Case of Parking, Part II: Windows

LockSupport.parkNanos() Under the Hood and the Curious Case of Parking

Hazelcast Jet Tutorial: Building Custom JDBC Sinks

View all blogs by the author

Free Hazelcast Online Training Center

Whether you're interested in learning the basics of in-memory systems, or you're looking for advanced, real-world production examples and best practices, we've got you covered.