In-Memory Database
An in-memory database (IMDB) is a computer system that stores and retrieves data records that reside in a computer’s main memory, e.g., random-access memory (RAM). With data in RAM, IMDBs have a speed advantage over traditional disk-based databases that incur access delays since storage media like hard disk drives and solid-state drives (SSD) have significantly slower access times than RAM. This means that IMDBs are useful for when fast reads and writes of data are crucial.
It is worth noting that the notion of an IMDB does not imply the structure of the stored data. In other words, IMDBs can store relational (tabular) data, document data, key-value data, or even a combination.
How Does an In-Memory Database Work?
IMDBs work by keeping all data in RAM. That is the medium in which data is stored in RAM versus disks or SSDs. IMDBs essentially replace the disk-accessing component of disk-based databases with RAM accesses. In some IMDBs, a disk-based component remains intact, but RAM is the primary storage medium. Since RAM is volatile (e.g., data is lost if the computer loses power), some IMDBs also store data on disk as a preventative measure to minimize the risk of data loss.
Most IMDBs also guard against data loss in a single data center (a capability known as “high availability”) by keeping copies (“replicas”) of all data records in multiple computers in a cluster. This data redundancy ensures that any data record will not be lost upon failure of any given computer.
Example Use Cases
An example use case would be a high traffic eCommerce site that needs to store shopping cart contents for hundreds of thousands of customers at any given time. Response times at that scale would be too slow for many disk-based databases, so IMDBs are used to keep up with the load and ensure a positive customer experience.
Another use case entails business intelligence (BI) analytics, in which data is retrieved and presented in a visual way (i.e., in dashboards). Many types of business users want to see data presented in multiple ways, as well as to drill down into greater data detail. To do so efficiently, they leverage an in-memory database that can quickly deliver data so users spend more time analyzing and making decisions, and less time waiting for the system to respond.
Alternatives to In-Memory Databases
The term in-memory database (IMDB) is also used loosely to refer to any general-purpose technology that stores data in memory. For example, an in-memory data grid (IMDG), which is a distributed system that can store and process data in memory to build high-speed applications, has provisions for IMDB capabilities. Data structures in IMDGs known as “maps” (a construct that most Java developers are familiar with) are basically key-value stores that can be used as an IMDB. Hazelcast IMDG is an example IMDG that supports high-speed distributed maps so it can be used as an IMDB for fast lookups and writes of record-oriented data.
As another example, in-memory caching systems are used to speed up access to slow databases (especially disk-based), essentially emulating the performance of IMDBs. While these caches are not true IMDBs since they do not store data long term, they work in conjunction with disk-based databases as a (mostly) functional equivalent to IMDBs. The IMDBs temporarily store frequently-accessed data that reside in the underlying database to deliver that data faster to applications. Hazelcast IMDG is often used as a caching system for speeding up access to data that resides in disk-based databases.