Hot Data Layer

A hot data layer facilitates real-time data collection in a central repository, serving as a foundation for various applications. Unlike analytical databases, it handles operational and transactional data updated by applications and parallel in-cluster tasks. While akin to other integration architectures like a digital integration hub or operational data store, a hot data layer focuses on supporting real-time application development.

On a technology level, a hot data layer consists of the following:

• A distributed processing engine, including for stream processing, to enable large-scale data transformations or calculations that need to be done in parallel
• A real-time data store to capture data from across otherwise disconnected silos and act as a common access point to curated data that is “application ready.”
• Client libraries to build applications that can read or modify data in the data store, as well as insert new data

A hot data layer can help deliver on many business requirements of your IT initiatives, including:

• Correlating and enriching data from multiple silos into a central access point upon which new applications can be built and deployed
• Remove the redundant effort of retrieving and transforming data in each individual application, which results in slower time to market
• Avoid continual access to slow or expensive data sources like mainframes, to enable modern applications built for web and mobile applications that leverage newer programming paradigms and environments
• Delivering a up-to-date, real-time view of your operations, including 360 degree customer analysis and live dashboards
• Immediate, dynamic data transformations that enable technical use cases like extract/load/transform (ELT) processes for downstream analytics, or personalized customer portals that calculate customer-specific prices/products/offers.

While it may appear that any database can be leveraged as a hot data layer, most databases lack specific functionality to deliver on the requirements of a true data layer. Such limitations include:

• Integrating function-specific technologies for a hot data layer can be complex and costly. The Hazelcast unified real-time data platform includes a fast data store, compute framework, and data integration capabilities in a single platform, simplifying hot data layer deployment and reducing hardware costs.
• Performance and scale issues, especially due to disk- and SSD-based accesses – Hazelcast Platform provides ultra-fast data access with its in-memory-based fast data store along with its high performance stream processing engine that can be used for continuous real-time data integration.
• Separate data integration tools add complexity, especially enabling real-time data in the data layer – Hazelcast stream processing capabilities can handle your real-time data integration requirements
• Inefficiency of running client/server applications that transform data outside of the data store – Hazelcast Platform provides server-side processing that takes developer-submitted code and runs that code on the same nodes as the data for high performance computing using server CPU power in a distributed and parallelized way.

Hazelcast Platform was built to be more than a fast data store. It provides a computing framework that distributes work across all nodes in a cluster, enabling application code to be run on the same nodes as the data (“data locality”) to support efficient, real-time data processing. This makes it ideal for use cases that leverage the hot data layer architecture.