What Is a Streaming Database?
A streaming database is broadly defined as a data store designed to collect, process, and/or enrich an incoming series of data points (i.e., a data stream) in real time, typically immediately after the data is created. This term does not refer to a discrete class of database management systems, but rather, applies to several types of databases that handle streaming data in real time, including in-memory data grids, in-memory databases, NewSQL databases, NoSQL databases, and time-series databases.
A streaming database is in contrast to traditional relational database management systems (RDBMSs), in which a database administrator would typically load data via an ETL tool/process at regular intervals such as nightly or weekly. A streaming database may sit alongside RDBMSs for modern use cases in larger enterprises. As the volume of data continues to grow and the velocity of data continues to accelerate, some technologies that once relied primarily on batch-oriented databases now rely more heavily on streaming database technologies (e.g., recommendation engines).
Business Use Cases for a Streaming Database
There are many reasons why business teams are encouraging their IT partners to adopt streaming databases. At a high level, business teams see that streaming databases can enable them to:
- Respond to events faster than competitors
- Enable real-time alerting for market changes
- Support preventive maintenance use cases
- Analyze data in real time as it is generated
- Deploy real-time machine learning inference
Technical Use Cases for a Streaming Database
Technologists are adopting streaming databases for a variety of use cases. These include:
- Stream data enrichment. One important use case for streaming databases is storing data that can enrich streaming data. Since streaming data, especially from Internet of Things sources, is almost always minimalistic, joining that data with reference data from a streaming database can provide more context for analysis.
- Real-time event capture and processing. From the C-level down, many companies want to become event-driven, and streaming databases can help IT teams get there while often providing some of the same benefits as traditional databases, such as the ability to interact with SQL-like languages.
- Microservices architectures. Streaming databases can move data from purpose-built app to purpose-built app in real time, so they can serve as the backbone for sharing data and messaging in microservices architectures, which are becoming more common.
- Stream processing. Much of the data that people, applications, and machines create today is generated as a series of ongoing events. Streaming databases can execute continuous queries to process these events as they occur rather than as idle batches of stale data.