Glossary Terms›Data and Middleware Technologies›Streaming Database

What Is a Streaming Database?

A streaming database is broadly defined as a data store designed to collect, process, and/or enrich an incoming series of data points (i.e., a data stream) in real time, typically immediately after the data is created. This term does not refer to a discrete class of database management systems, but rather, applies to several types of databases that handle streaming data in real time, including in-memory data grids, in-memory databases, NewSQL databases, NoSQL databases, and time-series databases.

A streaming database is in contrast to traditional relational database management systems (RDBMSs), in which a database administrator would typically load data via an ETL tool/process at regular intervals such as nightly or weekly. A streaming database may sit alongside RDBMSs for modern use cases in larger enterprises. As the volume of data continues to grow and the velocity of data continues to accelerate, some technologies that once relied primarily on batch-oriented databases now rely more heavily on streaming database technologies (e.g., recommendation engines).

Streaming database overview.
A streaming database collects streaming data in real time for immediate processing or for subsequent batch processing.

Business Use Cases for a Streaming Database

There are many reasons why business teams are encouraging their IT partners to adopt streaming databases. At a high level, business teams see that streaming databases can enable them to:

  • Respond to events faster than competitors
  • Enable real-time alerting for market changes
  • Support preventive maintenance use cases
  • Analyze data in real time as it is generated
  • Deploy real-time machine learning inference

Technical Use Cases for a Streaming Database

Technologists are adopting streaming databases for a variety of use cases. These include:

  • Stream data enrichment. One important use case for streaming databases is storing data that can enrich streaming data. Since streaming data, especially from Internet of Things sources, is almost always minimalistic, joining that data with reference data from a streaming database can provide more context for analysis.
  • Real-time event capture and processing. From the C-level down, many companies want to become event-driven, and streaming databases can help IT teams get there while often providing some of the same benefits as traditional databases, such as the ability to interact with SQL-like languages.
  • Microservices architectures. Streaming databases can move data from purpose-built app to purpose-built app in real time, so they can serve as the backbone for sharing data and messaging in microservices architectures, which are becoming more common.
  • Stream processing. Much of the data that people, applications, and machines create today is generated as a series of ongoing events. Streaming databases can execute continuous queries to process these events as they occur rather than as idle batches of stale data.

Related Topics

Back to Glossary Terms

Keep Reading

Video

Live from QCon London: Streaming in a World of Legacy Applications

There are common themes when people describe their reasons for rearchitecting legacy business applications, at a technical level: Speed & Scalability. At a business level: The need to gain new insights flowing from an increasing stream of data. These legacy applications commonly centre around some central datastore such as a relational database.  Moving away from this architecture requires massive migration effort. The costs and risks associated with such an effort can sometimes be prohibitive for business owners, you can’t just rip out your relational database.
Webinar
/ Video
/ 60 min

Fast Data: The Key Ingredients to Real-Time Success

Today, the average enterprise has data streaming into business-critical applications and systems from a dizzying array of endpoints, from smart devices and sensor networks, to web logs and financial transactions. This onslaught of fast data is growing in size, complexity and speed, fueled by increasing business demands along with the rise of the Internet of Things. Therefore, it is no surprise that operationalizing insights at the point-of-action has become a top priority.

Many new technologies are coming to the forefront to facilitate real-time analytics, including in-memory platforms, self-service BI tools and all-flash storage arrays. To educate its readers about the key ingredients for success in building a fast data system, Database Trends and Applications is hosting a special roundtable webinar on February 23rd. Attendees will learn about enabling technologies, important success factors and real-world use cases.