What is DeltaStream?
How DeltaStream combines streaming and batch processing for real-time analytics
Last updated
How DeltaStream combines streaming and batch processing for real-time analytics
Last updated
DeltaStream is a unified analytics platform that combines batch processing, stream processing, and real-time analytics in one platform. Write queries in SQL, and DeltaStream routes them to the optimal execution engine – for example, Flink for streaming, Spark for batch, and ClickHouse for materialized views. Hierarchical namespacing enables you to process and query streams, similar to the way in which you’d structure data in Iceberg, Snowflake, or Delta Table.
You can also use DeltaStream’s SQL-based interface to create stream processing applications such as streaming pipelines, materialized views, microservices, and more.
You deploy DeltaStream within your organization’s cloud, keeping your data confined to your environment. Depending on the scale and complexity of your workloads, you can do your real-time analytical processing in DeltaStream or continue to use your preferred 3rd-party engine such as AWS Athena or Trino. For example:
Query materialized views in ClickHouse with native support for upserts
Stream from Apache Kafka or AWS Kinesis to Clickhouse for extra low-latency queries
Add upsert capability to your Snowflake or Clickhouse processing
Read from and write to Iceberg tables (with support for both the Glue catalog and the REST catalog)
Augment your Databricks Lakehouse with Apache Flink for true real-time stream processing
Deltastream integrates with streaming storage services including Apache Kafka, AWS Kinesis, Confluent Cloud, AWS MSK, WarpStream, and Redpanda.
