What is DeltaStream?

How DeltaStream combines streaming and batch processing for real-time analytics using the compute engine of your choice.

DeltaStream is a unified analytics platform that combines batch and stream processing and real-time analytics in one platform. Write queries in SQL, and DeltaStream routes them to the optimal execution engine – for example, Flink for streaming, Spark for batch, and ClickHouse for materialized views. Hierarchical namespacing enables you to process and query streams, similar to the way in which you’d structure data in Iceberg, Snowflake, or Delta Table.

You can also use DeltaStream’s SQL-based interface to create stream processing applications such as streaming pipelines, materialized views, microservices, and more.

You deploy DeltaStream within your organization’s cloud, keeping your data confined to your environment. Depending on the scale and complexity of your workloads, you can do your real-time analytical processing in DeltaStream or continue to use your preferred 3rd-party engine such as AWS Athena or Trino. For example:

• Query materialized views in ClickHouse with native support for upserts

• Stream from Apache Kafka or AWS Kinesis to Clickhouse for extra low-latency queries

• Add upsert capability to your Snowflake or Clickhouse processing

• Read from and write to Iceberg tables (with support for both the Glue catalog and the REST catalog)

• Augment your Databricks Lakehouse with Apache Flink for true real-time stream processing

Deltastream integrates with streaming storage services including Apache Kafka, AWS Kinesis, Confluent Cloud, AWS MSK, WarpStream, and Redpanda.