Query

Querying data in DeltaStream -- how you get things done

There are two main types of queries:

1. Streaming or Continuous Queries

2. Queries on Materialized Views

In the DeltaStream UI Query tab, or when listing queries with the CLI, DeltaStream displays only long-running streaming queries. Queries on materialized views are snapshot queries and therefore not long-lived.

In DeltaStream, SQL queries that actually process data are just called queries.

Streaming or Continuous Queries

In DeltaStream, a streaming or continuous query does the following:

1. reads from one or more streams and/or changelogs

2. processes the data according to the query logic

3. generates one or more streams or changelogs.

Streaming queries are continuous. When you start them they keep running until you explicitly terminate them.

Streaming queries themselves take two forms in DeltaStream:

1. Persistence streaming (continuous) queries: These queries persist the results in the streaming storage by creating new streams or changelogs.

2. Interactive streaming (continuous) queries: When you start these queries, they run continuously and stream the results of the query to you.

Streaming queries start in the NEW state. When you run them they transition to the RUNNING state. At this point they process records from the source relation(s) and write into the sink relation. When a streaming query fails or otherwise stops operating, it displays as an ERRORED state.

The system may automatically restart a query if it detects an underlying problem, such as a network issue. In failure scenarios, DeltaStream restarts queries from the last known state and then they continue to operate normally.

Note You can report back unrecoverable failures either via the DESCRIBE QUERY command or by clicking on the query state on the DeltaStream Web app.

If you terminate a query on your own, DeltaStream considers the query complete and performs system cleanup.

For more details, see Query for a list of statements that can start a query for processing records within a Store.

Query on Materialized Views

When you build a Materialized View in DeltaStream, you can use standard SQL queries to query the content of the view. Unlike continuous queries, queries on materialized views compute and return results based on the content of the view at the time you issue the query. These queries behave like the queries on tables in traditional relational databases.