Query
There are two main types of queries:
In the DeltaStream UI Query tab, or when listing queries with the CLI, DeltaStream displays only long-running streaming queries. Queries on materialized views are snapshot queries and therefore not long-lived.
In DeltaStream, SQL queries that actually process data are just called queries.
Streaming or Continuous Queries
In DeltaStream, a streaming or continuous query does the following:
reads from one or more streams and/or changelogs
processes the data according to the query logic
generates one or more streams or changelogs.
Streaming queries are continuous. When you start them they keep running until you explicitly terminate them.
Streaming queries themselves take two forms in DeltaStream:
Persistence streaming (continuous) queries: These queries persist the results in the streaming storage by creating new streams or changelogs.
Interactive streaming (continuous) queries: When you start these queries, they run continuously and stream the results of the query to you.
Streaming queries start in the NEW state. When you run them they transition to the RUNNING state. At this point they process records from the source relation(s) and write into the sink relation. When a streaming query fails or otherwise stops operating, it displays as an ERRORED state.
The system may automatically restart a query if it detects an underlying problem, such as a network issue. In failure scenarios, DeltaStream restarts queries from the last known state and then they continue to operate normally.
Note Unrecoverable failures are reported back through the DESCRIBE QUERY command or by clicking on the Query state on the Web App.
If you terminate a query on your own, DeltaStream considers the query complete and performs system cleanup.
For more details, see Query for a list of statements that can start a query for processing records within a Store.
Query on Materialized Views
When you build a Materialized View in DeltaStream, you can use standard SQL queries to query the content of the view. Unlike continuous queries, queries on materialized views compute and return results based on the content of the view at the time you issue the query. These queries behave like the queries on tables in traditional relational databases.
Last updated