DeltaStream
Search
K

CREATE TOPIC

Syntax

CREATE TOPIC
topic_name
[IN STORE store_name]
[WITH (topic_parameter = value [, ...])];

Description

DeltaStream processes streaming data that is stored in streaming Store such as Apache Kafka and Amazon Kinesis. The Topic represents the data Stream entity that holds the individual data records in these streaming stores. For example, a Topic in a Kafka Store would be the Kafka topic, a Topic in a Kinesis Store would be a Kinesis data stream.

Arguments

topic_name

The name of the new Topic. For case-sensitive names, the name must be wrapped in double quotes, otherwise, the lowercased name will be used.

IN STORE store_name

Optionally, this creates the Topic in the specified Store. For case-sensitive names, the name must be wrapped in double quotes, otherwise, the lowercased name will be used.

WITH (topic_parameter = value [, …​])

This clause specifies _topic_parameters.

Topic Parameters

Parameter Name
Description
key.descriptor
A qualified https://github.com/deltastreaminc/ds-docs-gitbook/blob/main/reference/sql-syntax/ddl/broken-reference/README.md name used to decode a record's key, if applicable. Clear the Descriptor by specifying NULL.
Required: No Default value: None Type: String Valid values: See LIST DESCRIPTORS.
value.descriptor
A qualified https://github.com/deltastreaminc/ds-docs-gitbook/blob/main/reference/sql-syntax/ddl/broken-reference/README.md name used to decode a record's value. Clear the Descriptor by specifying NULL.
Required: No Default value: None Type: String Valid values: See LIST DESCRIPTORS.

Kafka Specific Topic Parameters

Parameters to be used if the associated Store is type KAFKA:
Parameter Name
Description
topic.partitions
The number of partitions to use when creating the Topic.
Required: No Default value: 1 Type: Integer Valid values: [1,…]
topic.replicas
The number of replicas to use when creating the Topic.
Required: No Default value: 1 Type: Integer Valid values: [1,…]

Kinesis Specific Topic Parameters

Parameters to be used if the associated Store is type KINESIS:
Parameter Name
Description
kinesis.shards
The number of shards to use when creating the Topic.
Required: No Default value: 1 Type: Integer Valid values: [1,…]

Examples

Create a new Topic with defaults

The following creates a Topic called pageviews using the default parameters in the user's default store:
CREATE TOPIC pageviews;

Create a new Topic with key/value Descriptors

The following creates a Topic called pageviews_pb in the user's default store. It also sets the key and value descriptors that are necessary for serializing its records:
SHOW DESCRIPTORS;
Name
----------------------
pb.pageviews_key
pb.pageviews_value
CREATE TOPIC
pageviews_pb
WITH (
'key.descriptor' = 'pb.pageviews_key',
'value.descriptor' = 'pb.pageviews_value'
);

Create a new Topic in a Kafka Store with Kafka parameters

The following creates a Topic called PageviewsKafka in the store named KafkaStore with 2 partitions and a replication factor of 3:
CREATE TOPIC
"PageviewsKafka"
IN STORE "KafkaStore"
WITH ('kafka.partitions' = 2, 'kafka.replicas' = 3);

Create a new Topic in Kinesis Store with Kinesis parameters

The following creates a Topic called pageviews_kinesis in the store named kinesis_store with 3 shards:
CREATE TOPIC
pageviews_kinesis
IN STORE kinesis_store
WITH ('kinesis.shards' = 3);