# Creating a Private Link from DeltaStream to Amazon MSK

Below is the procedure for creating private links for the data you’re streaming via DeltaStream. It includes the configuration needed within AWS MSK.

{% hint style="info" %}
**Note** Currently DeltaStream supports private links only in AWS.
{% endhint %}

### Before You Begin <a href="#h.w4x5gk7yaqvs" id="h.w4x5gk7yaqvs"></a>

* Review [Introducing Private Links](https://docs.deltastream.io/how-do-i.../creating-private-links/introducing-deltastream-private-links).
* You must have signed up with DeltaStream and created at least one organization. Private links function within the context of an organization; from a logical perspective, you enable your private links within a specific DeltaStream organization.
* [Download the DeltaStream CLI](https://docs.deltastream.io/getting-started/starting-with-cli) if you don’t already use it. Currently you cannot create private links via the DeltaStream UI.
* [Request that DeltaStream Operations enable private link](https://console.deltastream.io/support-center) connectivity for your account.
* Optionally, work with DeltaStream Operations if you wish to run all your queries (that is, stream processing) within a dedicated AWS dataplane. By default all DeltaStream customer queries run in a shared multi-tenant dataplane; network policies isolate all traffic among multiple customers. This dedicated data plane completely isolates your workload from other DeltaStream customers by using fully-isolated compute and VPC networking resources.

### Creating a Private Link <a href="#h.tvykf9iyprpc" id="h.tvykf9iyprpc"></a>

This procedure involves building a SQL statement. When you run the statement, DeltaStream processes the link request automatically. Note that the private link is not established until it is accepted or approved by administrators from your organization who are responsible for maintaining Kafka stores.

The entire statement resembles the below; we’ll go through it segment by segment, showing how and where to find the information you need to complete the statement.

{% code overflow="wrap" %}

```sql
CREATE AWS PRIVATE LINK msktest 
WITH ('private_link.target_type' = MSK_PRIVATE_LINK,
      'private_link.msk_cluster_arn' = 'arn:aws:kafka:us-east-1:123456789123:cluster/ds-cc118-stage-he17-0/349e3343-1547-458a-86ab-7d4b424288f9-8',
      'private_link.msk_auth_type' = MSK_AUTH_SASL_IAM,
      'private_link.hosts' (
            'b-1.iam.dscc118stagehe170.f4tzmf.c8.kafka.us-east-1.amazonaws.com:14001' USING PORT 14001 IN 'use1-az6'));
```

{% endcode %}

The private link created using the above SQL command is an inbound private link connection to your dedicated Kafka cluster within your Confluent cloud account.

{% hint style="info" %}
**Note** There are separate but related instructions for creating private links for [Confluent Kafka](https://docs.deltastream.io/how-do-i.../creating-private-links/broken-reference), [RDS Postgres](https://docs.deltastream.io/how-do-i.../creating-private-links/broken-reference), and [Confluent Enterprise edition](https://docs.deltastream.io/how-do-i.../creating-private-links/broken-reference).
{% endhint %}

### Section 1: Enable Multi-VPC Configuration (private link) for AWS MSK <a href="#h.g3a5blm71eb2" id="h.g3a5blm71eb2"></a>

Follow the below guide to enable multi-VPC for your AWS cluster.

[More details from AWS on MSK private link setup](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html).

When you have enabled the private link for the MSK cluster, follow the below procedure to capture the details you need to configure the private link to MSK in the DeltaStream CLI.

1. Go to your Amazon MSK console page and navigate to **Clusters** > **$$yourclustername** to open the cluster details page.
2. In the **Cluster** summary section, locate and copy the **MSK Cluster ARN**.
3. Save the Cluster ARN to a Notepad or other text-based application. You will need to refer back to it again.
4. Note down the cluster ARN as **$$yourclusterarn**

<figure><img src="https://1288764042-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fdbd9e6ZJodkgF1H6AVay%2Fuploads%2Fgit-blob-ad12d21c991f5b952c26764dba579913b0f5aa74%2Fimage2%20(3).png?alt=media" alt="" width="563"><figcaption></figcaption></figure>

3. Indicate the MSK auth type you must use for the connection. This varies, depending on your AWS MSK cluster setup and configuration; it could be SASL/SCRAM authentication or IAM role-based authentication. For the below example we use IAM-based authentication.
4. Capture the broker host URIs. To do this, return to the Amazon MSK **Cluster** details page, and toward the top right of the page click **View client information**.
5. In the **Private endpoint (multi-VPC)** column, multiple broker IAM endpoints display. Capture the **bootstrap URIs** next to the private endpoint column.
   * You must copy each of these endpoints into the SQL command you’re building in the DeltaStream UI.
6. Capture the **availability zone** associated with each broker in the MSK cluster. To do this, return to your **Cluster** details page. Below the **Cluster summary** section click **Properties** and then scroll down to the **Broker details** section.
7. Although there is an **Availability Zone** tab in this section, that is the virtual zone name, and it is not what you need. Instead, you must locate and copy the physical zone ID. To get the physical zone ID, click the corresponding link in the **Client subnets** column. (Each broker endpoint has an associated subnet.)
8. When the **Client subnet** details page displays, locate and copy the **Availability Zone ID** value for each broker subnet.

### Section 2: Writing the SQL Statement in DeltaStream to Create the Private Link <a href="#h.8arlkfu00ge" id="h.8arlkfu00ge"></a>

1. Launch the DeltaStream CLI. The prompt opens to your default organization.
   * If you’re uncertain which organization this is, type `list ORGANIZATIONS`, and in the list that displays, scan down the column labeled `is Current` and find the organization with a value of `true`.
2. Return to the prompt and specify the organization:\
   `Use ORGANIZATION [$$yourorganization]`

The private link connection you’re creating will be in this specific DeltaStream organization. No other DeltaStream organization has access to this private link.

3. Create the private link. To do this, update the SQL with the variables captured in **Section 1** above.

<pre class="language-sql" data-overflow="wrap"><code class="lang-sql"><strong>CREATE AWS PRIVATE LINK msktest
</strong>WITH ('private_link.target_type' = MSK_PRIVATE_LINK,
      'private_link.msk.cluster.arn' = '$$Yourclusterarn',
      'private_link.msk_auth_type' = MSK_AUTH_SASL_IAM,
      'private_link.hosts' (
            '$$brokerhost1:$$brokerhost1PORT' USING PORT $$brokerhost1port IN 'brokerhost1az',
            '$$brokerhost2:$$brokerhost2PORT' USING PORT $$brokerhost2port IN 'brokerhost2az',
            '$$brokerhost3:$$brokerhost3PORT' USING PORT $$brokerhost3port IN 'brokerhost3az'));
</code></pre>

{% code overflow="wrap" %}

```sql
CREATE AWS PRIVATE LINK msktest
WITH ('private_link.target_type' = MSK_PRIVATE_LINK,
      'private_link.msk_cluster_arn' = 'arn:aws:kafka:us-east-1:123456789012:cluster/ds-test-express-broker/11112222-ec9e-4ffe-3333-e1750154c038-8',
      'private_link.msk_auth_type' = MSK_AUTH_SASL_IAM,
      'private_link.hosts' (
            'boot-qui.iam.cluster.gpzcm2.c8.kafka.us-east-1.amazonaws.com:14100' 
USING PORT 14100 IN '*',
            'boot-st9.iam.cluster.gpzcm2.c8.kafka.us-east-1.amazonaws.com:14099' 
USING PORT 14099 IN '*',
            'boot-32r.iam.cluster.gpzcm2.c8.kafka.us-east-1.amazonaws.com:14098' 
USING PORT 14098 IN '*'));
```

{% endcode %}

Your SQL statement in the DeltaStream CLI is now ready. When you run the statement, DeltaStream displays a confirmation and begins creating the private link.

### Section 3 – Establishing Private Link Connectivity Between the DeltaStream Platform and the MSK Cluster <a href="#h.4ook253b4ett" id="h.4ook253b4ett"></a>

To complete the private link connection you must edit your MSK cluster policy to add the Discovery IAM Role ARN that the DeltaStream platform uses. To do this:

1. In the DeltaStream CLI, list your available private links by typing\
   `LIST AWS PRIVATE LINKS;`
2. Locate your newly-created `msktest` private link entry, and copy the **Discovery IAM role ARN**.
3. Navigate back to your MSK **Cluster** details page and under the **Properties** tab scroll down to the **Security settings** section and click **Edit cluster policy**. The **Edit cluster policy** page displays.
4. Replace the code following “AWS”: with the ARN you just copied.
5. Click **Save changes**.

<figure><img src="https://1288764042-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fdbd9e6ZJodkgF1H6AVay%2Fuploads%2Fgit-blob-0e0dce6df5ac5c01c5f5e482cd0fc6295915629d%2Fimage3%20(3).png?alt=media" alt="" width="563"><figcaption></figcaption></figure>

6. Return to the DeltaStream CLI and again list the status of the private link by typing `list AWS private links`. In a few moments you should see the private link transition from in progress to ready.

The cluster is ready to use as a Kafka store from within the DeltaStream platform.

### Verifying Private Link Connectivity <a href="#h.d1yjynj8rjca" id="h.d1yjynj8rjca"></a>

To verify connectivity you can use the DeltaStream console UI to add a Kafka topic.

1. Launch the DeltaStream UI and create a new data store. [Follow the instructions to create a store](https://docs.deltastream.io/how-do-i.../create-and-manage-data-stores/store), if you need. Within about a minute, topics begin to populate your data store. This indicates the private connection is successful and working.

To do this you must set the number of partitions to 1 and the number of replicas to 3.

<figure><img src="https://1288764042-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fdbd9e6ZJodkgF1H6AVay%2Fuploads%2Fgit-blob-9e78c308fa2fa0f162472c9d7b4efbf219668209%2Fimage1%20(3).png?alt=media" alt=""><figcaption></figcaption></figure>

From DeltaStream you should get an alert that the operation was successful. You should also be able to see the newly-created topic in DeltaStream.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.deltastream.io/how-do-i.../creating-private-links/creating-a-private-link-from-deltastream-to-amazon-msk.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.