Skip to content

Commit 12f2c40

Browse files
authored
Merge pull request #3635 from ClickHouse/kk/clickpipes-rpe-update
Add prinicipal requirements to ClickPipes RPE doc
2 parents 9633da8 + 0354340 commit 12f2c40

File tree

1 file changed

+40
-21
lines changed

1 file changed

+40
-21
lines changed

docs/integrations/data-ingestion/clickpipes/aws-privatelink.md

Lines changed: 40 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
2-
sidebar_label: 'AWS PrivateLink for ClickPipes'
3-
description: 'Establish a secure connection between ClickPipes and a data source using AWS PrivateLink.'
2+
sidebar_label: "AWS PrivateLink for ClickPipes"
3+
description: "Establish a secure connection between ClickPipes and a data source using AWS PrivateLink."
44
slug: /integrations/clickpipes/aws-privatelink
5-
title: 'AWS PrivateLink for ClickPipes'
5+
title: "AWS PrivateLink for ClickPipes"
66
---
77

88
import cp_service from '@site/static/images/integrations/data-ingestion/clickpipes/cp_service.png';
@@ -40,9 +40,14 @@ Your VPC resources can be accessed in ClickPipes using PrivateLink.
4040
Resource configuration can be targeted with a specific host or RDS cluster ARN.
4141
Cross-region is not supported.
4242

43+
It's the preferred choice for Postgres CDC ingesting data from an RDS cluster.
44+
4345
See a [getting started](https://docs.aws.amazon.com/vpc/latest/privatelink/resource-configuration.html) guide for more details.
4446

45-
It's a preferred choice for Postgres CDC ingesting data from RDS cluster.
47+
:::info
48+
VPC resource needs to be shared with a ClickPipes account. Add `072088201116` to the allowed principals to your resource share configuration.
49+
See AWS guide for [sharing resources](https://docs.aws.amazon.com/ram/latest/userguide/working-with-sharing-create.html) for more details.
50+
:::
4651

4752
### MSK multi-VPC connectivity {#msk-multi-vpc}
4853

@@ -53,6 +58,11 @@ Cross-region is not supported.
5358
It is a recommended option for ClickPipes for MSK.
5459
See the [getting started](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-getting-started.html) guide for more details.
5560

61+
:::info
62+
Update your MSK cluster policy and add `072088201116` to the allowed principals to your MSK cluster.
63+
See AWS guide for [attaching a cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) for more details.
64+
:::
65+
5666
### VPC endpoint service {#vpc-endpoint-service}
5767

5868
VPC service is another approach to share your data source with ClickPipes.
@@ -62,22 +72,30 @@ and configuring the VPC endpoint service to use the NLB.
6272
VPC endpoint service can be [configured with a private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html),
6373
that will be accessible in a ClickPipes VPC.
6474

65-
Cross-region is supported.
66-
6775
It's a preferred choice for:
68-
- any on-premise Kafka setup that requires private DNS support
69-
- cross-region connectivity for Postgres CDC
7076

71-
Cross-region MSK cluster connectivity can be set up using VPC endpoint service as well.
72-
Please reach out to the ClickHouse support team for assistance.
77+
- Any on-premise Kafka setup that requires private DNS support
78+
- Cross-region connectivity for Postgres CDC
79+
- Cross-region connectivity for MSK cluster. Please reach out to the ClickHouse support team for assistance.
80+
81+
See the [getting started](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html) guide for more details.
82+
83+
:::info
84+
Add ClickPipes account ID `072088201116` to the allowed principals to your VPC endpoint service.
85+
See AWS guide for [managing permissions](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permissions) for more details.
86+
:::
87+
88+
:::info
89+
[Cross-region access](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html#endpoint-service-cross-region)
90+
can be configured for ClickPipes. Add [your ClickPipe region](#supported-aws-regions-aws-privatelink-regions) to the allowed regions in your VPC endpoint service.
91+
:::
7392

7493
## Creating a ClickPipe with reverse private endpoint {#creating-clickpipe}
7594

7695
1. Access the SQL Console for your ClickHouse Cloud Service.
7796

7897
<Image img={cp_service} alt="ClickPipes service" size="md" border/>
7998

80-
8199
2. Select the `Data Sources` button on the left-side menu and click on "Set up a ClickPipe"
82100

83101
<Image img={cp_step0} alt="Select imports" size="lg" border/>
@@ -104,22 +122,22 @@ Please reach out to the ClickHouse support team for assistance.
104122

105123
7. Click on `Create` and wait for the reverse private endpoint to be ready.
106124

107-
If you are creating a new endpoint, it will take some time to set up the endpoint.
108-
The page will refresh automatically once the endpoint is ready.
109-
VPC endpoint service might require accepting the connection request in your AWS console.
125+
If you are creating a new endpoint, it will take some time to set up the endpoint.
126+
The page will refresh automatically once the endpoint is ready.
127+
VPC endpoint service might require accepting the connection request in your AWS console.
110128

111129
<Image img={cp_rpe_step3} alt="Select reverse private endpoint" size="lg" border/>
112130

113131
8. Once the endpoint is ready, you can use a DNS name to connect to the data source.
114-
115-
On a list of endpoints, you can see the DNS name for the available endpoint.
116-
It can be either an internally ClickPipes provisioned DNS name or a private DNS name supplied by a PrivateLink service.
117-
DNS name is not a complete network address.
118-
Add the port according to the data source.
119132

120-
MSK connection string can be accessed in the AWS console.
133+
On a list of endpoints, you can see the DNS name for the available endpoint.
134+
It can be either an internally ClickPipes provisioned DNS name or a private DNS name supplied by a PrivateLink service.
135+
DNS name is not a complete network address.
136+
Add the port according to the data source.
121137

122-
To see a full list of DNS names, access it in the cloud service settings.
138+
MSK connection string can be accessed in the AWS console.
139+
140+
To see a full list of DNS names, access it in the cloud service settings.
123141

124142
## Managing existing reverse private endpoints {#managing-existing-endpoints}
125143

@@ -140,6 +158,7 @@ You can manage existing reverse private endpoints in the ClickHouse Cloud servic
140158
## Supported AWS regions {#aws-privatelink-regions}
141159

142160
The following AWS regions are supported for AWS PrivateLink:
161+
143162
- `us-east-1` - for ClickHouse services running in `us-east-1` region
144163
- `eu-central-1` for ClickHouse services running in EU regions
145164
- `us-east-2` - for ClickHouse services running everywhere else

0 commit comments

Comments
 (0)