Merge branch 'main' into more_visibility_best_practices

Author: Blargian

docs/cloud/bestpractices/index.md

@@ -1,6 +1,6 @@
 ---
 slug: /cloud/bestpractices
-keywords: ['Cloud', 'Best Practices', 'Bulk Inserts', 'Asynchronous Inserts', 'Avoid Mutations', 'Avoid Nullable Columns', 'Avoid Optimize Final', 'Low Cardinality Partitioning Key']
+keywords: ['Cloud', 'Best Practices', 'Bulk Inserts', 'Asynchronous Inserts', 'Avoid Mutations', 'Avoid Nullable Columns', 'Avoid Optimize Final', 'Low Cardinality Partitioning Key', 'Multi Tenancy']
 title: 'Overview'
 hide_title: true
 description: 'Landing page for Best Practices section in ClickHouse'
@@ -18,4 +18,5 @@ This section provides six best practices you will want to follow to get the most
 | [Avoid Nullable Columns](/cloud/bestpractices/avoid-nullable-columns)                            | Learn why you should ideally avoid Nullable columns                        |
 | [Avoid Optimize Final](/cloud/bestpractices/avoid-optimize-final)                              | Learn why you should avoid `OPTIMIZE TABLE ... FINAL`                      |
 | [Choose a Low Cardinality Partitioning Key](/cloud/bestpractices/low-cardinality-partitioning-key)         | Learn how to choose a low cardinality partitioning key.                    |
-| [Usage Limits](/cloud/bestpractices/usage-limits)| Explore the limits of ClickHouse.                                          |
+| [Usage Limits](/cloud/bestpractices/usage-limits)| Explore the limits of ClickHouse.                                          |
+| [Multi tenancy](/cloud/bestpractices/multi-tenancy)| Learn about different strategies to implement multi-tenancy.                                          |

docs/cloud/bestpractices/multitenancy.md

@@ -8,3 +8,7 @@ description: 'Landing page for Cloud API'
 if you've spotted an error or want to change something, please edit the YAML
 frontmatter of the files themselves.
 -->
+| Page | Description |
+|-----|-----|
+| [ClickHouse Cloud API](/cloud/manage/api/api-overview) | Learn about ClickHouse Cloud API |
+| [Cloud API](/cloud/manage/api/) | Landing page for Cloud API |

docs/cloud/manage/api/api-reference-index.md

@@ -34,17 +34,19 @@ To get started, you first need to make sure that your Postgres database is set u
 
 1. [Amazon RDS Postgres](./postgres/source/rds)
 
-2. [Supabase Postgres](./postgres/source/supabase)
+2. [Amazon Aurora Postgres](./postgres/source/aurora)
 
-3. [Google Cloud SQL Postgres](./postgres/source/google-cloudsql)
+3. [Supabase Postgres](./postgres/source/supabase)
 
-4. [Azure Flexible Server for Postgres](./postgres/source/azure-flexible-server-postgres)
+4. [Google Cloud SQL Postgres](./postgres/source/google-cloudsql)
 
-5. [Neon Postgres](./postgres/source/neon-postgres)
+5. [Azure Flexible Server for Postgres](./postgres/source/azure-flexible-server-postgres)
 
-6. [Crunchy Bridge Postgres](./postgres/source/crunchy-postgres)
+6. [Neon Postgres](./postgres/source/neon-postgres)
 
-7. [Generic Postgres Source](./postgres/source/generic), if you are using any other Postgres provider or using a self-hosted instance
+7. [Crunchy Bridge Postgres](./postgres/source/crunchy-postgres)
+
+8. [Generic Postgres Source](./postgres/source/generic), if you are using any other Postgres provider or using a self-hosted instance
 
 
 :::warning

docs/integrations/data-ingestion/clickpipes/postgres/index.md

@@ -0,0 +1,133 @@
+---
+sidebar_label: 'Amazon Aurora Postgres'
+description: 'Set up Amazon Aurora Postgres as a source for ClickPipes'
+slug: /integrations/clickpipes/postgres/source/aurora
+title: 'Aurora Postgres Source Setup Guide'
+---
+
+import parameter_group_in_blade from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/parameter_group_in_blade.png';
+import change_rds_logical_replication from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/change_rds_logical_replication.png';
+import change_wal_sender_timeout from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/change_wal_sender_timeout.png';
+import modify_parameter_group from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/modify_parameter_group.png';
+import reboot_rds from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/reboot_rds.png';
+import security_group_in_rds_postgres from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/security_group_in_rds_postgres.png';
+import edit_inbound_rules from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/source/rds/edit_inbound_rules.png';
+import Image from '@theme/IdealImage';
+
+# Aurora Postgres Source Setup Guide
+
+## Supported Postgres versions {#supported-postgres-versions}
+
+ClickPipes supports Aurora PostgreSQL-Compatible Edition version 12 and later.
+
+## Enable Logical Replication {#enable-logical-replication}
+
+You can skip this section if your Aurora instance already has the following settings configured:
+- `rds.logical_replication = 1`
+- `wal_sender_timeout = 0`
+
+These settings are typically pre-configured if you previously used another data replication tool.
+
+```text
+postgres=> SHOW rds.logical_replication ;
+ rds.logical_replication
+-------------------------
+ on
+(1 row)
+
+postgres=> SHOW wal_sender_timeout ;
+ wal_sender_timeout
+--------------------
+ 0
+(1 row)
+```
+
+If not already configured, follow these steps:
+
+1. Create a new parameter group for your Aurora PostgreSQL version with the required settings:
+    - Set `rds.logical_replication` to 1
+    - Set `wal_sender_timeout` to 0
+
+<Image img={parameter_group_in_blade} alt="Where to find Parameter groups in Aurora" size="lg" border/>
+
+<Image img={change_rds_logical_replication} alt="Changing rds.logical_replication" size="lg" border/>
+
+<Image img={change_wal_sender_timeout} alt="Changing wal_sender_timeout" size="lg" border/>
+
+2. Apply the new parameter group to your Aurora PostgreSQL cluster
+
+<Image img={modify_parameter_group} alt="Modifying Aurora PostgreSQL with new parameter group" size="lg" border/>
+
+3. Reboot your Aurora cluster to apply the changes
+
+<Image img={reboot_rds} alt="Reboot Aurora PostgreSQL" size="lg" border/>
+
+## Configure Database User {#configure-database-user}
+
+Connect to your Aurora PostgreSQL writer instance as an admin user and execute the following commands:
+
+1. Create a dedicated user for ClickPipes:
+
+    ```sql
+    CREATE USER clickpipes_user PASSWORD 'some-password';
+    ```
+
+2. Grant schema permissions. The following example shows permissions for the `public` schema. Repeat these commands for each schema you want to replicate:
+
+    ```sql
+    GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
+    GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
+    ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;
+    ```
+
+3. Grant replication privileges:
+
+    ```sql
+    GRANT rds_replication TO clickpipes_user;
+    ```
+
+4. Create a publication for replication:
+
+    ```sql
+    CREATE PUBLICATION clickpipes_publication FOR ALL TABLES;
+    ```
+
+
+## Configure Network Access {#configure-network-access}
+
+### IP-based Access Control {#ip-based-access-control}
+
+If you want to restrict traffic to your Aurora cluster, please add the [documented static NAT IPs](../../index.md#list-of-static-ips) to the `Inbound rules` of your Aurora security group.
+
+<Image img={security_group_in_rds_postgres} alt="Where to find security group in Aurora PostgreSQL?" size="lg" border/>
+
+<Image img={edit_inbound_rules} alt="Edit inbound rules for the above security group" size="lg" border/>
+
+### Private Access via AWS PrivateLink {#private-access-via-aws-privatelink}
+
+To connect to your Aurora cluster through a private network, you can use AWS PrivateLink. Follow our [AWS PrivateLink setup guide for ClickPipes](/knowledgebase/aws-privatelink-setup-for-clickpipes) to set up the connection.
+
+### Aurora-Specific Considerations {#aurora-specific-considerations}
+
+When setting up ClickPipes with Aurora PostgreSQL, keep these considerations in mind:
+
+1. **Connection Endpoint**: Always connect to the writer endpoint of your Aurora cluster, as logical replication requires write access to create replication slots and must connect to the primary instance.
+
+2. **Failover Handling**: In the event of a failover, Aurora will automatically promote a reader to be the new writer. ClickPipes will detect the disconnection and attempt to reconnect to the writer endpoint, which will now point to the new primary instance.
+
+3. **Global Database**: If you're using Aurora Global Database, you should connect to the primary region's writer endpoint, as cross-region replication already handles data movement between regions.
+
+4. **Storage Considerations**: Aurora's storage layer is shared across all instances in a cluster, which can provide better performance for logical replication compared to standard RDS.
+
+### Dealing with Dynamic Cluster Endpoints {#dealing-with-dynamic-cluster-endpoints}
+
+While Aurora provides stable endpoints that automatically route to the appropriate instance, here are some additional approaches for ensuring consistent connectivity:
+
+1. For high-availability setups, configure your application to use the Aurora writer endpoint, which automatically points to the current primary instance.
+
+2. If using cross-region replication, consider setting up separate ClickPipes for each region to reduce latency and improve fault tolerance.
+
+## What's next? {#whats-next}
+
+You can now [create your ClickPipe](../index.md) and start ingesting data from your Aurora PostgreSQL cluster into ClickHouse Cloud.
+Make sure to note down the connection details you used while setting up your Aurora PostgreSQL cluster as you will need them during the ClickPipe creation process.

docs/integrations/data-ingestion/clickpipes/postgres/source/aurora.md

@@ -0,0 +1,118 @@
+---
+title: 'Connecting Chartbrew to ClickHouse'
+sidebar_label: 'Chartbrew'
+sidebar_position: 131
+slug: /integrations/chartbrew-and-clickhouse
+keywords: ['ClickHouse', 'Chartbrew', 'connect', 'integrate', 'visualization']
+description: 'Connect Chartbrew to ClickHouse to create real-time dashboards and client reports.'
+---
+
+import chartbrew_01 from '@site/static/images/integrations/data-visualization/chartbrew_01.png';
+import chartbrew_02 from '@site/static/images/integrations/data-visualization/chartbrew_02.png';
+import chartbrew_03 from '@site/static/images/integrations/data-visualization/chartbrew_03.png';
+import chartbrew_04 from '@site/static/images/integrations/data-visualization/chartbrew_04.png';
+import chartbrew_05 from '@site/static/images/integrations/data-visualization/chartbrew_05.png';
+import chartbrew_06 from '@site/static/images/integrations/data-visualization/chartbrew_06.png';
+import chartbrew_07 from '@site/static/images/integrations/data-visualization/chartbrew_07.png';
+import chartbrew_08 from '@site/static/images/integrations/data-visualization/chartbrew_08.png';
+import chartbrew_09 from '@site/static/images/integrations/data-visualization/chartbrew_09.png';
+
+import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx';
+import CommunityMaintainedBadge from '@theme/badges/CommunityMaintained';
+import Image from '@theme/IdealImage';
+
+# Connecting Chartbrew to ClickHouse
+
+<CommunityMaintainedBadge/>
+
+[Chartbrew](https://chartbrew.com) is a data visualization platform that allows users to create dashboards and monitor data in real time. It supports multiple data sources, including ClickHouse, and provides a no-code interface for building charts and reports.
+
+## Goal {#goal}
+
+In this guide, you will connect Chartbrew to ClickHouse, run a SQL query, and create a visualization. By the end, your dashboard may look something like this:
+
+<Image img={chartbrew_01} size="lg" alt="Chartbrew dashboard" />
+
+:::tip Add some data
+If you do not have a dataset to work with, you can add one of the examples. This guide uses the [UK Price Paid](/getting-started/example-datasets/uk-price-paid.md) dataset.
+:::
+
+## 1. Gather your connection details {#1-gather-your-connection-details}
+
+<ConnectionDetails />
+
+## 2. Connect Chartbrew to ClickHouse {#2-connect-chartbrew-to-clickhouse}
+
+1. Log in to [Chartbrew](https://chartbrew.com/login) and go to the **Connections** tab.
+2. Click **Create connection** and select **ClickHouse** from the available database options.
+
+   <Image img={chartbrew_02} size="lg" alt="Select ClickHouse connection in Chartbrew" />
+
+3. Enter the connection details for your ClickHouse database:
+
+   - **Display Name**: A name to identify the connection in Chartbrew.
+   - **Host**: The hostname or IP address of your ClickHouse server.
+   - **Port**: Typically `8443` for HTTPS connections.
+   - **Database Name**: The database you want to connect to.
+   - **Username**: Your ClickHouse username.
+   - **Password**: Your ClickHouse password.
+
+   <Image img={chartbrew_03} size="lg" alt="ClickHouse connection settings in Chartbrew" />
+
+4. Click **Test connection** to verify that Chartbrew can connect to ClickHouse.
+5. If the test is successful, click **Save connection**. Chartbrew will automatically retrieve the schema from ClickHouse.
+
+   <Image img={chartbrew_04} size="lg" alt="ClickHouse JSON schema in Chartbrew" />
+
+## 3. Create a dataset and run a SQL query {#3-create-a-dataset-and-run-a-sql-query}
+
+  1. Click on the **Create dataset** button or navigate to the **Datasets** tab to create one.
+  2. Select the ClickHouse connection you created earlier.
+
+  <Image img={chartbrew_05} size="lg" alt="Select ClickHouse connection for dataset" />
+
+  Write a SQL query to retrieve the data you want to visualize. For example, this query calculates the average price paid per year from the `uk_price_paid` dataset:
+
+  ```sql
+  SELECT toYear(date) AS year, avg(price) AS avg_price
+  FROM uk_price_paid
+  GROUP BY year
+  ORDER BY year;
+  ```
+
+  <Image img={chartbrew_07} size="lg" alt="ClickHouse SQL query in Chartbrew" />
+
+  Click **Run query** to fetch the data.
+
+  If you're unsure how to write the query, you can use **Chartbrew's AI assistant** to generate SQL queries based on your database schema.
+
+<Image img={chartbrew_06} size="lg" alt="ClickHouse AI SQL assistant in Chartbrew" />
+
+Once the data is retrieved, click **Configure dataset** to set up the visualization parameters.
+
+## 4. Create a visualization {#4-create-a-visualization}
+   
+  1. Define a metric (numerical value) and dimension (categorical value) for your visualization.
+  2. Preview the dataset to ensure the query results are structured correctly.
+  3. Choose a chart type (e.g., line chart, bar chart, pie chart) and add it to your dashboard.
+  4. Click **Complete dataset** to finalize the setup.
+
+  <Image img={chartbrew_08} size="lg" alt="Chartbrew dashboard with ClickHouse data" />
+
+  You can create as many datasets as you want to visualize different aspects of your data. Using these datasets, you can create multiple dashboards to keep track of different metrics.
+
+  <Image img={chartbrew_01} size="lg" alt="Chartbrew dashboard with ClickHouse data" />
+
+## 5. Automate data updates {#5-automate-data-updates}
+   
+  To keep your dashboard up-to-date, you can schedule automatic data updates:
+
+  1. Click the Calendar icon next to the dataset refresh button.
+  2. Configure the update interval (e.g., every hour, every day).
+  3. Save the settings to enable automatic refresh.
+
+  <Image img={chartbrew_09} size="lg" alt="Chartbrew dataset refresh settings" />
+
+## Learn more {#learn-more}
+
+For more details, check out the blog post about [Chartbrew and ClickHouse](https://chartbrew.com/blog/visualizing-clickhouse-data-with-chartbrew-a-step-by-step-guide/).

docs/integrations/data-visualization/chartbrew-and-clickhouse.md

@@ -26,6 +26,7 @@ Now that your data is in ClickHouse, it's time to analyze it, which often involv
 
 - [Apache Superset](./superset-and-clickhouse.md)
 - [Astrato](./astrato-and-clickhouse.md)
+- [Chartbrew](./chartbrew-and-clickhouse.md)
 - [Deepnote](./deepnote.md)
 - [Draxlr](./draxlr-and-clickhouse.md)
 - [Explo](./explo-and-clickhouse.md)
@@ -47,6 +48,7 @@ Now that your data is in ClickHouse, it's time to analyze it, which often involv
 | [Apache Superset](./superset-and-clickhouse.md)      | ClickHouse official connector | ✅      | ✅          |                                                                                                                                         |
 | [Astrato](./astrato-and-clickhouse.md)      | Native connector | ✅      | ✅          | Works natively using pushdown SQL (direct query only). |
 | [AWS QuickSight](./quicksight-and-clickhouse.md)     | MySQL interface               | ✅      | ✅          | Works with some limitations, see [the documentation](./quicksight-and-clickhouse.md) for more details                |
+| [Chartbrew](./chartbrew-and-clickhouse.md)           | ClickHouse official connector              | ✅      | ✅          |                                                                                                                                         |
 | [Deepnote](./deepnote.md)                            | Native connector              | ✅      | ✅          |                                                                                                                                         |
 | [Explo](./explo-and-clickhouse.md)                   | Native connector              | ✅      | ✅          |                                                                                                                                         |
 | [Grafana](./grafana/index.md)                        | ClickHouse official connector | ✅      | ✅          |                                                                                                                                         |

docs/integrations/data-visualization/index.md

@@ -174,6 +174,7 @@ import great_expectations_logo from '@site/static/images/integrations/logos/grea
 import Hashboardsvg from '@site/static/images/integrations/logos/hashboard.svg';
 import luzmo_logo from '@site/static/images/integrations/logos/luzmo.png';
 import vs_logo from '@site/static/images/integrations/logos/logo_vs.png';
+import chartbrew_logo from '@site/static/images/integrations/logos/logo_chartbrew.png';
 import Image from '@theme/IdealImage';
 
 ClickHouse integrations are organized by their support level:
@@ -258,6 +259,7 @@ We are actively compiling this list of ClickHouse integrations below, so it's no
 |BlinkOps|<Image img={blinkops_logo} size="logo" alt="BlinkOps Logo"/>|Security automation|Create automations to manage data and user permissions.|[Documentation](https://docs.blinkops.com/docs/integrations/clickhouse)|
 |Bytewax|<Bytewaxsvg alt="ByteWax Logo" style={{width: '3rem'}}/>|Data ingestion|Open source Python stream processor for transforming and ingesting data to ClickHouse|[Documentation](https://bytewax.io/blog/building-a-click-house-sink-for-bytewax)|
 |Calyptia (Fluent Bit)|<Image img={calyptia_logo} size="logo" alt="Calyptia logo"/>|Data ingestion|CNCF graduated open-source project for the collection, processing, and delivery of logs, metrics, and traces|[Blog](https://clickhouse.com/blog/kubernetes-logs-to-clickhouse-fluent-bit)|
+|Chartbrew|<Image img={chartbrew_logo} size="logo" alt="Chartbrew logo" style={{width: '3rem', 'backgroundColor': 'transparent', 'boxShadow': 'none'}}/>|Data visualization|Chartbrew is a data visualization platform that allows users to create dashboards and monitor data in real time.|[Documentation](/integrations/chartbrew-and-clickhouse),<br />[Website](https://chartbrew.com/integrations/clickhouse),<br />[Blog](https://chartbrew.com/blog/visualizing-clickhouse-data-with-chartbrew-a-step-by-step-guide/)|
 |CloudCanal|<Cloudcanalsvg className="image" alt="CloudCanal logo" style={{width: '3rem'}}/>|Data integration|A data synchronization and migration tool.|[Website](https://www.cloudcanalx.com/us/)|
 |CloudQuery|<Cloudquerysvg className="image" alt="CloudQuery logo" style={{width: '3rem'}}/>|Data ingestion|Open source high-performance ELT framework.|[Documentation](https://www.cloudquery.io/docs/plugins/destinations/clickhouse/overview)|
 |Cube.js|<Cubejssvg  alt="Cubejs logo" style={{width: '3rem'}}/>|Data visualization|Cube is the Semantic Layer for building data apps.|[Website](https://cube.dev/for/clickhouse-dashboard)|