Merge branch 'main' into tp/clickpipes-advanced-settings

Author: tpanetti

docs/best-practices/json_type.md

@@ -224,7 +224,7 @@ ORDER BY doc.update_date
 We provide a type hint for the `update_date` column in the JSON definition, as we use it in the ordering/primary key. This helps ClickHouse to know that this column won't be null and ensures it knows which `update_date` sub-column to use (there may be multiple for each type, so this is ambiguous otherwise).
 :::
 
-We can insert into this table and view the subsequently inferred schema using the [`JSONAllPathsWithTypes`](/sql-reference/functions/json-functions#jsonallpathswithtypes) function and [`PrettyJSONEachRow`](/interfaces/formats/PrettyJSONEachRow) output format:
+We can insert into this table and view the subsequently inferred schema using the [`JSONAllPathsWithTypes`](/sql-reference/functions/json-functions#JSONAllPathsWithTypes) function and [`PrettyJSONEachRow`](/interfaces/formats/PrettyJSONEachRow) output format:
 
 ```sql
 INSERT INTO arxiv FORMAT JSONAsObject 

docs/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md

@@ -101,7 +101,7 @@ input_format_parquet_case_insensitive_column_matching = 1 -- Column matching bet
 :::note Note on nested column structures
 The `VARIANT` and `OBJECT` columns in the original Snowflake table schema will be output as JSON strings by default, forcing us to cast these when inserting them into ClickHouse.
 
-Nested structures such as `some_file` are converted to JSON strings on copy by Snowflake. Importing this data requires us to transform these structures to Tuples at insert time in ClickHouse, using the [JSONExtract function](/sql-reference/functions/json-functions#jsonextract) as shown above.
+Nested structures such as `some_file` are converted to JSON strings on copy by Snowflake. Importing this data requires us to transform these structures to Tuples at insert time in ClickHouse, using the [JSONExtract function](/sql-reference/functions/json-functions#JSONExtract) as shown above.
 :::
 
 ## Test successful data export {#3-testing-successful-data-export}

docs/getting-started/index.md

@@ -42,6 +42,7 @@ by https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-
 | [Foursquare places](/getting-started/example-datasets/foursquare-places) | Dataset with over 100 million records containing information about places on a map, such as shops, restaurants, parks, playgrounds, and monuments. |
 | [GitHub Events Dataset](/getting-started/example-datasets/github-events) | Dataset containing all events on GitHub from 2011 to Dec 6 2020, with a size of 3.1 billion records. |
 | [Hacker News dataset](/getting-started/example-datasets/hacker-news) | Dataset containing 28 million rows of hacker news data. |
+| [Hacker News Vector Search dataset](/getting-started/example-datasets/hackernews-vector-search-dataset) | Dataset containing 28+ million Hacker News postings & their vector embeddings |
 | [LAION 5B dataset](/getting-started/example-datasets/laion-5b-dataset) | Dataset containing 100 million vectors from the LAION 5B dataset |
 | [Laion-400M dataset](/getting-started/example-datasets/laion-400m-dataset) | Dataset containing 400 million images with English image captions |
 | [New York Public Library "What's on the Menu?" Dataset](/getting-started/example-datasets/menus) | Dataset containing 1.3 million records of historical data on the menus of hotels, restaurants and cafes with the dishes along with their prices. |

docs/integrations/data-ingestion/clickpipes/postgres/faq.md

@@ -6,6 +6,9 @@ sidebar_position: 2
 title: 'ClickPipes for Postgres FAQ'
 ---
 
+import failover_slot from '@site/static/images/integrations/data-ingestion/clickpipes/postgres/failover_slot.png'
+import Image from '@theme/IdealImage';
+
 # ClickPipes for Postgres FAQ
 
 ### How does idling affect my Postgres CDC ClickPipe? {#how-does-idling-affect-my-postgres-cdc-clickpipe}
@@ -32,7 +35,7 @@ To set the replica identity to FULL, you can use the following SQL command:
 ```sql
 ALTER TABLE your_table_name REPLICA IDENTITY FULL;
 ```
-REPLICA IDENTITY FULL also enabled replication of unchanged TOAST columns. More on that [here](./toast).
+REPLICA IDENTITY FULL also enables replication of unchanged TOAST columns. More on that [here](./toast).
 
 Note that using `REPLICA IDENTITY FULL` can have performance implications and also faster WAL growth, especially for tables without a primary key and with frequent updates or deletes, as it requires more data to be logged for each change. If you have any doubts or need assistance with setting up primary keys or replica identities for your tables, please reach out to our support team for guidance.
 
@@ -345,3 +348,10 @@ If your initial load has completed without error but your destination ClickHouse
 Also worth checking:
 - If the user has sufficient permissions to read the source tables.
 - If there are any row policies on ClickHouse side which might be filtering out rows.
+
+### Can I have the ClickPipe create a replication slot with failover enabled? {#failover-slot}
+Yes, for a Postgres ClickPipe with replication mode as CDC or Snapshot + CDC, you can have ClickPipes create a replication slot with failover enabled, by toggling the below switch in the `Advanced Settings` section while creating the ClickPipe. Note that your Postgres version must be 17 or above to use this feature.
+
+<Image img={failover_slot} border size="md"/>
+
+If the source is configured accordingly, the slot is preserved after failovers to a Postgres read replica, ensuring continuous data replication. Learn more [here](https://www.postgresql.org/docs/current/logical-replication-failover.html).

docs/integrations/data-ingestion/clickpipes/postgres/schema-changes.md

@@ -10,6 +10,8 @@ ClickPipes for Postgres can detect schema changes in the source tables and, in s
 
 | Schema Change Type                                                                  | Behaviour                             |
 | ----------------------------------------------------------------------------------- | ------------------------------------- |
-| Adding a new column (`ALTER TABLE ADD COLUMN ...`)                                  | Propagated automatically. The new column(s) will be populated for all rows replicated after the schema change                                                   |
-| Adding a new column with a default value (`ALTER TABLE ADD COLUMN ... DEFAULT ...`) | Propagated automatically. The new column(s) will be populated for all rows replicated after the schema change, but existing rows will not show the default value without a full table refresh |
+| Adding a new column (`ALTER TABLE ADD COLUMN ...`)                                  | Propagated automatically once the table gets an insert/update/delete. The new column(s) will be populated for all rows replicated after the schema change                                                   |
+| Adding a new column with a default value (`ALTER TABLE ADD COLUMN ... DEFAULT ...`) | Propagated automatically once the table gets an insert/update/delete. The new column(s) will be populated for all rows replicated after the schema change, but existing rows will not show the default value without a full table refresh |
 | Dropping an existing column (`ALTER TABLE DROP COLUMN ...`)                         | Detected, but **not** propagated. The dropped column(s) will be populated with `NULL` for all rows replicated after the schema change                                                                |
+
+Note that column addition will be propagated at the end of a batch's sync, which could occur after the sync interval or pull batch size is reached. More information on controlling syncs [here](./controlling_sync.md)

docs/integrations/data-ingestion/data-formats/json/other.md

@@ -70,7 +70,7 @@ SELECT JSONExtractString(tags, 'holidays') AS holidays FROM people
 1 row in set. Elapsed: 0.002 sec.
 ```
 
-Notice how the functions require both a reference to the `String` column `tags` and a path in the JSON to extract. Nested paths require functions to be nested e.g. `JSONExtractUInt(JSONExtractString(tags, 'car'), 'year')` which extracts the column `tags.car.year`. The extraction of nested paths can be simplified through the functions [`JSON_QUERY`](/sql-reference/functions/json-functions#json_query) and [`JSON_VALUE`](/sql-reference/functions/json-functions#json_value).
+Notice how the functions require both a reference to the `String` column `tags` and a path in the JSON to extract. Nested paths require functions to be nested e.g. `JSONExtractUInt(JSONExtractString(tags, 'car'), 'year')` which extracts the column `tags.car.year`. The extraction of nested paths can be simplified through the functions [`JSON_QUERY`](/sql-reference/functions/json-functions#JSON_QUERY) and [`JSON_VALUE`](/sql-reference/functions/json-functions#json_value).
 
 Consider the extreme case with the `arxiv` dataset where we consider the entire body to be a `String`.
 

docs/integrations/data-visualization/tableau/tableau-and-clickhouse.md

@@ -30,7 +30,7 @@ the [Tableau Exchange](https://exchange.tableau.com/products/1064).
 The connector is based on ClickHouse's advanced [JDBC driver](/integrations/language-clients/java/jdbc).
 
 With this connector, Tableau integrates ClickHouse databases and tables as data sources. To enable this functionality,
-follow the setup guide bellow.
+follow the setup guide below.
 
 <TOCInline toc={toc}/>
 
@@ -45,10 +45,10 @@ follow the setup guide bellow.
    of <a href="https://github.com/ClickHouse/clickhouse-java/releases/" target="_blank">ClickHouse JDBC driver</a>.
 
 :::note
-Make sure you download the **clickhouse-jdbc-x.x.x-shaded-all.jar** JAR file. Currently, we recommended using versions `0.8.X`.
+Make sure you download the [clickhouse-jdbc-X.X.X-all-dependencies.jar](https://github.com/ClickHouse/clickhouse-java/releases) JAR file. This artifact is available from version `0.9.2`.
 :::
 
-4. Store the JDBC driver in the following folder (based on your OS, if the folder doesn't exist you can create it):
+4. Store the JDBC driver in the following folder (based on your OS, if the folder doesn't exist, you can create it):
     - macOS: `~/Library/Tableau/Drivers`
     - Windows: `C:\Program Files\Tableau\Drivers`
 5. Configure a ClickHouse data source in Tableau and start building data visualizations!
@@ -76,7 +76,7 @@ To solve that, consider upgrading your Tableau Desktop application, or [install
 <br/>
 
 4. Click **Install and Restart Tableau**. Restart the application.
-5. After restarting, the connector will have its full name: `ClickHouse JDBC by ClickHouse, Inc.`. When clicking it the following dialog will pop up:
+5. After restarting, the connector will have its full name: `ClickHouse JDBC by ClickHouse, Inc.`. When clicking it, the following dialog will pop up:
 
 <Image size="md" img={tableau_connector_dialog} alt="ClickHouse connection dialog in Tableau showing fields for server, port, database, username and password" border />
 <br/>
@@ -116,7 +116,7 @@ You are now ready to build some visualizations in Tableau!
 
 ## Building Visualizations in Tableau {#building-visualizations-in-tableau}
 
-Now that have a ClickHouse data source configured in Tableau, let's visualize the data...
+Now that we have a ClickHouse data source configured in Tableau, let's visualize the data...
 
 1. Drag the **CUSTOMER** table onto the workbook. Notice the columns appear, but the data table is empty:
 

docs/use-cases/AI_ML/AIChat/index.md

@@ -0,0 +1,110 @@
+---
+slug: /use-cases/AI_ML/AIChat
+sidebar_label: 'AI Chat'
+title: 'Using AI Chat in ClickHouse Cloud'
+pagination_prev: null
+pagination_next: null
+description: 'Guide to enabling and using the AI Chat feature in ClickHouse Cloud Console'
+keywords: ['AI', 'ClickHouse Cloud', 'Chat', 'SQL Console', 'Agent', 'Docs AI']
+show_related_blogs: true
+sidebar_position: 2
+---
+
+import Link from '@docusaurus/Link';
+import Image from '@theme/IdealImage';
+import img_open from '@site/static/images/use-cases/AI_ML/AIChat/1_open_chat.png';
+import img_consent from '@site/static/images/use-cases/AI_ML/AIChat/2_consent.png';
+import img_modes from '@site/static/images/use-cases/AI_ML/AIChat/3_modes.png';
+import img_thinking from '@site/static/images/use-cases/AI_ML/AIChat/4_thinking.png';
+import img_history from '@site/static/images/use-cases/AI_ML/AIChat/5_history.png';
+import img_result_actions from '@site/static/images/use-cases/AI_ML/AIChat/6_result_actions.png';
+import img_new_tab from '@site/static/images/use-cases/AI_ML/AIChat/7_open_in_editor.png';
+
+# Using AI Chat in ClickHouse Cloud
+
+> This guide explains how to enable and use the AI Chat feature in the ClickHouse Cloud Console.
+
+<VerticalStepper headerLevel="h2">
+
+## Prerequisites {#prerequisites}
+
+1. You must have access to a ClickHouse Cloud organization with AI features enabled (contact your org admin or support if unavailable).
+
+## Open the AI Chat panel {#open-panel}
+
+1. Navigate to a ClickHouse Cloud service.
+2. In the left sidebar, click the sparkle icon labeled “Ask AI”.
+3. (Shortcut) Press <kbd>⌘</kbd> + <kbd>'</kbd> (macOS) or <kbd>Ctrl</kbd> + <kbd>'</kbd> (Linux/Windows) to toggle open.
+
+<Image img={img_open} alt="Open AI Chat flyout" size="md"/>
+
+## Accept the data usage consent (first run) {#consent}
+
+1. On first use you are prompted with a consent dialog describing data handling and third‑party LLM sub-processors.
+2. Review and accept to proceed. If you decline, the panel will not open.
+
+<Image img={img_consent} alt="Consent dialog" size="md"/>
+
+## Choose a chat mode {#modes}
+
+AI Chat currently supports:
+
+- **Agent**: Multi‑step reasoning over schema + metadata (service must be awake).
+- **Docs AI (Ask)**: Focused Q&A grounded in official ClickHouse documentation and best‑practice references.
+
+Use the mode selector at the bottom-left of the flyout to switch.
+
+<Image img={img_modes} alt="Mode selection" size="sm"/>
+
+## Compose and send a message {#compose}
+
+1. Type your question (e.g. “Create a materialized view to aggregate daily events by user”).  
+2. Press <kbd>Enter</kbd> to send (use <kbd>Shift</kbd> + <kbd>Enter</kbd> for a newline).  
+3. While the model is processing you can click “Stop” to interrupt.
+
+## Understanding “Agent” thinking steps {#thinking-steps}
+
+In Agent mode you may see expandable intermediate “thinking” or planning steps. These provide transparency into how the assistant forms its answer. Collapse or expand as needed.
+
+<Image img={img_thinking} alt="Thinking steps" size="md"/>
+
+## Starting new chats {#new-chats}
+
+Click the “New Chat” button to clear context and begin a fresh session.
+
+## Viewing chat history {#history}
+
+1. The lower section lists your recent chats.
+2. Select a previous chat to load its messages.
+3. Delete a conversation using the trash icon.
+
+<Image img={img_history} alt="Chat history list" size="md"/>
+
+## Working with generated SQL {#sql-actions}
+
+When the assistant returns SQL:
+
+- Review for correctness.
+- Click “Open in editor” to load the query into a new SQL tab.
+- Modify and execute within the Console.
+
+<Image img={img_result_actions} alt="Result actions" size="md"/>
+
+<Image img={img_new_tab} alt="Open generated query in editor" size="md"/>
+
+## Stopping or interrupting a response {#interrupt}
+
+If a response is taking too long or diverging:
+
+1. Click the “Stop” button (visible while processing).
+2. The message is marked as interrupted; you can refine your prompt and resend.
+
+## Keyboard shortcuts {#shortcuts}
+
+| Action | Shortcut |
+| ------ | -------- |
+| Open AI Chat | `⌘ + '` / `Ctrl + '` |
+| Send message | `Enter` |
+| New line | `Shift + Enter` |
+
+</VerticalStepper>

scripts/aspell-dict-file.txt

@@ -271,6 +271,7 @@ autovacuum
 VACUUM
 resync
 Resync
+failovers
 --docs/integrations/data-ingestion/clickpipes/mysql/faq.md--
 PlanetScale
 Planetscale

scripts/aspell-ignore/en/aspell-dict.txt

@@ -3034,6 +3034,7 @@ resultset
 resync
 resynchronization
 resyncing
+failovers
 retentions
 rethrow
 retransmit