Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 56 additions & 47 deletions docs/en/sql-reference/10-sql-commands/00-ddl/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,50 +4,59 @@ title: DDL (Data Definition Language) Commands

These topics provide reference information for the DDL (Data Definition Language) commands in Databend.

## Database and Table Management

- [Database](00-database/index.md)
- [Table](01-table/index.md)
- [View](05-view/index.md)
- [Dictionary](17-dictionary/index.md)

## Database Performance and Indexing

- [Cluster Key](06-clusterkey/index.md)
- [Aggregating Index](07-aggregating-index/index.md)
- [Inverted Index](07-inverted-index/index.md)
- [Ngram Index](07-ngram-index/index.md)
- [Virtual Column](07-virtual-column/index.md)

## User, Role, and Security Management

- [User](02-user/index.md)
- [Network Policy](12-network-policy/index.md)
- [Mask Policy](12-mask-policy/index.md)
- [Password Policy](12-password-policy/index.md)

## Data Staging and Processing

- [Stage](03-stage/index.md)
- [Stream](04-stream/index.md)
- [Sequence](04-sequence/index.md)
- [Task](04-task/index.md)
- [Connection](13-connection/index.md)
- [File Format](13-file-format/index.md)

## Transaction and Variable Management

- [Transaction](14-transaction/index.md)
- [Variable](15-variable/index.md)

## Function and External Integration

- [UDF (User Defined Function)](10-udf/index.md)
- [External Function](11-external-function/index.md)
- [Procedure](18-procedure/index.md)
- [Notification](16-notification/index.md)

## Compute Resource Management

- [Warehouse](19-warehouse/index.md)
- [Workload Group](20-workload-group/index.md)
## Database & Table Management

| Component | Description |
|-----------|-------------|
| **[Database](00-database/index.md)** | Create, alter, and drop databases |
| **[Table](01-table/index.md)** | Create, alter, and manage tables |
| **[View](05-view/index.md)** | Create and manage virtual tables based on queries |
| **[Dictionary](17-dictionary/index.md)** | Create key-value lookup tables for data enrichment |

## Performance & Indexing

| Component | Description |
|-----------|-------------|
| **[Cluster Key](06-clusterkey/index.md)** | Define data clustering for query optimization |
| **[Aggregating Index](07-aggregating-index/index.md)** | Pre-compute aggregations for faster queries |
| **[Inverted Index](07-inverted-index/index.md)** | Full-text search index for text columns |
| **[Ngram Index](07-ngram-index/index.md)** | Substring search index for LIKE patterns |
| **[Virtual Column](07-virtual-column/index.md)** | Extract and index JSON fields as virtual columns |

## Security & Access Control

| Component | Description |
|-----------|-------------|
| **[User](02-user/index.md)** | Create and manage database users |
| **[Network Policy](12-network-policy/index.md)** | Control network access to databases |
| **[Mask Policy](12-mask-policy/index.md)** | Apply data masking for sensitive information |
| **[Password Policy](12-password-policy/index.md)** | Enforce password requirements and rotation |

## Data Integration & Processing

| Component | Description |
|-----------|-------------|
| **[Stage](03-stage/index.md)** | Define storage locations for data loading |
| **[Stream](04-stream/index.md)** | Capture and process data changes |
| **[Task](04-task/index.md)** | Schedule and automate SQL operations |
| **[Sequence](04-sequence/index.md)** | Generate unique sequential numbers |
| **[Connection](13-connection/index.md)** | Configure external data source connections |
| **[File Format](13-file-format/index.md)** | Define formats for data import/export |

## Functions & Procedures

| Component | Description |
|-----------|-------------|
| **[UDF](10-udf/index.md)** | Create custom functions in Python or JavaScript |
| **[External Function](11-external-function/index.md)** | Integrate external APIs as SQL functions |
| **[Procedure](18-procedure/index.md)** | Create stored procedures for complex logic |
| **[Notification](16-notification/index.md)** | Set up event notifications and webhooks |

## Resource Management

| Component | Description |
|-----------|-------------|
| **[Warehouse](19-warehouse/index.md)** | Manage compute resources for query execution |
| **[Workload Group](20-workload-group/index.md)** | Control resource allocation and priorities |
| **[Transaction](14-transaction/index.md)** | Manage database transactions |
| **[Variable](15-variable/index.md)** | Set and use session/global variables |
19 changes: 17 additions & 2 deletions docs/en/sql-reference/10-sql-commands/10-dml/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,23 @@
---
title: DML (Data Manipulation Language) Commands
---
import IndexOverviewList from '@site/src/components/IndexOverviewList';

This page provides reference information for the DML (Data Manipulation Language) commands in Databend.

<IndexOverviewList />
## Data Modification

| Command | Description |
|---------|-------------|
| **[INSERT](dml-insert)** | Add new rows to a table |
| **[INSERT MULTI](dml-insert-multi)** | Insert data into multiple tables in one statement |
| **[UPDATE](dml-update)** | Modify existing rows in a table |
| **[DELETE](dml-delete-from)** | Remove rows from a table |
| **[REPLACE](dml-replace)** | Insert new rows or update existing ones |
| **[MERGE](dml-merge)** | Perform upsert operations based on conditions |

## Data Loading & Export

| Command | Description |
|---------|-------------|
| **[COPY INTO Table](dml-copy-into-table)** | Load data from files into tables |
| **[COPY INTO Location](dml-copy-into-location)** | Export table data to files |
57 changes: 54 additions & 3 deletions docs/en/sql-reference/10-sql-commands/20-query-syntax/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,59 @@
---
title: Query Syntax
---
import IndexOverviewList from '@site/src/components/IndexOverviewList';

This page provides reference information for the query syntax in Databend.
This page provides reference information for the query syntax in Databend. Each component can be used individually or combined to build powerful queries.

<IndexOverviewList />
## Core Query Components

| Component | Description |
|-----------|-------------|
| **[SELECT](query-select)** | Retrieve data from tables - the foundation of all queries |
| **[FROM / JOIN](query-join)** | Specify data sources and combine multiple tables |
| **[WHERE](query-select#where-clause)** | Filter rows based on conditions |
| **[GROUP BY](query-group-by)** | Group rows and perform aggregations (SUM, COUNT, AVG, etc.) |
| **[HAVING](query-group-by#having-clause)** | Filter grouped results |
| **[ORDER BY](query-select#order-by-clause)** | Sort query results |
| **[LIMIT / TOP](top)** | Restrict the number of rows returned |

## Advanced Features

| Component | Description |
|-----------|-------------|
| **[WITH (CTE)](with-clause)** | Define reusable query blocks for complex logic |
| **[PIVOT](query-pivot)** | Convert rows to columns (wide format) |
| **[UNPIVOT](query-unpivot)** | Convert columns to rows (long format) |
| **[QUALIFY](qualify)** | Filter rows after window function calculations |
| **[VALUES](values)** | Create inline temporary data sets |

## Time Travel & Streaming

| Component | Description |
|-----------|-------------|
| **[AT](query-at)** | Query data at a specific point in time |
| **[CHANGES](changes)** | Track insertions, updates, and deletions |
| **[WITH CONSUME](with-consume)** | Process streaming data with offset management |
| **[WITH STREAM HINTS](with-stream-hints)** | Optimize stream processing behavior |

## Query Execution

| Component | Description |
|-----------|-------------|
| **[Settings](settings)** | Configure query optimization and execution parameters |

## Query Structure

A typical Databend query follows this structure:

```sql
[WITH cte_expressions]
SELECT [TOP n] columns
FROM table
[JOIN other_tables]
[WHERE conditions]
[GROUP BY columns]
[HAVING group_conditions]
[QUALIFY window_conditions]
[ORDER BY columns]
[LIMIT n]
```
101 changes: 100 additions & 1 deletion docs/en/sql-reference/10-sql-commands/20-query-syntax/with-clause.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,103 @@
title: WITH Clause
---

Databend uses the WITH clause for common table expressions (CTEs).
The WITH clause is an optional clause that precedes the body of the SELECT statement, and defines one or more CTEs (common table expressions) that can be referenced later in the statement.


## Syntax

### Basic CTE

```sql
[ WITH
cte_name1 [ ( cte_column_list ) ] AS ( SELECT ... )
[ , cte_name2 [ ( cte_column_list ) ] AS ( SELECT ... ) ]
[ , cte_nameN [ ( cte_column_list ) ] AS ( SELECT ... ) ]
]
SELECT ...
```

### Recursive CTE

```sql
[ WITH [ RECURSIVE ]
cte_name1 ( cte_column_list ) AS ( anchorClause UNION ALL recursiveClause )
[ , cte_name2 ( cte_column_list ) AS ( anchorClause UNION ALL recursiveClause ) ]
[ , cte_nameN ( cte_column_list ) AS ( anchorClause UNION ALL recursiveClause ) ]
]
SELECT ...
```

Where:
- `anchorClause`: `SELECT anchor_column_list FROM ...`
- `recursiveClause`: `SELECT recursive_column_list FROM ... [ JOIN ... ]`

## Parameters

| Parameter | Description |
|-----------|-------------|
| `cte_name` | The CTE name must follow standard identifier rules |
| `cte_column_list` | The names of the columns in the CTE |
| `anchor_column_list` | The columns used in the anchor clause for the recursive CTE |
| `recursive_column_list` | The columns used in the recursive clause for the recursive CTE |

## Examples

### Basic CTE

```sql
WITH high_value_customers AS (
SELECT customer_id, customer_name, total_spent
FROM customers
WHERE total_spent > 10000
)
SELECT c.customer_name, o.order_date, o.order_amount
FROM high_value_customers c
JOIN orders o ON c.customer_id = o.customer_id
ORDER BY o.order_date DESC;
```

### Multiple CTEs

```sql
WITH
regional_sales AS (
SELECT region, SUM(sales_amount) as total_sales
FROM sales_data
GROUP BY region
),
top_regions AS (
SELECT region, total_sales
FROM regional_sales
WHERE total_sales > 1000000
)
SELECT r.region, r.total_sales
FROM top_regions r
ORDER BY r.total_sales DESC;
```

### Recursive CTE

```sql
WITH RECURSIVE countdown AS (
-- Anchor clause: starting point
SELECT 10 as num

UNION ALL

-- Recursive clause: repeat until condition
SELECT num - 1
FROM countdown
WHERE num > 1 -- Stop condition
)
SELECT num FROM countdown
ORDER BY num DESC;
```

## Usage Notes

- CTEs are temporary named result sets that exist only for the duration of the query
- CTE names must be unique within the same WITH clause
- A CTE can reference previously defined CTEs in the same WITH clause
- Recursive CTEs require both an anchor clause and a recursive clause connected by UNION ALL
- The RECURSIVE keyword is required when using recursive CTEs
12 changes: 10 additions & 2 deletions docs/en/sql-reference/10-sql-commands/30-query-operators/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
---
title: Query Operators
---
import IndexOverviewList from '@site/src/components/IndexOverviewList';

This page provides reference information for the query operators in Databend.

<IndexOverviewList />
## Operator Types

| Operator Type | Description |
|--------------|-------------|
| **[Arithmetic](arithmetic)** | Mathematical operations (+, -, *, /, %, DIV) |
| **[Comparison](comparison)** | Value comparisons (=, !=, &lt;, &gt;, &lt;=, &gt;=, BETWEEN, IN) |
| **[Logical](logical)** | Boolean logic (AND, OR, NOT, XOR) |
| **[JSON](json)** | JSON data operations (::, -&gt;, -&gt;&gt;, @&gt;, &lt;@) |
| **[Set](set)** | Combine query results (UNION, INTERSECT, EXCEPT) |
| **[Subquery](subquery)** | Nested queries (EXISTS, IN, ANY, ALL, SOME) |
Original file line number Diff line number Diff line change
@@ -1,8 +1,46 @@
---
title: Administration Commands
---
import IndexOverviewList from '@site/src/components/IndexOverviewList';

This page provides reference information for the system administration commands in Databend.

<IndexOverviewList />
## System Monitoring

| Command | Description |
|---------|-------------|
| **[SHOW PROCESSLIST](07-show-processlist.md)** | Display active queries and connections |
| **[SHOW METRICS](08-show-metrics.md)** | View system performance metrics |
| **[KILL](01-kill.md)** | Terminate running queries or connections |
| **[RUST BACKTRACE](rust-backtrace.md)** | Debug Rust stack traces |

## Configuration Management

| Command | Description |
|---------|-------------|
| **[SET](02-set-global.md)** | Set global configuration parameters |
| **[UNSET](02-unset.md)** | Remove configuration settings |
| **[SET VARIABLE](03-set-var.md)** | Manage user-defined variables |
| **[SHOW SETTINGS](03-show-settings.md)** | Display current system settings |

## Function Management

| Command | Description |
|---------|-------------|
| **[SHOW FUNCTIONS](04-show-functions.md)** | List built-in functions |
| **[SHOW USER FUNCTIONS](05-show-user-functions.md)** | List user-defined functions |
| **[SHOW TABLE FUNCTIONS](06-show-table-functions.md)** | List table-valued functions |

## Storage Maintenance

| Command | Description |
|---------|-------------|
| **[VACUUM TABLE](09-vacuum-table.md)** | Reclaim storage space from tables |
| **[VACUUM DROP TABLE](09-vacuum-drop-table.md)** | Clean up dropped table data |
| **[VACUUM TEMP FILES](09-vacuum-temp-files.md)** | Remove temporary files |
| **[SHOW INDEXES](show-indexes.md)** | Display table indexes |

## Dynamic Execution

| Command | Description |
|---------|-------------|
| **[EXECUTE IMMEDIATE](execute-immediate.md)** | Execute dynamically constructed SQL statements |
Loading
Loading