Skip to content

Commit 2692a1e

Browse files
authored
Update 60-optimize-table.md
1 parent 223c674 commit 2692a1e

File tree

1 file changed

+1
-85
lines changed

1 file changed

+1
-85
lines changed

docs/en/sql-reference/10-sql-commands/00-ddl/01-table/60-optimize-table.md

Lines changed: 1 addition & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,7 @@ Databend creates a unique ID for each database and table for storing the snapsho
4747

4848
## Table Optimizations
4949

50-
In Databend, it's advisable to aim for an ideal block size of either 100MB (uncompressed) or 1,000,000 rows, with each segment consisting of 1,000 blocks. To maximize table optimization, it's crucial to gain a clear understanding of when and how to apply various optimization techniques, such as [Segment Compaction](#segment-compaction), [Block Compaction](#block-compaction), and [Purging](#purging).
51-
50+
In Databend, it's advisable to aim for an ideal block size of either 100MB (uncompressed) or 1,000,000 rows, with each segment consisting of 1,000 blocks. To maximize table optimization, it's crucial to gain a clear understanding of when and how to apply various optimization techniques, such as [Segment Compaction](#segment-compaction) and [Block Compaction](#block-compaction).
5251
- When using the COPY INTO or REPLACE INTO command to write data into a table that includes a cluster key, Databend will automatically initiate a re-clustering process, as well as a segment and block compact process.
5352

5453
- Segment & block compactions support distributed execution in cluster environments. You can enable them by setting ENABLE_DISTRIBUTED_COMPACT to 1. This helps enhance data query performance and scalability in cluster environments.
@@ -172,86 +171,3 @@ Compacts the table data by merging small blocks and segments into larger ones.
172171
```sql
173172
OPTIMIZE TABLE my_database.my_table COMPACT LIMIT 50;
174173
```
175-
176-
### Purging
177-
178-
Purging permanently removes historical data, including unused snapshots, segments, and blocks, except for the snapshots within the retention period (including the segments and blocks referenced by this snapshot), which will be retained. This can save storage space but may affect the Time Travel feature. Consider purging when:
179-
180-
- The storage cost is a major concern, and you don't require historical data for Time Travel or other purposes.
181-
- You've compacted your table and want to remove older, unused data.
182-
183-
:::note
184-
Historical data within the default retention period of 24 hours will not be removed. To adjust the retention period, use the *data_retention_time_in_days* setting.
185-
:::
186-
187-
**Syntax**
188-
189-
```sql
190-
OPTIMIZE TABLE <table_name> PURGE
191-
[ BEFORE
192-
(SNAPSHOT => '<SNAPSHOT_ID>') |
193-
(TIMESTAMP => '<TIMESTAMP>'::TIMESTAMP) |
194-
(STREAM => <stream_name>)
195-
]
196-
[ LIMIT <snapshot_count> ]
197-
```
198-
199-
| Parameter | Description |
200-
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
201-
| BEFORE | Specifies the condition for purging historical data. It is used with the `SNAPSHOT`, `TIMESTAMP`, or `STREAM` option to define the point in time before which data should be purged. <br/>When a `BEFORE` option is specified, the command purges historical data by first selecting a base snapshot as indicated by the specified option. Subsequently, it removes snapshots generated before this base snapshot. In the case of specifying a stream with `BEFORE STREAM`, the command identifies the nearest snapshot preceding the creation of the stream as the base snapshot. It then proceeds to remove snapshots generated before this nearest snapshot.|
202-
| LIMIT | Sets the maximum number of snapshots to be purged. When specified, Databend will select and purge the oldest snapshots, up to the specified count. |
203-
204-
**Examples**
205-
206-
This example demonstrates purging historical data using the `BEFORE STREAM` option.
207-
208-
1. Create a table named `t` with a single column `a`, and insert two rows with values 1 and 2 into the table.
209-
210-
```sql
211-
CREATE TABLE t(a INT);
212-
213-
INSERT INTO t VALUES(1);
214-
INSERT INTO t VALUES(2);
215-
```
216-
217-
2. Create a stream named `s` on the table `t`, and add an additional row with value 3 into the table.
218-
219-
```sql
220-
CREATE STREAM s ON TABLE t;
221-
222-
INSERT INTO t VALUES(3);
223-
```
224-
225-
3. Return snapshot IDs and corresponding timestamps for the table `t`.
226-
227-
```sql
228-
SELECT snapshot_id, timestamp FROM FUSE_SNAPSHOT('default', 't');
229-
230-
┌───────────────────────────────────────────────────────────────┐
231-
│ snapshot_id │ timestamp
232-
├──────────────────────────────────┼────────────────────────────┤
233-
│ 00dd8ca67c1f461987f31a6b3a1c3c84 │ 2024-04-02 18:09:39.157702
234-
│ e448bb2bf488489dae7294b0a8af38d1 │ 2024-04-02 18:09:34.986507
235-
│ 2ac038dd83e741afbae543b170105d63 │ 2024-04-02 18:09:34.966336
236-
└───────────────────────────────────────────────────────────────┘
237-
238-
-- Setting data retention time to 0 for demonstration purposes only. Not recommended in production.
239-
SET data_retention_time_in_days = 0;
240-
```
241-
242-
4. Purge historical snapshots using the `BEFORE STREAM` option.
243-
244-
```sql
245-
OPTIMIZE TABLE t PURGE BEFORE (STREAM => s);
246-
247-
-- The command selects snapshot ID e448bb2bf488489dae7294b0a8af38d1 as the base, which was generated immediately before the creation of stream 's'.
248-
-- As a result, snapshot ID 2ac038dd83e741afbae543b170105d63, generated before the base, is removed.
249-
SELECT snapshot_id, timestamp FROM FUSE_SNAPSHOT('default', 't');
250-
251-
┌───────────────────────────────────────────────────────────────┐
252-
│ snapshot_id │ timestamp
253-
├──────────────────────────────────┼────────────────────────────┤
254-
│ 00dd8ca67c1f461987f31a6b3a1c3c84 │ 2024-04-02 18:09:39.157702
255-
│ e448bb2bf488489dae7294b0a8af38d1 │ 2024-04-02 18:09:34.986507
256-
└───────────────────────────────────────────────────────────────┘
257-
```

0 commit comments

Comments
 (0)