Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .translation-init
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Translation initialization: 2025-09-03T09:07:20.601831
Translation initialization: 2025-09-03T13:05:30.749701
Original file line number Diff line number Diff line change
Expand Up @@ -8,66 +8,65 @@ import FunctionDescription from '@site/src/components/FunctionDescription';

import DetailsWrap from '@site/src/components/DetailsWrap';

ๅœจ Databend ไธญไผ˜ๅŒ–่กจๅŒ…ๆ‹ฌๅŽ‹็ผฉๆˆ–ๆธ…้™คๅކๅฒๆ•ฐๆฎ๏ผŒไปฅ่Š‚็œๅญ˜ๅ‚จ็ฉบ้—ดๅนถๆ้ซ˜ๆŸฅ่ฏขๆ€ง่ƒฝใ€‚
ๅœจ Databend ไธญไผ˜ๅŒ–่กจ๏ผŒๅณ้€š่ฟ‡ๅˆๅนถๆˆ–ๆธ…็†ๅކๅฒๆ•ฐๆฎๆฅ่Š‚็œๅญ˜ๅ‚จ็ฉบ้—ดๅนถๆๅ‡ๆŸฅ่ฏขๆ€ง่ƒฝใ€‚

<DetailsWrap>

<details>
<summary>ไธบไป€ไนˆ่ฆไผ˜ๅŒ–๏ผŸ</summary>
<div>Databend ไฝฟ็”จ Parquet ๆ ผๅผๅฐ†ๆ•ฐๆฎๅญ˜ๅ‚จๅœจ่กจไธญ๏ผŒParquet ๆ ผๅผ่ขซ็ป„็ป‡ๆˆๅ—ใ€‚ๆญคๅค–๏ผŒDatabend ๆ”ฏๆŒๆ—ถ้—ดๅ›žๆบฏๅŠŸ่ƒฝ๏ผŒๅ…ถไธญๆฏไธชไฟฎๆ”น่กจ็š„ๆ“ไฝœ้ƒฝไผš็”Ÿๆˆไธ€ไธช Parquet ๆ–‡ไปถ๏ผŒ่ฏฅๆ–‡ไปถๆ•่Žทๅนถๅๆ˜ ๅฏน่กจๆ‰€ๅš็š„ๆ›ดๆ”นใ€‚</div><br/>
<summary>ไธบไฝ•้œ€่ฆไผ˜ๅŒ–๏ผŸ</summary>
<div>Databend ไฝฟ็”จ Parquet ๆ ผๅผๅฐ†ๆ•ฐๆฎๅญ˜ๅ‚จๅœจ่กจไธญ๏ผŒๅนถๆŒ‰ๅ—๏ผˆBlock๏ผ‰็ป„็ป‡ใ€‚ๆญคๅค–๏ผŒDatabend ๆ”ฏๆŒๆ—ถ้—ดๅ›žๆบฏ๏ผˆTime Travel๏ผ‰ๅŠŸ่ƒฝ๏ผŒๆฏๆฌกไฟฎๆ”น่กจ็š„ๆ“ไฝœ้ƒฝไผš็”Ÿๆˆไธ€ไธช Parquet ๆ–‡ไปถ๏ผŒ็”จไบŽๆ•่Žทๅนถๅๆ˜ ๅฏน่กจ็š„ๅ˜ๆ›ดใ€‚</div><br/>

<div>้š็€ๆ—ถ้—ด็š„ๆŽจ็งป๏ผŒๅฝ“ไธ€ไธช่กจ็งฏ็ดฏไบ†ๆ›ดๅคš็š„ Parquet ๆ–‡ไปถๆ—ถ๏ผŒๅฏ่ƒฝไผšๅฏผ่‡ดๆ€ง่ƒฝ้—ฎ้ข˜ๅ’Œๅญ˜ๅ‚จ้œ€ๆฑ‚็š„ๅขžๅŠ ใ€‚ไธบไบ†ไผ˜ๅŒ–่กจ็š„ๆ€ง่ƒฝ๏ผŒๅฏไปฅๅˆ ้™คไธๅ†้œ€่ฆ็š„ๅކๅฒ Parquet ๆ–‡ไปถใ€‚่ฟ™็งไผ˜ๅŒ–ๆœ‰ๅŠฉไบŽๆ้ซ˜ๆŸฅ่ฏขๆ€ง่ƒฝๅนถๅ‡ๅฐ‘่กจไฝฟ็”จ็š„ๅญ˜ๅ‚จ็ฉบ้—ด้‡ใ€‚</div>
<div>้š็€ๆ—ถ้—ดๆŽจ็งป๏ผŒ่กจไผš็ดฏ็งฏๅคง้‡ Parquet ๆ–‡ไปถ๏ผŒๅฏ่ƒฝๅฏผ่‡ดๆ€ง่ƒฝไธ‹้™ๅ’Œๅญ˜ๅ‚จ้œ€ๆฑ‚ๅขžๅŠ ใ€‚ไธบไผ˜ๅŒ–่กจๆ€ง่ƒฝ๏ผŒๅฏๅœจไธๅ†้œ€่ฆๆ—ถๅˆ ้™คๅކๅฒ Parquet ๆ–‡ไปถใ€‚่ฟ™็งไผ˜ๅŒ–ๆœ‰ๅŠฉไบŽๆๅ‡ๆŸฅ่ฏขๆ€ง่ƒฝๅนถๅ‡ๅฐ‘่กจๅ ็”จ็š„ๅญ˜ๅ‚จ็ฉบ้—ดใ€‚</div>
</details>

</DetailsWrap>

## Databend ๆ•ฐๆฎๅญ˜ๅ‚จ๏ผšๅฟซ็…งใ€ๆฎตๅ’Œๅ—
## Databend ๆ•ฐๆฎๅญ˜ๅ‚จ๏ผšๅฟซ็…งใ€ๆฎตไธŽๅ—

ๅฟซ็…งใ€ๆฎตๅ’Œๅ—ๆ˜ฏ Databend ็”จไบŽๆ•ฐๆฎๅญ˜ๅ‚จ็š„ๆฆ‚ๅฟตใ€‚Databend ไฝฟ็”จๅฎƒไปฌๆฅๆž„ๅปบ็”จไบŽๅญ˜ๅ‚จ่กจๆ•ฐๆฎ็š„ๅˆ†ๅฑ‚็ป“ๆž„ใ€‚
ๅฟซ็…ง๏ผˆSnapshot๏ผ‰ใ€ๆฎต๏ผˆSegment๏ผ‰ๅ’Œๅ—๏ผˆBlock๏ผ‰ๆ˜ฏ Databend ็”จไบŽๆ•ฐๆฎๅญ˜ๅ‚จ็š„ๆ ธๅฟƒๆฆ‚ๅฟต๏ผŒDatabend ๅˆฉ็”จๅฎƒไปฌๆž„ๅปบ่กจๆ•ฐๆฎ็š„ๅˆ†ๅฑ‚ๅญ˜ๅ‚จ็ป“ๆž„ใ€‚

![](/img/sql/storage-structure.PNG)

Databend ๅœจๆ•ฐๆฎๆ›ดๆ–ฐๆ—ถ่‡ชๅŠจๅˆ›ๅปบ่กจๅฟซ็…งใ€‚ๅฟซ็…ง่กจ็คบ่กจๆฎตๅ…ƒๆ•ฐๆฎ็š„็‰ˆๆœฌใ€‚
Databend ๅœจๆ•ฐๆฎๆ›ดๆ–ฐๆ—ถไผš่‡ชๅŠจๅˆ›ๅปบ่กจๅฟซ็…งใ€‚ๅฟซ็…งไปฃ่กจ่กจๆŸไธ€็‰ˆๆœฌ็š„ๆฎตๅ…ƒๆ•ฐๆฎใ€‚

ๅฝ“ไฝฟ็”จ Databend ๆ—ถ๏ผŒๅฝ“ๆ‚จไฝฟ็”จ [AT](../../20-query-syntax/03-query-at.md) ๅญๅฅๆฃ€็ดขๅ’ŒๆŸฅ่ฏข่กจ็š„ๅ…ˆๅ‰็‰ˆๆœฌ็š„ๆ•ฐๆฎๆ—ถ๏ผŒๆ‚จๆœ€ๆœ‰ๅฏ่ƒฝไฝฟ็”จๅฟซ็…ง ID ่ฎฟ้—ฎๅฟซ็…งใ€‚
ไฝฟ็”จ Databend ๆ—ถ๏ผŒ่‹ฅ้€š่ฟ‡ [AT](../../20-query-syntax/03-query-at.md) ๅญๅฅๆฃ€็ดขๅนถๆŸฅ่ฏข่กจ็š„ๅކๅฒ็‰ˆๆœฌๆ•ฐๆฎ๏ผŒ้€šๅธธ้œ€ๅ€ŸๅŠฉๅฟซ็…ง ID ่ฎฟ้—ฎๅฟซ็…งใ€‚

ๅฟซ็…งๆ˜ฏไธ€ไธช JSON ๆ–‡ไปถ๏ผŒๅฎƒไธไฟๅญ˜่กจ็š„ๆ•ฐๆฎ๏ผŒไฝ†ๆŒ‡็คบๅฟซ็…ง้“พๆŽฅๅˆฐ็š„ๆฎตใ€‚ๅฆ‚ๆžœๆ‚จๅฏน่กจ่ฟ่กŒ [FUSE_SNAPSHOT](../../../20-sql-functions/16-system-functions/fuse_snapshot.md)๏ผŒๆ‚จๅฏไปฅๆ‰พๅˆฐไธบ่ฏฅ่กจไฟๅญ˜็š„ๅฟซ็…งใ€‚
ๅฟซ็…งๆ˜ฏไธ€ไธช JSON ๆ–‡ไปถ๏ผŒไธๅญ˜ๅ‚จ่กจๆ•ฐๆฎ๏ผŒไฝ†่ฎฐๅฝ•ไบ†ๅฟซ็…งๅ…ณ่”็š„ๆฎตใ€‚ๅฏน่กจๆ‰ง่กŒ [FUSE_SNAPSHOT](../../../20-sql-functions/16-system-functions/fuse_snapshot.md) ๅฏๆŸฅ็œ‹ไฟๅญ˜็š„ๅฟซ็…งใ€‚

ๆฎตๆ˜ฏไธ€ไธช JSON ๆ–‡ไปถ๏ผŒ็”จไบŽ็ป„็ป‡ๅญ˜ๅ‚จๆ•ฐๆฎ็š„ๅญ˜ๅ‚จๅ—๏ผˆ่‡ณๅฐ‘ 1 ไธช๏ผŒๆœ€ๅคš 1,000 ไธช๏ผ‰ใ€‚ๅฆ‚ๆžœๆ‚จไฝฟ็”จๅฟซ็…ง ID ๅฏนๅฟซ็…ง่ฟ่กŒ [FUSE_SEGMENT](../../../20-sql-functions/16-system-functions/fuse_segment.md)๏ผŒๆ‚จๅฏไปฅๆ‰พๅˆฐๅฟซ็…งๅผ•็”จ็š„ๆฎตใ€‚
ๆฎตๆ˜ฏไธ€ไธช JSON ๆ–‡ไปถ๏ผŒ็”จไบŽ็ป„็ป‡ๅญ˜ๅ‚จๆ•ฐๆฎ็š„ๅ—๏ผˆ่‡ณๅฐ‘ 1 ไธช๏ผŒๆœ€ๅคš 1,000 ไธช๏ผ‰ใ€‚ไฝฟ็”จๅฟซ็…ง ID ๅฏนๅฟซ็…งๆ‰ง่กŒ [FUSE_SEGMENT](../../../20-sql-functions/16-system-functions/fuse_segment.md) ๅฏๆŸฅ็œ‹่ฏฅๅฟซ็…งๅผ•็”จ็š„ๆฎตใ€‚

Databends ๅฐ†ๅฎž้™…่กจๆ•ฐๆฎไฟๅญ˜ๅœจ parquet ๆ–‡ไปถไธญ๏ผŒๅนถๅฐ†ๆฏไธช parquet ๆ–‡ไปถ่ง†ไธบไธ€ไธชๅ—ใ€‚ๅฆ‚ๆžœๆ‚จไฝฟ็”จๅฟซ็…ง ID ๅฏนๅฟซ็…ง่ฟ่กŒ [FUSE_BLOCK](../../../20-sql-functions/16-system-functions/fuse_block.md)๏ผŒๆ‚จๅฏไปฅๆ‰พๅˆฐๅฟซ็…งๅผ•็”จ็š„ๅ—ใ€‚
Databend ๅฐ†ๅฎž้™…่กจๆ•ฐๆฎๅญ˜ๅ‚จๅœจ Parquet ๆ–‡ไปถไธญ๏ผŒๅนถๅฐ†ๆฏไธช Parquet ๆ–‡ไปถ่ง†ไธบไธ€ไธชๅ—ใ€‚ไฝฟ็”จๅฟซ็…ง ID ๅฏนๅฟซ็…งๆ‰ง่กŒ [FUSE_BLOCK](../../../20-sql-functions/16-system-functions/fuse_block.md) ๅฏๆŸฅ็œ‹่ฏฅๅฟซ็…งๅผ•็”จ็š„ๅ—ใ€‚

Databend ไธบๆฏไธชๆ•ฐๆฎๅบ“ๅ’Œ่กจๅˆ›ๅปบไธ€ไธชๅ”ฏไธ€็š„ ID๏ผŒ็”จไบŽๅญ˜ๅ‚จๅฟซ็…งใ€ๆฎตๅ’Œๅ—ๆ–‡ไปถ๏ผŒๅนถๅฐ†ๅฎƒไปฌไฟๅญ˜ๅˆฐๅฏน่ฑกๅญ˜ๅ‚จไธญ็š„่ทฏๅพ„ `<bucket_name>/<tenant_id>/<db_id>/<table_id>/`ใ€‚ๆฏไธชๅฟซ็…งใ€ๆฎตๅ’Œๅ—ๆ–‡ไปถ้ƒฝไฝฟ็”จ UUID๏ผˆ32 ไธชๅญ—็ฌฆ็š„ๅฐๅ†™ๅๅ…ญ่ฟ›ๅˆถๅญ—็ฌฆไธฒ๏ผ‰ๅ‘ฝๅใ€‚
Databend ไธบๆฏไธชๆ•ฐๆฎๅบ“ๅ’Œ่กจๅˆ›ๅปบๅ”ฏไธ€ ID๏ผŒ็”จไบŽๅญ˜ๅ‚จๅฟซ็…งใ€ๆฎตๅ’Œๅ—ๆ–‡ไปถ๏ผŒๅนถๅฐ†ๅ…ถไฟๅญ˜่‡ณๅฏน่ฑกๅญ˜ๅ‚จ่ทฏๅพ„ `<bucket_name>/<tenant_id>/<db_id>/<table_id>/`ใ€‚ๆฏไธชๅฟซ็…งใ€ๆฎตๅ’Œๅ—ๆ–‡ไปถๅ‡ไปฅ UUID๏ผˆ32 ไฝๅฐๅ†™ๅๅ…ญ่ฟ›ๅˆถๅญ—็ฌฆไธฒ๏ผ‰ๅ‘ฝๅใ€‚

| ๆ–‡ไปถ | ๆ ผๅผ | ๆ–‡ไปถๅ | ๅญ˜ๅ‚จๆ–‡ไปถๅคน |
| ๆ–‡ไปถ | ๆ ผๅผ | ๆ–‡ไปถๅ | ๅญ˜ๅ‚จๆ–‡ไปถๅคน |
|----------|---------|---------------------------------|-----------------------------------------------------|
| Snapshot | JSON | `<32bitUUID>_<version>.json` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_ss/` |
| Segment | JSON | `<32bitUUID>_<version>.json` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_sg/` |
| Block | parquet | `<32bitUUID>_<version>.parquet` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_b/` |
| ๅฟซ็…ง | JSON | `<32bitUUID>_<version>.json` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_ss/` |
| ๆฎต | JSON | `<32bitUUID>_<version>.json` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_sg/` |
| ๅ— | parquet | `<32bitUUID>_<version>.parquet` | `<bucket_name>/<tenant_id>/<db_id>/<table_id>/_b/` |

## ่กจไผ˜ๅŒ–

ๅœจ Databend ไธญ๏ผŒๅปบ่ฎฎไปฅ 100MB๏ผˆๆœชๅŽ‹็ผฉ๏ผ‰ๆˆ– 1,000,000 ่กŒไฝœไธบ็†ๆƒณ็š„ๅ—ๅคงๅฐ๏ผŒๆฏไธชๆฎต็”ฑ 1,000 ไธชๅ—็ป„ๆˆใ€‚ไธบไบ†ๆœ€ๅคง้™ๅบฆๅœฐไผ˜ๅŒ–่กจ๏ผŒ่‡ณๅ…ณ้‡่ฆ็š„ๆ˜ฏ่ฆๆธ…ๆฅšๅœฐไบ†่งฃไฝ•ๆ—ถไปฅๅŠๅฆ‚ไฝ•ๅบ”็”จๅ„็งไผ˜ๅŒ–ๆŠ€ๆœฏ๏ผŒไพ‹ๅฆ‚ [ๆฎตๅŽ‹็ผฉ](#segment-compaction)ใ€[ๅ—ๅŽ‹็ผฉ](#block-compaction) ๅ’Œ [ๆธ…้™ค](#purging)ใ€‚
ๅœจ Databend ไธญ๏ผŒๅปบ่ฎฎ็š„็†ๆƒณๅ—ๅคงๅฐไธบ 100 MB๏ผˆๆœชๅŽ‹็ผฉ๏ผ‰ๆˆ– 1,000,000 ่กŒ๏ผŒๆฏไธชๆฎตๅŒ…ๅซ 1,000 ไธชๅ—ใ€‚ไธบๆœ€ๅคงๅŒ–่กจไผ˜ๅŒ–ๆ•ˆๆžœ๏ผŒ้œ€ๆ˜Ž็กฎไฝ•ๆ—ถไปฅๅŠๅฆ‚ไฝ•ๅบ”็”จๅ„็งไผ˜ๅŒ–ๆŠ€ๆœฏ๏ผŒๅฆ‚[ๆฎตๅˆๅนถ](#segment-compaction)ๅ’Œ[ๅ—ๅˆๅนถ](#block-compaction)ใ€‚
- ไฝฟ็”จ COPY INTO ๆˆ– REPLACE INTO ๅ‘ฝไปคๅ‘ๅŒ…ๅซ่š็ฐ‡้”ฎ๏ผˆCluster Key๏ผ‰็š„่กจๅ†™ๅ…ฅๆ•ฐๆฎๆ—ถ๏ผŒDatabend ไผš่‡ชๅŠจ่งฆๅ‘้‡่š็ฐ‡ไปฅๅŠๆฎตๅ’Œๅ—ๅˆๅนถๆต็จ‹ใ€‚

- ๅฝ“ไฝฟ็”จ COPY INTO ๆˆ– REPLACE INTO ๅ‘ฝไปคๅฐ†ๆ•ฐๆฎๅ†™ๅ…ฅๅŒ…ๅซ่š็ฐ‡้”ฎ็š„่กจๆ—ถ๏ผŒDatabend ๅฐ†่‡ชๅŠจๅฏๅŠจ้‡ๆ–ฐ่š็ฐ‡่ฟ‡็จ‹๏ผŒไปฅๅŠๆฎตๅ’Œๅ—ๅŽ‹็ผฉ่ฟ‡็จ‹ใ€‚

- ๆฎตๅ’Œๅ—ๅŽ‹็ผฉๆ”ฏๆŒๅœจ้›†็พค็Žฏๅขƒไธญ่ฟ›่กŒๅˆ†ๅธƒๅผๆ‰ง่กŒใ€‚ๆ‚จๅฏไปฅ้€š่ฟ‡ๅฐ† ENABLE_DISTRIBUTED_COMPACT ่ฎพ็ฝฎไธบ 1 ๆฅๅฏ็”จๅฎƒไปฌใ€‚่ฟ™ๆœ‰ๅŠฉไบŽๆ้ซ˜้›†็พค็Žฏๅขƒไธญ็š„ๆ•ฐๆฎๆŸฅ่ฏขๆ€ง่ƒฝๅ’Œๅฏไผธ็ผฉๆ€งใ€‚
- ๆฎตไธŽๅ—ๅˆๅนถๆ”ฏๆŒๅœจ้›†็พค็Žฏๅขƒไธญๅˆ†ๅธƒๅผๆ‰ง่กŒใ€‚ๅฏ้€š่ฟ‡่ฎพ็ฝฎ ENABLE_DISTRIBUTED_COMPACT ไธบ 1 ๅฏ็”จ๏ผŒไปฅๆๅ‡้›†็พค็Žฏๅขƒไธญ็š„ๆŸฅ่ฏขๆ€ง่ƒฝๅ’Œๅฏๆ‰ฉๅฑ•ๆ€งใ€‚

```sql
SET enable_distributed_compact = 1;
```

### ๆฎตๅŽ‹็ผฉ
### ๆฎตๅˆๅนถ๏ผˆSegment Compaction๏ผ‰

ๅฝ“ไธ€ไธช่กจๆœ‰ๅคชๅคšๅฐๆฎต๏ผˆๆฏไธชๆฎตๅฐไบŽ `100 blocks`๏ผ‰ๆ—ถ๏ผŒๅŽ‹็ผฉๆฎตใ€‚
ๅฝ“่กจๅญ˜ๅœจ่ฟ‡ๅคšๅฐๆฎต๏ผˆๆฏๆฎตๅฐ‘ไบŽ `100 ไธชๅ—`๏ผ‰ๆ—ถ๏ผŒ้œ€่ฟ›่กŒๆฎตๅˆๅนถใ€‚
```sql
SELECT
block_count,
segment_count,
IF(
block_count / segment_count < 100,
'The table needs segment compact now',
'The table does not need segment compact now'
'่ฏฅ่กจๅฝ“ๅ‰้œ€่ฟ›่กŒๆฎตๅˆๅนถ',
'่ฏฅ่กจๅฝ“ๅ‰ๆ— ้œ€ๆฎตๅˆๅนถ'
) AS advice
FROM
fuse_snapshot('your-database', 'your-table')
Expand All @@ -80,21 +79,21 @@ FROM
OPTIMIZE TABLE [database.]table_name COMPACT SEGMENT [LIMIT <segment_count>]
```

้€š่ฟ‡ๅฐ†ๅฐๆฎตๅˆๅนถไธบๅคงๆฎตๆฅๅŽ‹็ผฉ่กจๆ•ฐๆฎใ€‚
้€š่ฟ‡ๅˆๅนถๅฐๆฎตไธบ่พƒๅคงๆฎตๆฅไผ˜ๅŒ–่กจๆ•ฐๆฎใ€‚

- ้€‰้กน LIMIT ่ฎพ็ฝฎ่ฆๅŽ‹็ผฉ็š„ๆœ€ๅคงๆฎตๆ•ฐใ€‚ๅœจ่ฟ™็งๆƒ…ๅ†ตไธ‹๏ผŒDatabend ๅฐ†้€‰ๆ‹ฉๅนถๅŽ‹็ผฉๆœ€ๆ–ฐ็š„ๆฎตใ€‚
- LIMIT ้€‰้กน่ฎพ็ฝฎๆœ€ๅคงๅˆๅนถๆฎตๆ•ฐ๏ผŒDatabend ๅฐ†้€‰ๆ‹ฉๅนถๅˆๅนถๆœ€ๆ–ฐๆฎตใ€‚

**็คบไพ‹**

```sql
-- Check whether need segment compact
-- ๆฃ€ๆŸฅๆ˜ฏๅฆ้œ€ๆฎตๅˆๅนถ
SELECT
block_count,
segment_count,
IF(
block_count / segment_count < 100,
'The table needs segment compact now',
'The table does not need segment compact now'
'่ฏฅ่กจๅฝ“ๅ‰้œ€่ฟ›่กŒๆฎตๅˆๅนถ',
'่ฏฅ่กจๅฝ“ๅ‰ๆ— ้œ€ๆฎตๅˆๅนถ'
) AS advice
FROM
fuse_snapshot('hits', 'hits');
Expand All @@ -105,17 +104,17 @@ FROM
| 751 | 32 | The table needs segment compact now |
+-------------+---------------+-------------------------------------+

-- Compact segment
-- ๆ‰ง่กŒๆฎตๅˆๅนถ
OPTIMIZE TABLE hits COMPACT SEGMENT;

-- Check again
-- ๅ†ๆฌกๆฃ€ๆŸฅ
SELECT
block_count,
segment_count,
IF(
block_count / segment_count < 100,
'The table needs segment compact now',
'The table does not need segment compact now'
'่ฏฅ่กจๅฝ“ๅ‰้œ€่ฟ›่กŒๆฎตๅˆๅนถ',
'่ฏฅ่กจๅฝ“ๅ‰ๆ— ้œ€ๆฎตๅˆๅนถ'
) AS advice
FROM
fuse_snapshot('hits', 'hits')
Expand All @@ -128,130 +127,47 @@ FROM
+-------------+---------------+---------------------------------------------+
```

### ๅ—ๅŽ‹็ผฉ
### ๅ—ๅˆๅนถ๏ผˆBlock Compaction๏ผ‰

ๅฝ“ไธ€ไธช่กจๆœ‰ๅคง้‡ๅฐๅ—ๆˆ–ๅฝ“่ฏฅ่กจๆœ‰ๅพˆ้ซ˜ๆฏ”ไพ‹็š„ๆ’ๅ…ฅใ€ๅˆ ้™คๆˆ–ๆ›ดๆ–ฐ็š„่กŒๆ—ถ๏ผŒๅŽ‹็ผฉๅ—ใ€‚
ๅฝ“่กจๅญ˜ๅœจๅคง้‡ๅฐๅ—๏ผŒๆˆ–ๆ’ๅ…ฅใ€ๅˆ ้™คใ€ๆ›ดๆ–ฐ่กŒๆฏ”ไพ‹่พƒ้ซ˜ๆ—ถ๏ผŒ้œ€่ฟ›่กŒๅ—ๅˆๅนถใ€‚

ๆ‚จๅฏไปฅ้€š่ฟ‡ๆฃ€ๆŸฅๆฏไธชๅ—็š„ๆœชๅŽ‹็ผฉๅคงๅฐๆ˜ฏๅฆๆŽฅ่ฟ‘ `100MB` ็š„ๅฎŒ็พŽๅคงๅฐๆฅๆฃ€ๆŸฅๅฎƒใ€‚
ๅฏ้€š่ฟ‡ๆฃ€ๆŸฅๆฏไธชๅ—็š„ๆœชๅŽ‹็ผฉๅคงๅฐๆ˜ฏๅฆๆŽฅ่ฟ‘็†ๆƒณๅ€ผ `100 MB` ๆฅๅˆคๆ–ญใ€‚

ๅฆ‚ๆžœๅคงๅฐๅฐไบŽ `50MB`๏ผŒๆˆ‘ไปฌๅปบ่ฎฎ่ฟ›่กŒๅ—ๅŽ‹็ผฉ๏ผŒๅ› ไธบๅฎƒ่กจ็คบๆœ‰ๅคชๅคšๅฐๅ—๏ผš
่‹ฅๅคงๅฐๅฐไบŽ `50 MB`๏ผŒๅปบ่ฎฎๆ‰ง่กŒๅ—ๅˆๅนถ๏ผŒ่กจๆ˜Žๅญ˜ๅœจ่ฟ‡ๅคšๅฐๅ—๏ผš

```sql
SELECT
block_count,
humanize_size(bytes_uncompressed / block_count) AS per_block_uncompressed_size,
IF(
bytes_uncompressed / block_count / 1024 / 1024 < 50,
'The table needs block compact now',
'The table does not need block compact now'
'่ฏฅ่กจๅฝ“ๅ‰้œ€่ฟ›่กŒๅ—ๅˆๅนถ',
'่ฏฅ่กจๅฝ“ๅ‰ๆ— ้œ€ๅ—ๅˆๅนถ'
) AS advice
FROM
fuse_snapshot('your-database', 'your-table')
LIMIT 1;
```

:::info
ๆˆ‘ไปฌๅปบ่ฎฎๅ…ˆๆ‰ง่กŒๆฎตๅŽ‹็ผฉ๏ผŒ็„ถๅŽๅ†ๆ‰ง่กŒๅ—ๅŽ‹็ผฉใ€‚
ๅปบ่ฎฎๅ…ˆๆ‰ง่กŒๆฎตๅˆๅนถ๏ผŒๅ†ๆ‰ง่กŒๅ—ๅˆๅนถใ€‚
:::

**่ฏญๆณ•**
```sql
OPTIMIZE TABLE [database.]table_name COMPACT [LIMIT <segment_count>]
```
้€š่ฟ‡ๅฐ†ๅฐๅ—ๅ’Œๆฎตๅˆๅนถไธบๅคงๅ—ๅ’ŒๆฎตๆฅๅŽ‹็ผฉ่กจๆ•ฐๆฎใ€‚
้€š่ฟ‡ๅˆๅนถๅฐๅ—ๅ’Œๆฎตไธบ่พƒๅคงๅ—ๅ’Œๆฎตๆฅไผ˜ๅŒ–่กจๆ•ฐๆฎใ€‚

- ๆญคๅ‘ฝไปคๅˆ›ๅปบๆœ€ๆ–ฐ่กจๆ•ฐๆฎ็š„ๆ–ฐๅฟซ็…ง๏ผˆไปฅๅŠๅŽ‹็ผฉ็š„ๆฎตๅ’Œๅ—๏ผ‰๏ผŒ่€Œไธๅฝฑๅ“็Žฐๆœ‰ๅญ˜ๅ‚จๆ–‡ไปถ๏ผŒๅ› ๆญคๅœจๆ‚จๆธ…้™คๅކๅฒๆ•ฐๆฎไน‹ๅ‰๏ผŒไธไผš้‡Šๆ”พๅญ˜ๅ‚จ็ฉบ้—ดใ€‚
- ่ฏฅๅ‘ฝไปคไผšไธบๆœ€ๆ–ฐ่กจๆ•ฐๆฎๅˆ›ๅปบๆ–ฐๅฟซ็…ง๏ผˆๅซๅˆๅนถๅŽ็š„ๆฎตๅ’Œๅ—๏ผ‰๏ผŒไธๅฝฑๅ“็Žฐๆœ‰ๅญ˜ๅ‚จๆ–‡ไปถ๏ผŒๆ•…้œ€ๆธ…็†ๅކๅฒๆ•ฐๆฎๅŽๆ‰ไผš้‡Šๆ”พๅญ˜ๅ‚จ็ฉบ้—ดใ€‚

- ๆ นๆฎ็ป™ๅฎš่กจ็š„ๅคงๅฐ๏ผŒๅฎŒๆˆๆ‰ง่กŒๅฏ่ƒฝ้œ€่ฆ็›ธๅฝ“้•ฟ็š„ๆ—ถ้—ดใ€‚
- ๆ นๆฎ่กจๅคงๅฐ๏ผŒๆ‰ง่กŒๅฏ่ƒฝ่€—ๆ—ถ่พƒ้•ฟใ€‚

- ้€‰้กน LIMIT ่ฎพ็ฝฎ่ฆๅŽ‹็ผฉ็š„ๆœ€ๅคงๆฎตๆ•ฐใ€‚ๅœจ่ฟ™็งๆƒ…ๅ†ตไธ‹๏ผŒDatabend ๅฐ†้€‰ๆ‹ฉๅนถๅŽ‹็ผฉๆœ€ๆ–ฐ็š„ๆฎตใ€‚
- LIMIT ้€‰้กน่ฎพ็ฝฎๆœ€ๅคงๅˆๅนถๆฎตๆ•ฐ๏ผŒDatabend ๅฐ†้€‰ๆ‹ฉๅนถๅˆๅนถๆœ€ๆ–ฐๆฎตใ€‚

- Databend ๅฐ†ๅœจๅŽ‹็ผฉ่ฟ‡็จ‹ๅŽ่‡ชๅŠจ้‡ๆ–ฐ่š็ฐ‡่š็ฐ‡่กจใ€‚
- ๅˆๅนถๅฎŒๆˆๅŽ๏ผŒDatabend ไผš่‡ชๅŠจๅฏน่š็ฐ‡่กจๆ‰ง่กŒ้‡่š็ฐ‡ใ€‚

**็คบไพ‹**
```sql
OPTIMIZE TABLE my_database.my_table COMPACT LIMIT 50;
```

### ๆธ…้™ค

ๆธ…้™คไผšๆฐธไน…ๅˆ ้™คๅކๅฒๆ•ฐๆฎ๏ผŒๅŒ…ๆ‹ฌๆœชไฝฟ็”จ็š„ๅฟซ็…งใ€ๆฎตๅ’Œๅ—๏ผŒไฝ†ไฟ็•™ๆœŸๅ†…็š„ๅฟซ็…ง๏ผˆๅŒ…ๆ‹ฌๆญคๅฟซ็…งๅผ•็”จ็š„ๆฎตๅ’Œๅ—๏ผ‰้™คๅค–ใ€‚่ฟ™ๅฏไปฅ่Š‚็œๅญ˜ๅ‚จ็ฉบ้—ด๏ผŒไฝ†ๅฏ่ƒฝไผšๅฝฑๅ“ๆ—ถ้—ดๅ›žๆบฏๅŠŸ่ƒฝใ€‚ๅœจไปฅไธ‹ๆƒ…ๅ†ตไธ‹่€ƒ่™‘ๆธ…้™ค๏ผš

- ๅญ˜ๅ‚จๆˆๆœฌๆ˜ฏไธ€ไธชไธป่ฆ้—ฎ้ข˜๏ผŒๅนถไธ”ๆ‚จไธ้œ€่ฆๅކๅฒๆ•ฐๆฎ็”จไบŽๆ—ถ้—ดๅ›žๆบฏๆˆ–ๅ…ถไป–็›ฎ็š„ใ€‚
- ๆ‚จๅทฒ็ปๅŽ‹็ผฉไบ†ๆ‚จ็š„่กจ๏ผŒๅนถไธ”ๆƒณ่ฆๅˆ ้™ค่พƒๆ—ง็š„ใ€ๆœชไฝฟ็”จ็š„็š„ๆ•ฐๆฎใ€‚

:::note
้ป˜่ฎคไฟ็•™ๆœŸไธบ 24 ๅฐๆ—ถ็š„ๅކๅฒๆ•ฐๆฎๅฐ†ไธไผš่ขซๅˆ ้™คใ€‚่ฆ่ฐƒๆ•ดไฟ็•™ๆœŸ๏ผŒ่ฏทไฝฟ็”จ *data_retention_time_in_days* ่ฎพ็ฝฎใ€‚
:::

**่ฏญๆณ•**

```sql
OPTIMIZE TABLE <table_name> PURGE
[ BEFORE
(SNAPSHOT => '<SNAPSHOT_ID>') |
(TIMESTAMP => '<TIMESTAMP>'::TIMESTAMP) |
(STREAM => <stream_name>)
]
[ LIMIT <snapshot_count> ]
```

| ๅ‚ๆ•ฐ | ๆ่ฟฐ |
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| BEFORE | ๆŒ‡ๅฎšๆธ…้™คๅކๅฒๆ•ฐๆฎ็š„ๆกไปถใ€‚ๅฎƒไธŽ `SNAPSHOT`ใ€`TIMESTAMP` ๆˆ– `STREAM` ้€‰้กนไธ€่ตทไฝฟ็”จ๏ผŒไปฅๅฎšไน‰ๅบ”ๆธ…้™คๆ•ฐๆฎ็š„ๆ—ถ้—ด็‚นใ€‚<br/>ๅฝ“ๆŒ‡ๅฎš `BEFORE` ้€‰้กนๆ—ถ๏ผŒ่ฏฅๅ‘ฝไปค้ฆ–ๅ…ˆ้€‰ๆ‹ฉไธ€ไธชๅŸบๆœฌๅฟซ็…ง๏ผˆๅฆ‚ๆŒ‡ๅฎš้€‰้กนๆ‰€็คบ๏ผ‰ๆฅๆธ…้™คๅކๅฒๆ•ฐๆฎใ€‚้šๅŽ๏ผŒๅฎƒไผšๅˆ ้™คๅœจๆญคๅŸบๆœฌๅฟซ็…งไน‹ๅ‰็”Ÿๆˆ็š„ๅฟซ็…งใ€‚ๅœจๆŒ‡ๅฎšๅธฆๆœ‰ `BEFORE STREAM` ็š„ๆต็š„ๆƒ…ๅ†ตไธ‹๏ผŒ่ฏฅๅ‘ฝไปคไผšๅฐ†ๅˆ›ๅปบๆตไน‹ๅ‰็š„ๆœ€่ฟ‘ๅฟซ็…ง่ฏ†ๅˆซไธบๅŸบๆœฌๅฟซ็…งใ€‚็„ถๅŽ๏ผŒๅฎƒไผšๅˆ ้™คๅœจๆญคๆœ€่ฟ‘ๅฟซ็…งไน‹ๅ‰็”Ÿๆˆ็š„ๅฟซ็…งใ€‚|
| LIMIT | ่ฎพ็ฝฎ่ฆๆธ…้™ค็š„ๆœ€ๅคงๅฟซ็…งๆ•ฐใ€‚ๆŒ‡ๅฎšๅŽ๏ผŒDatabend ๅฐ†้€‰ๆ‹ฉๅนถๆธ…้™คๆœ€ๆ—ง็š„ๅฟซ็…ง๏ผŒๆœ€ๅคš่พพๅˆฐๆŒ‡ๅฎš็š„่ฎกๆ•ฐใ€‚ |

**็คบไพ‹**

ๆญค็คบไพ‹ๆผ”็คบๅฆ‚ไฝ•ไฝฟ็”จ `BEFORE STREAM` ้€‰้กนๆธ…้™คๅކๅฒๆ•ฐๆฎใ€‚

1. ๅˆ›ๅปบไธ€ไธชๅไธบ `t` ็š„่กจ๏ผŒๅ…ถไธญๅŒ…ๅซไธ€ไธชๅˆ— `a`๏ผŒๅนถๅฐ†ไธค่กŒๅ€ผ 1 ๅ’Œ 2 ๆ’ๅ…ฅๅˆฐ่กจไธญใ€‚

```sql
CREATE TABLE t(a INT);

INSERT INTO t VALUES(1);
INSERT INTO t VALUES(2);
```

2. ๅœจ่กจ `t` ไธŠๅˆ›ๅปบไธ€ไธชๅไธบ `s` ็š„ๆต๏ผŒๅนถๅฐ†ไธ€ไธชๅ€ผไธบ 3 ็š„้™„ๅŠ ่กŒๆทปๅŠ ๅˆฐ่กจไธญใ€‚

```sql
CREATE STREAM s ON TABLE t;

INSERT INTO t VALUES(3);
```

3. ่ฟ”ๅ›ž่กจ `t` ็š„ๅฟซ็…ง ID ๅ’Œ็›ธๅบ”็š„ๆ—ถ้—ดๆˆณใ€‚

```sql
SELECT snapshot_id, timestamp FROM FUSE_SNAPSHOT('default', 't');

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ snapshot_id โ”‚ timestamp โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 00dd8ca67c1f461987f31a6b3a1c3c84 โ”‚ 2024-04-02 18:09:39.157702 โ”‚
โ”‚ e448bb2bf488489dae7294b0a8af38d1 โ”‚ 2024-04-02 18:09:34.986507 โ”‚
โ”‚ 2ac038dd83e741afbae543b170105d63 โ”‚ 2024-04-02 18:09:34.966336 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

-- ไป…ๅ‡บไบŽๆผ”็คบ็›ฎ็š„๏ผŒๅฐ†ๆ•ฐๆฎไฟ็•™ๆ—ถ้—ด่ฎพ็ฝฎไธบ 0ใ€‚ไธๅปบ่ฎฎๅœจ็”Ÿไบงไธญไฝฟ็”จใ€‚
SET data_retention_time_in_days = 0;
```

4. ไฝฟ็”จ `BEFORE STREAM` ้€‰้กนๆธ…้™คๅކๅฒๅฟซ็…งใ€‚

```sql
OPTIMIZE TABLE t PURGE BEFORE (STREAM => s);

-- ่ฏฅๅ‘ฝไปค้€‰ๆ‹ฉๅฟซ็…ง ID e448bb2bf488489dae7294b0a8af38d1 ไฝœไธบๅŸบๆœฌๅฟซ็…ง๏ผŒ่ฏฅๅฟซ็…งๆ˜ฏๅœจๅˆ›ๅปบๆตโ€œsโ€ไน‹ๅ‰็ซ‹ๅณ็”Ÿๆˆ็š„ใ€‚
-- ๅ› ๆญค๏ผŒๅฐ†ๅˆ ้™คๅœจๅŸบๆœฌๅฟซ็…งไน‹ๅ‰็”Ÿๆˆ็š„ๅฟซ็…ง ID 2ac038dd83e741afbae543b170105d63ใ€‚
SELECT snapshot_id, timestamp FROM FUSE_SNAPSHOT('default', 't');

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ snapshot_id โ”‚ timestamp โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 00dd8ca67c1f461987f31a6b3a1c3c84 โ”‚ 2024-04-02 18:09:39.157702 โ”‚
โ”‚ e448bb2bf488489dae7294b0a8af38d1 โ”‚ 2024-04-02 18:09:34.986507 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```