-
Notifications
You must be signed in to change notification settings - Fork 326
Description
What
Add documentation for the --checkpoint-interval serve option to the config-options reference page. This flag is available in both Core and Enterprise.
Where
- File:
content/shared/influxdb3-cli/config-options.md - Section: Logical placement near the Write-Ahead Log (WAL) section, since checkpoints aggregate the snapshot files produced by the WAL snapshot process.
Flag details from source code
| Detail | Value |
|---|---|
| CLI flag | --checkpoint-interval |
| Env var | INFLUXDB3_CHECKPOINT_INTERVAL |
| Type | humantime::Duration (e.g., 1h, 30m, 10m) |
| Default | Disabled (not set) |
| Available in | Core and Enterprise |
How it works
Without checkpoints, the server loads many snapshot files from object storage on startup — the number is determined by the lookback window (gen1-lookback-duration, default 1 month) divided by gen1-duration (default 10 minutes), with a minimum of 100 snapshots. For the defaults, that's up to ~4,320 snapshots.
With checkpointing enabled:
- During runtime: At the configured interval, snapshot metadata for a calendar month is aggregated into a single checkpoint file and persisted to object storage.
- On startup: The server loads 1-2 checkpoint files per month instead of individual snapshots, then only loads snapshots newer than the latest checkpoint (delta loading).
- Incremental updates: A cached file index enables O(1) lookups for file operations instead of O(n) scanning.
Key implementation details:
- Up to 10 checkpoints loaded concurrently on startup
- 2 checkpoints retained per month (latest + 1 previous)
- Checkpoints are granular per calendar month
- Month rollover is handled automatically
Suggested documentation content
#### checkpoint-interval
Interval between snapshot checkpoint creation. Checkpoints aggregate multiple snapshots into a single file per month, speeding up server startup by reducing the number of files to load.
When enabled, the server periodically consolidates snapshot metadata into monthly checkpoint files in object storage. On startup, the server loads these checkpoint files instead of individual snapshots, then loads only snapshots newer than the latest checkpoint.
This is especially beneficial for long-running servers with continuous data ingestion, where startup time can grow as the number of snapshot files increases.
**Default:** Disabled
| influxdb3 serve option | Environment variable |
| :------------------------ | :-------------------------------- |
| `--checkpoint-interval` | `INFLUXDB3_CHECKPOINT_INTERVAL` |Related Note
As of https://github.com/influxdata/influxdb_pro/issues/2314 we actually have the ability to clean up gen1 and snapshot files, but it's currently only a manually-triggered behavior. In the future, we may push this to some kind of background task which would obviate any need for --checkpoint-interval.