Skip to content

influxdb3: document --checkpoint-interval flag for reducing startup time #6884

@waynr

Description

@waynr

What

Add documentation for the --checkpoint-interval serve option to the config-options reference page. This flag is available in both Core and Enterprise.

Where

  • File: content/shared/influxdb3-cli/config-options.md
  • Section: Logical placement near the Write-Ahead Log (WAL) section, since checkpoints aggregate the snapshot files produced by the WAL snapshot process.

Flag details from source code

Detail Value
CLI flag --checkpoint-interval
Env var INFLUXDB3_CHECKPOINT_INTERVAL
Type humantime::Duration (e.g., 1h, 30m, 10m)
Default Disabled (not set)
Available in Core and Enterprise

How it works

Without checkpoints, the server loads many snapshot files from object storage on startup — the number is determined by the lookback window (gen1-lookback-duration, default 1 month) divided by gen1-duration (default 10 minutes), with a minimum of 100 snapshots. For the defaults, that's up to ~4,320 snapshots.

With checkpointing enabled:

  1. During runtime: At the configured interval, snapshot metadata for a calendar month is aggregated into a single checkpoint file and persisted to object storage.
  2. On startup: The server loads 1-2 checkpoint files per month instead of individual snapshots, then only loads snapshots newer than the latest checkpoint (delta loading).
  3. Incremental updates: A cached file index enables O(1) lookups for file operations instead of O(n) scanning.

Key implementation details:

  • Up to 10 checkpoints loaded concurrently on startup
  • 2 checkpoints retained per month (latest + 1 previous)
  • Checkpoints are granular per calendar month
  • Month rollover is handled automatically

Suggested documentation content

#### checkpoint-interval

Interval between snapshot checkpoint creation. Checkpoints aggregate multiple snapshots into a single file per month, speeding up server startup by reducing the number of files to load.

When enabled, the server periodically consolidates snapshot metadata into monthly checkpoint files in object storage. On startup, the server loads these checkpoint files instead of individual snapshots, then loads only snapshots newer than the latest checkpoint.

This is especially beneficial for long-running servers with continuous data ingestion, where startup time can grow as the number of snapshot files increases.

**Default:** Disabled

| influxdb3 serve option    | Environment variable              |
| :------------------------ | :-------------------------------- |
| `--checkpoint-interval`   | `INFLUXDB3_CHECKPOINT_INTERVAL`   |

Related Note

As of https://github.com/influxdata/influxdb_pro/issues/2314 we actually have the ability to clean up gen1 and snapshot files, but it's currently only a manually-triggered behavior. In the future, we may push this to some kind of background task which would obviate any need for --checkpoint-interval.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions