[Proposal] Improved client status reporting

[rkistner]

Summary

1. Report issues with checkpoint requests (write checkpoints) in downloadError.
2. Refactor sync status to cover all states and increase usability of it.

Status

2025-08-14 Basic ideas proposed, but detailed sync status design is pending.

Background

For general background on problems experienced, see see Improve client-side sync diagnostics.

This diagram summarizes most of the state transitions on the client:

stateDiagram-v2

direction TB
[*] --> disconnected
disconnected --> connect(): connect()
connect() --> disconnected: disconnect()

state connect() {
direction LR
state DownloadStatus {
    direction LR
    [*] -->  connecting
    download_error: waiting_for_retry
    download_error --> connecting: retry after delay
    connecting --> waiting_for_checkpoint
    waiting_for_checkpoint --> downloading
    downloading --> waiting_for_empty_queue
    downloading --> applying
    waiting_for_empty_queue --> waiting_for_requested_checkpoint
    waiting_for_requested_checkpoint --> applying

    applying --> idle
    idle --> downloading
}

state UploadStatus {
    direction LR
    [*] -->  uploading
    [*] -->  upload_idle
    upload_idle: idle
    uploading --> creating_checkpoint_request
    creating_checkpoint_request --> upload_idle
    upload_idle --> uploading
}
}

Loading

The state can currently be inferred using useStatus() hook in React, or similar functionality in other SDKs. However, there are multiple current issues:

1. Not all the states above are covered - some issues are only visible in debug logs.
2. The status has many individual flags (e.g. connecting, connected, downloading, uploading), and it is difficult to predict how that would map to the states above.
3. The "waiting" states are specifically difficult to detect and diagnose issues with, such as when the service never responds to a checkpoint request due to replication lag.
4. It is not obvious why the client could have download_progress: 1, but the data is still not available (typical when in the applying state or one of the waiting states above).

Proposal

Part 1a: Add timeout on waiting_for_requested_checkpoint

This depends on: #317

The above proposal covers re-posting checkpoint requests if not received in 10 seconds.
We additionally add a timeout to report an error if we reach 30 seconds (configurable).

• The timeout is reset when a new checkpoint request is posted.
  • This includes re-posting a checkpoint request when starting a connection.
  • This does not include re-posting an existing one in the same connection (after the 10-second delay).
• The timeout is delayed while we're in state other than waiting_for_requested_checkpoint.

In other words, if we're in the waiting_for_requested_checkpoint state, and the last checkpoint request was created and posted more than 30 seconds ago, we report an error.

Unlike most other errors, this error does not trigger a reconnect - this only reports the issue.

Part 1b: Report checkpoint request errors as downloadError

Currently, errors in creating a new checkpoint request is treated as an upload error, even though the upload queue may be empty at that point. Instead, we should report this as a downloadError.

Part 2: Refactor sync status

We need to refactor sync status to cover all cases in the state diagram above.

Some design considerations:

1. The developer should be able to access the status on varying levels of granularity:
   a. A basic sync indicator may only need to distinguish between "busy / idle / error / disconnected" states.
   b. Others may want selectively include more granularity such as uploading state and download progress.
   c. For diagnostics, the full details should be available.
2. Keep in mind that "downloadError" is not a state by itself, but rather an property preserved until the next successful sync.
3. Some fields such as download progress is usually coupled with the downloading state, but there may be cases where it is available in other states, such when reconnecting.

TODO: Propose an actual design here.