Skip to content

[bug] Kobo sync times out on large libraries even with shelf-based sync enabled #1112

@rfsbraz

Description

@rfsbraz

Kobo sync consistently fails on a large library (~13,000 books) because the sync endpoint takes 30-45+ seconds to respond, exceeding the Kobo e-reader's ~30-second HTTP timeout. This happens even when "Only sync books in selected shelves" is enabled with a
shelf containing only 63 books.

Environment

  • CWA version: v4.0.6
  • Kobo model: Libra Colour (firmware 4.44)
  • Reverse proxy: Traefik
  • Library size: ~13,000 books
  • Kobo sync shelf: 63 books
  • kobo_only_shelves_sync: enabled
  • config_kobo_proxy: enabled

Evidence

Traefik access logs show every sync request returns 499 (client disconnect) after ~30s:

"GET /kobo/.../v1/library/sync?Filter=ALL&..." HTTP/1.1" 499 21 ... 29895ms
"GET /kobo/.../v1/library/sync?Filter=ALL&..." HTTP/1.1" 499 21 ... 29712ms
"GET /kobo/.../v1/library/sync?Filter=ALL&..." HTTP/1.1" 499 21 ... 30301ms
"GET /kobo/.../v1/library/sync?Filter=ALL&..." HTTP/1.1" 499 21 ... 30222ms

Requests made from curl (which waits longer) complete successfully with HTTP 200 after 40-80s.

CWA debug logs show each sync step takes 10-13s even for a single changed book:

09:39:31.106 - Kobo library sync request received
09:39:31.205 - Kobo Sync: books last modified (0.1s)
09:39:44.512 - Kobo Sync: changed entries: 1 (13s to scan!)
09:39:55.314 - Kobo Sync: selected to sync: 1 (11s)
09:40:05.344 - Kobo Sync: cover processing (10s)
09:40:15.821 - Kobo Sync: remaining books to sync: 0 (10s)
09:40:15.830 - Kobo Sync: changed states: 1

Total: ~45 seconds to sync 1 book.

The bottleneck is database scanning, not conversion or network

  • All books are pre-converted to kepub (ran kepubify --update --inplace --calibre)
  • No on-the-fly conversion happening during sync
  • Each query step appears to scan the full 13K book database rather than being filtered by the sync shelf

Expected behavior

With shelf-based sync enabled (63 books), the sync queries should be scoped to only those 63 books, completing well within the Kobo's ~30-second timeout.

Steps to reproduce

  1. Set up CWA with a large Calibre library (10K+ books)
  2. Enable Kobo sync with shelf-based filtering on a small shelf
  3. Attempt to sync from a Kobo e-reader
  4. Observe "Sync failed" on the Kobo due to timeout

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions