feat: allow watch progress restore via backup file

FAQ.md

@@ -203,7 +203,7 @@ No, webhooks are **not required** for the tool to function. You can use the buil
 
 > [!NOTE]
 > There are problems with jellyfin API, which are fixed by using webhooks, please check out
-> the [webhook guide](/guides/webhooks.md#media-backends-webhook-limitations) to learn more.
+> the [backend limitations guide](/guides/backend-limitations.md#jellyfin) to learn more.
 
 ---
 

config/config.php

@@ -232,7 +232,7 @@
         'file' => [
             'type' => 'stream',
             'enabled' => (bool) env('WS_LOGGER_FILE_ENABLE', true),
-            'level' => env('WS_LOGGER_FILE_LEVEL', Level::Warning),
+            'level' => env('WS_LOGGER_FILE_LEVEL', Level::Error),
             'filename' => ag($config, 'tmpDir') . '/logs/app.' . $logDateFormat . '.log',
         ],
         'stderr' => [

frontend/app/components/BackendAdd.vue

@@ -45,6 +45,17 @@
             </NuxtLink>
             to learn more.
           </li>
+          <li>
+            Each backend has specific requirements. Review the
+            <NuxtLink
+              to="/help/backend-limitations"
+              class="inline-flex items-center gap-1 text-primary"
+            >
+              <UIcon name="i-lucide-circle-help" class="size-4" />
+              <span>backend limitations</span>
+            </NuxtLink>
+            page before adding.
+          </li>
           <li v-if="api_user === 'main'">
             Do not add identity backends manually after finishing the main identity backend setup.
             Visit

frontend/app/pages/help/index.vue

@@ -125,5 +125,11 @@ const choices: Array<{ number: number; title: string; text: string; url: string
     text: 'How to enable path-based matching.',
     url: '/help/path-match',
   },
+  {
+    number: 11,
+    title: 'Backend limitations',
+    text: 'Requirements and known limitations for supported backends.',
+    url: '/help/backend-limitations',
+  },
 ];
 </script>

guides/backend-limitations.md

@@ -0,0 +1,62 @@
+# Backend Limitations
+
+Each backend has specific requirements and limitations to be aware of before adding it to WatchState.
+
+## Plex
+
+### Requirements
+
+- **PlexPass** subscription is required for webhooks.
+- An **admin-level token** is required if you plan to [provision identities](/guides/identities.md).
+- Plex version **1.41.6.9606 or newer** is recommended. Older versions have a bug where marking an item as watched without prior progress does not show it in continue-watching [[reference](https://forums.plex.tv/t/continue-watching-is-buggy-unable-to-figure-out-why/869224/65)].
+
+### Limitations
+
+- **Limited tokens** cannot list or impersonate users. The users list will be empty, and you must enter the user UUID manually.
+- **.plex.direct URLs** often fail in Docker. Add your custom domain via **Plex Settings > Network > Custom server access URLs**.
+- Only **movie** and **show** libraries are imported for play state. Music, photos, and other library types are skipped.
+
+### Webhook Limitations
+
+- Plex does not send events for *marked as played/unplayed* actions.
+- Webhook events may be **skipped** when multiple items are added at once.
+- When items are marked as **unwatched**, Plex resets the date on the media object.
+- Plex does not send watch progress update events during playback. It only sends progress updates during `play`, `pause`, `stop`, and `resume` events, so progress data from Plex will not be reflected until one of those events triggers.
+
+### Plex via Tautulli
+
+- **Marking items as unplayed** is not reliable, as Tautulli's webhook payload lacks the data needed to detect this change.
+- Similarly to Plex, Tautulli does not send watch progress update events during playback.
+
+## Jellyfin
+
+### Requirements
+
+- **v10.9.x or higher** is required for watch progress sync.
+- The **Notifications > Webhook** plugin (free) must be installed from the plugin catalog for webhook support.
+- An **API key** is required if you plan to [provision identities](/guides/identities.md). The `username:password` format will not work for identity provisioning.
+
+### Limitations
+
+- Only **movie** and **show** libraries are imported for play state. Music, photos, and other library types are skipped.
+
+### Webhook Limitations
+
+- Even if a user ID is selected, Jellyfin may **still send events without user data**.
+- Items may be marked as **unplayed** if the setting *Libraries > Display > Date Added Behavior for New Content: Use Date Scanned into Library* is enabled. This can happen when media files are replaced or updated.
+
+### Known Issues
+
+- **Played without date bug**: Jellyfin marks items as played without updating the `LastPlayedDate`, causing export conflicts. The recommanded approch is to enable webhooks. However, if you prefer scheduled imports, you can enable the experimental `WS_CLIENTS_JELLYFIN_FIX_PLAYED` environment variable as a workaround, then run `state:import`.
+
+## Emby
+
+### Requirements
+
+- **Emby Premiere** subscription is required for webhooks.
+- An **API key** is required if you plan to [provision identities](/guides/identities.md). The `username:password` format will not work for identity provisioning.
+
+### Limitations
+
+- Only **movie** and **show** libraries are imported for play state. Music, photos, and other library types are skipped.
+- The **webhook test event** previously contained no data, but this issue appears to be fixed in version `4.9.0.37+`. To verify if your Emby webhook setup works, try playing or marking an item as played/unplayed and check if the changes appear in the database.

guides/webhooks.md

@@ -232,38 +232,8 @@ Click *Save*.
 
 # Media Backends Webhook Limitations
 
-Here are some known limitations and issues when using webhooks with different media backends:
-
-### Plex
-
-- Plex doesn't send events for *marked as played/unplayed* actions.
-- Webhook events may be **skipped** when multiple items are added at once.
-- When items are marked as **unwatched**, Plex resets the date on the media object.
-- In old version of plex i.e. pre `1.41.6.9606` marking items as watched and if you didn't have progress on the show
-  will not show the item in continue watching, this is a limitation of the old plex version and not
-  watchstate. [reference](https://forums.plex.tv/t/continue-watching-is-buggy-unable-to-figure-out-why/869224/65)
-- Plex doesn't send watch progress update events during playback, it only sends the progress update during `play`,
-  `pause`, `stop`, `resume` events. So the progress update from plex will not be reflected until one of those events
-  kicks in.
-
-### Plex via Tautulli
-
-- **Marking items as unplayed** is not reliable, as Tautulli’s webhook payload lacks the data needed to detect this
-  change.
-- Similarly to plex, Tautulli doesn't send watch progress update events during playback.
-
-### Emby
-
-- The **webhook test event** previously contained no data, but this issue appears to be fixed in version `4.9.0.37+`.
-    - To verify if your Emby webhook setup works, try playing or marking an item as played/unplayed, and check if the
-      changes appear in the database.
-
-### Jellyfin
-
-- Even if a user ID is selected, Jellyfin may **still send events without user data**.
-- Items may be marked as **unplayed** if the setting *Libraries > Display > Date Added Behavior for New Content: Use
-  Date Scanned into Library* is enabled.
-    - This can happen when media files are replaced or updated.
+See the [backend limitations](/guides/backend-limitations.md) for a comprehensive list of per-backend requirements 
+and limitations, including webhook-specific event behaviour.
 
 # Sometimes Newly Added Content Does Not Show Up
 
@@ -272,7 +242,3 @@ complement webhook functionality.
 
 Simply go to the *Tasks* page and enable the *Import* and *Export* tasks. and set the schedule to `every 12 hours` or
 `every 24 hours` depending on your needs.
-
-# Troubleshooting
-
-TBA

src/Commands/Backend/RestoreCommand.php

@@ -89,6 +89,7 @@ protected function configure(): void
                 InputOption::VALUE_NONE,
                 'Send one request at a time instead of all at once. note: Slower but more reliable.',
             )
+            ->addOption('restore-watch-progress', 'P', InputOption::VALUE_NONE, 'Restore the watch progress from backup as well.')
             ->addOption(
                 'async-requests',
                 null,
@@ -132,6 +133,8 @@ protected function runCommand(iInput $input, iOutput $output): int
      */
     protected function process(iInput $input, iOutput $output): int
     {
+        $this->queue->reset();
+
         $userName = $input->getOption('user');
         if (empty($userName)) {
             $output->writeln(r('<error>ERROR: User not specified. Please use [-u, --user].</error>'));
@@ -266,6 +269,7 @@ protected function process(iInput $input, iOutput $output): int
 
         $opts = [
             Options::IGNORE_DATE => true,
+            Options::REPLAY_PROGRESS => true === $input->getOption('restore-watch-progress'),
             Options::DEBUG_TRACE => true === $input->getOption('trace'),
             Options::DRY_RUN => false === $input->getOption('execute'),
         ];
@@ -292,64 +296,163 @@ protected function process(iInput $input, iOutput $output): int
             $syncRequests = false;
         }
 
-        $requests = $backend->export($mapper, $this->queue, null);
-
-        $start = microtime(true);
-        $this->logger->notice("SYSTEM: Sending '{total}' play state comparison requests for '{user}@{backend}'.", [
-            'backend' => $name,
-            'total' => count($requests),
-            'user' => $userContext->name,
-        ]);
-
-        send_requests(requests: $requests, client: $this->http, sync: $syncRequests, logger: $this->logger);
-
-        $this->logger->notice("SYSTEM: Completed '{total}' requests in '{duration}'s for '{user}@{backend}'.", [
-            'backend' => $name,
-            'total' => count($requests),
-            'user' => $userContext->name,
-            'duration' => round(microtime(true) - $start, 4),
-        ]);
+        try {
+            $requests = $backend->export($mapper, $this->queue, null);
 
-        $total = count($this->queue->getQueue());
+            $stats = ['sent' => count($requests), 'failed' => 0];
 
-        if ($total >= 1) {
-            $this->logger->notice("SYSTEM: Sending '{total}' change state requests for '{user}@{backend}'.", [
+            $start = microtime(true);
+            $this->logger->notice("SYSTEM: Sending '{total}' play state comparison requests for '{user}@{backend}'.", [
                 'backend' => $name,
+                'total' => $stats['sent'],
                 'user' => $userContext->name,
-                'total' => $total,
             ]);
-        }
 
-        if ((int) Message::get("{$userContext->name}.{$name}.export", 0) < 1) {
-            $this->logger->notice("SYSTEM: No difference detected between backup file and '{user}@{backend}'.", [
+            send_requests(
+                requests: $requests,
+                client: $this->http,
+                sync: $syncRequests,
+                logger: $this->logger,
+                opts: [
+                    'error' => static function () use (&$stats): array {
+                        $stats['failed']++;
+                        return [];
+                    },
+                ],
+            );
+
+            $this->logger->notice("SYSTEM: Completed '{total}' requests in '{duration}'s for '{user}@{backend}'.", [
                 'backend' => $name,
+                'total' => $stats['sent'],
                 'user' => $userContext->name,
+                'duration' => round(microtime(true) - $start, 4),
             ]);
-        }
 
-        if ($total < 1 || false === $input->getOption('execute')) {
-            return self::SUCCESS;
-        }
+            $total = count($this->queue->getQueue());
+            $sendStats = ['sent' => $total, 'failed' => 0];
 
-        send_requests(
-            requests: $this->queue->getQueue(),
-            client: $this->http,
-            sync: $syncRequests,
-            logger: $this->logger,
-        );
+            if ($total >= 1) {
+                $this->logger->notice("SYSTEM: Sending '{total}' change state requests for '{user}@{backend}'.", [
+                    'backend' => $name,
+                    'user' => $userContext->name,
+                    'total' => $total,
+                ]);
+            }
 
-        $this->logger->notice(
-            "SYSTEM: Sent '{total}' change play state requests to '{client}: {user}@{backend}' in '{duration}'s.",
-            [
-                'total' => $total,
-                'backend' => $name,
-                'user' => $userContext->name,
-                'client' => $backend->getContext()->clientName,
-                'duration' => round(microtime(true) - $opStart, 4),
-            ],
-        );
+            if ((int) Message::get("{$userContext->name}.{$name}.export", 0) < 1) {
+                $this->logger->notice("SYSTEM: No difference detected between backup file and '{user}@{backend}'.", [
+                    'backend' => $name,
+                    'user' => $userContext->name,
+                ]);
+            }
+
+            if ($total >= 1 && false !== $input->getOption('execute')) {
+                send_requests(
+                    requests: $this->queue->getQueue(),
+                    client: $this->http,
+                    sync: $syncRequests,
+                    logger: $this->logger,
+                    opts: [
+                        'error' => static function () use (&$sendStats): array {
+                            $sendStats['failed']++;
+                            return [];
+                        },
+                    ],
+                );
+
+                $this->logger->notice(
+                    "SYSTEM: Sent '{total}' change play state requests to '{client}: {user}@{backend}' in '{duration}'s.",
+                    [
+                        'total' => $total,
+                        'backend' => $name,
+                        'user' => $userContext->name,
+                        'client' => $backend->getContext()->clientName,
+                        'duration' => round(microtime(true) - $opStart, 4),
+                    ],
+                );
+            }
+
+            $progressStats = ['sent' => 0, 'failed' => 0];
+
+            if (true === $input->getOption('restore-watch-progress')) {
+                $this->queue->reset();
+                $backend->progress($mapper->getObjects(), $this->queue, null);
+
+                $progressStats['sent'] = count($this->queue->getQueue());
+
+                if ($progressStats['sent'] >= 1) {
+                    $this->logger->notice("SYSTEM: Sending '{total}' watch progress requests for '{user}@{backend}'.", [
+                        'backend' => $name,
+                        'user' => $userContext->name,
+                        'total' => $progressStats['sent'],
+                    ]);
+                } else {
+                    $this->logger->notice("SYSTEM: No watch progress changes detected between backup file and '{user}@{backend}'.", [
+                        'backend' => $name,
+                        'user' => $userContext->name,
+                    ]);
+                }
+
+                if ($progressStats['sent'] >= 1 && false !== $input->getOption('execute')) {
+                    send_requests(
+                        requests: $this->queue->getQueue(),
+                        client: $this->http,
+                        sync: $syncRequests,
+                        logger: $this->logger,
+                        opts: [
+                            'error' => static function () use (&$progressStats): array {
+                                $progressStats['failed']++;
+                                return [];
+                            },
+                        ],
+                    );
+
+                    $this->logger->notice(
+                        "SYSTEM: Sent '{total}' watch progress requests to '{client}: {user}@{backend}' in '{duration}'s.",
+                        [
+                            'total' => $progressStats['sent'],
+                            'backend' => $name,
+                            'user' => $userContext->name,
+                            'client' => $backend->getContext()->clientName,
+                            'duration' => round(microtime(true) - $opStart, 4),
+                        ],
+                    );
+                }
+            }
+
+            if ($stats['failed'] > 0 || $sendStats['failed'] > 0 || $progressStats['failed'] > 0) {
+                $this->logger->warning(
+                    "Restore completed with item-level request failures for '{user}@{backend}'.",
+                    [
+                        'user' => $userContext->name,
+                        'backend' => $name,
+                        'comparison' => $stats,
+                        'apply' => $sendStats,
+                        'progress' => $progressStats,
+                    ],
+                );
+            }
 
-        return self::SUCCESS;
+            return self::SUCCESS;
+        } catch (Throwable $e) {
+            $this->logger->error(
+                "SYSTEM: Unhandled exception '{error.kind}' was thrown during '{user}@{backend}' restore operation. '{error.message}' at '{error.file}:{error.line}'.",
+                [
+                    'user' => $userContext->name,
+                    'backend' => $name,
+                    'error' => [
+                        'kind' => $e::class,
+                        'line' => $e->getLine(),
+                        'message' => $e->getMessage(),
+                        'file' => after($e->getFile(), ROOT_PATH),
+                    ],
+                ],
+            );
+
+            return self::FAILURE;
+        } finally {
+            $this->queue->reset();
+        }
     }
 
     /**