docs: improve examples, add new & updated working-with-data doc, reference aw-client examples better

Makefile

@@ -56,6 +56,7 @@ help:
 	@echo "  linkcheck  to check all external links for integrity"
 	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
 	@echo "  coverage   to run coverage check of the documentation (if enabled)"
+	@echo "  redirectcheck to check that deleted files have proper redirects"
 	@echo "  dummy      to check syntax errors of document sources"
 
 .PHONY: clean
@@ -243,8 +244,14 @@ pseudoxml:
 	@echo
 	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
 
+.PHONY: redirectcheck
+redirectcheck:
+	$(SPHINXBUILD) -b rediraffecheckdiff $(ALLSPHINXOPTS) $(BUILDDIR)/rediraffe
+	@echo
+	@echo "Redirect check complete."
+
 .PHONY: dummy
 dummy:
 	$(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy
 	@echo
-	@echo "Build finished. Dummy builder generates no files."
+	@echo "Build finished. Dummy builder generates no files."

pyproject.toml

@@ -25,6 +25,7 @@ m2r2 = "*"
 myst-parser = "*"
 recommonmark = "^0.7.0"
 setuptools = "*"  # needed in Python 3.12+: https://github.com/CrossNox/m2r2/issues/72
+sphinxext-rediraffe = "^0.2.7"
 
 [build-system]
 requires = ["poetry-core"]

src/conf.py

@@ -41,6 +41,7 @@
     "sphinx.ext.extlinks",
     "sphinx_tabs.tabs",
     "sphinx_click",
+    "sphinxext.rediraffe",
 ]
 
 extlinks = {
@@ -50,6 +51,10 @@
     "gh-aw": ("https://github.com/ActivityWatch/%s", ""),
 }
 
+# Redirects for moved pages
+rediraffe_redirects = "redirects.txt"
+rediraffe_branch = "HEAD~1"
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
@@ -391,4 +396,4 @@
 
 # If true, do not generate a @detailmenu in the "Top" node's menu.
 #
-# texinfo_no_detailmenu = False
+# texinfo_no_detailmenu = False

src/examples.rst

@@ -1,9 +1,54 @@
 Examples
 ========
 
-For more examples, see the examples in the aw-client repo: https://github.com/ActivityWatch/aw-client
+This section provides practical examples for working with ActivityWatch, from retrieving your data to extending functionality with custom watchers.
+
+Getting Your Data Out
+----------------------
+
+**Most users should start here:**
+
+.. toctree::
+    :maxdepth: 1
+
+    examples/working-with-data
+
+This comprehensive guide covers:
+
+* **Canonical Events** - Get processed activity data (what the web UI uses)
+* **Custom Queries** - Write your own analysis using the query language
+* **Raw Events** - Advanced direct access to bucket data
+* **Safety Best Practices** - Avoiding data corruption with proper testing and dry-run modes
+
+The guide includes links to production-ready examples from the `aw-client repository <https://github.com/ActivityWatch/aw-client/tree/master/examples>`_.
+
+Writing Your Own Watchers
+-------------------------
+
+Want to collect custom data? **See:**
 
 .. toctree::
-    examples/querying-data
+    :maxdepth: 1
+
     examples/writing-watchers
-    examples/extending
+
+These guides cover:
+
+* **Minimal watcher example** - Get started with a simple template
+* **Full-featured watcher** - Complete example with heartbeats and proper structure
+* **Best practices** - Error handling, bucket management, and testing modes
+* **Rust examples** - Alternative implementation for performance-critical watchers
+
+Watchers are small programs that collect data and send it to ActivityWatch. You can track anything with a timestamp!
+
+Example Code Repository
+-----------------------
+
+The `aw-client examples <https://github.com/ActivityWatch/aw-client/tree/master/examples>`_ contain comprehensive, well-documented examples including:
+
+* **Time analysis** - ``time_spent_today.py``, ``working_hours.py``
+* **Data export** - ``load_dataframe.py`` for pandas integration
+* **Data management** - ``redact_sensitive.py`` with safe dry-run mode
+* **Advanced analysis** - ``suggest_categories.py`` with AI categorization
+
+All examples follow safety best practices with testing modes and error handling.

src/examples/querying-data.rst

@@ -1,92 +1,27 @@
-Querying Data
-=============
-
-There are a couple of ways to query data in ActivityWatch.
-
-aw-server supplies a "/query" endpoint (also accessible via aw-client's query method) which supplies a basic scripting language which you can utilize to do transformations on the server-side.
-This option is good for basic analysis and for lightweight clients (such as aw-webui).
-
-Another option is to fetch events from the "/buckets/bucketname/events" endpoint (also accessible via aw-client's get_events method) and either program your own transformations or use transformation methods available in the aw-analysis python library (which includes all transformations available in the query endpoint). This requires a lot of more work since you will likely have to reprogram transformations already available in the query API, but on the other hand it is much more flexible.
-
-
-Writing a Query
----------------
+Querying Data (Moved)
+====================
 
 .. note::
-    This section is still WIP.
-    There is still no documentation of all the transform functions, but for most simple queries these examples should be enough.
-
-Queries are the easiest yet advanced way to get events from aw-server buckets in a format which fits most needs.
-Queries can be done by doing a POST request to aw-server either manually or with the aw-client library.
-
-For an incomplete API reference of the transform functions, see the API reference for :py:mod:`aw_transform` and :py:mod:`aw_query`.
-
-In a query you start by getting events from a bucket and assign that collection of events to a variable, then there are multiple transform functions which you can use to for example filter, limit, sort, and merge events from a bucket.
-After that you assign what you want to receive from the request to the RETURN variable.
-
-Magic Variables:
-    There is a magic variable ``__CATEGORIES__`` you can use in the web UI's Query Explorer to include your configured categories in your query.
-
-    Here's an example of using this variable to find all events categorized as "Web Browsing"
-
-    .. code-block:: python
-
-        events = flood(query_bucket(find_bucket("aw-watcher-window_")));
-        not_afk = flood(query_bucket(find_bucket("aw-watcher-afk_")));
-        not_afk = filter_keyvals(not_afk, "status", ["not-afk"]);
-        events = filter_period_intersect(events, not_afk);
-        events = categorize(events, __CATEGORIES__);
-        events = filter_keyvals(events, "$category", [["Work"]]);
-        RETURN = sort_by_duration(events);
-
-Minimal example:
-    Minimal query which only gets events from a bucket and returns it:
-
-    .. code-block:: bash
-
-        events = query_bucket("my_bucket");
-        RETURN = events;
-
-
-Example which arranges a hierarchy:
-    A query which merges events from a bucket in a key1->key2 hierarchy:
-
-    .. code-block:: bash
-
-        events = query_bucket("my_bucket");
-        events = merge_events_by_keys(events, "merged_key1", "merged_key2");
-        RETURN = events;
-
-
-Example combining window and AFK events:
-    A simplified query example of how to summarize what programs used while not afk.
-    The query intersects the not-afk events from the afk bucket with the events from the window bucket, merges keys from the result and sorts by duration.
-
-    .. code-block:: bash
-
-        afk_events = query_bucket(find_bucket("aw-watcher-afk_"));
-        window_events = query_bucket(find_bucket("aw-watcher-window_"));
-        window_events = filter_period_intersect(window_events, filter_keyvals(afk_events, "status", ["not-afk"]));
-        merged_events = merge_events_by_keys(window_events, ["app", "title"]);
-        RETURN = sort_by_duration(merged_events);
-
-Example including aw-client:
-    This is an example of how you can do analysis and aggregation with the query method in Python with aw-client.
-    You probably need to install the client library by following the instructions in its `repository <https://github.com/ActivityWatch/aw-client>`_.
-
-	.. note:: This example runs the client in *testing* mode, which means that it will try to connect to an aw-server in testing mode on the port 5666 instead of the normal 5600.
+   This page has been moved and significantly improved.
+
+   **Please update your bookmarks to:** :doc:`working-with-data`
 
-    .. literalinclude:: query_client.py
+This page has been replaced by a more comprehensive guide: :doc:`working-with-data`
 
-Fetching Raw Events
--------------------
-It is possible to fetch the raw events from a bucket. This is useful if you want to do your own analysis on the data, or if you want to use the aw-analysis library to do transformations on the data.
+The new page covers:
 
-Example fetching raw events from the "aw-watcher-window_" bucket:
-    This is an example that you can run in a Python to fetch raw events posted by the window watcher.
-    The scripts sums the time spent on each window title and showcases a data redaction use case.
+* **Canonical Events** (recommended) - Get processed activity data like the web UI
+* **Custom Queries** - Write your own analysis queries  
+* **Raw Events** (advanced) - Direct bucket access for maximum flexibility
+* **Comprehensive Examples** - Safe, up-to-date examples from the aw-client repository
 
-    .. literalinclude:: raw_events.py
+**Automatic redirect:** You will be redirected in 5 seconds.
 
+.. raw:: html
 
-.. TODO `Bucket REST API <./rest.html#get-events>`_
+   <script>
+   setTimeout(function() {
+       window.location.href = 'working-with-data.html';
+   }, 5000);
+   </script>
+   <meta http-equiv="refresh" content="5;url=working-with-data.html">