Sync resources between android instances, ditch DHT

Author: joepio

.claude/settings.local.json

@@ -78,7 +78,16 @@
       "Read(//opt/flutter/bin/**)",
       "Read(//Users/joep/**)",
       "Bash(mise which:*)",
-      "Bash(mise ls:*)"
+      "Bash(mise ls:*)",
+      "Bash(export PATH=\"/Users/joep/.local/share/mise/installs/flutter/3.22.1-stable/bin:$PATH\")",
+      "Bash(flutter build:*)",
+      "Bash(cargo doc:*)",
+      "Bash(grep -rn \"pub fn.*change\\\\|pub fn.*history\\\\|pub fn.*oplog\\\\|get_all_change\" /Users/joep/.cargo/registry/src/*/loro-1.10.8/src/)",
+      "WebSearch",
+      "Bash(find /Users/joep/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/simple-dns-0.9.3/ -name \"*.rs\" -exec grep -l \"enum RData\" {} \\\\;)",
+      "Bash(cargo test *)",
+      "WebFetch(domain:docs.iroh.computer)",
+      "Bash(grep -v \"^W/atomic\\\\|^$\")"
     ]
   }
 }

AGENTS.md

@@ -33,11 +33,12 @@ In E2E tests, most specs use `test.beforeEach(before)` from `test-utils.ts`, whi
 Atomic Server is a graph database with real-time sync, built on **Loro CRDT** for conflict-free collaborative editing.
 
 ### Crates
-- **`atomic_lib`** (`lib/`) — Core library. WASM-compatible (no `ring`, no `rt-multi-thread`). Contains Resource, Commit, Store, Loro integration, WS client, connected Client API.
-- **`atomic-server`** (`server/`) — Actix-web HTTP/WS server. Uses `atomic_lib` + sled DB + search (tantivy).
+- **`atomic_lib`** (`lib/`) — Core library. WASM-compatible (no `ring`, no `rt-multi-thread`). Contains Resource, Commit, Store, Loro integration, Iroh P2P sync, WS client, connected Client API.
+- **`atomic-server`** (`server/`) — Actix-web HTTP/WS server. Uses `atomic_lib` + redb + search (tantivy).
 - **`@tomic/lib`** (`browser/lib/`) — TypeScript client library.
 - **`@tomic/react`** (`browser/react/`) — React hooks.
 - **`data-browser`** (`browser/data-browser/`) — The web app (React + TipTap + Loro).
+- **`flutter/`** — Cross-platform canvas app (Android/iOS/Web). Uses `flutter_rust_bridge` to call `atomic_lib`. See `flutter/README.md` and `flutter/AGENTS.md`.
 
 ### Data model
 - **Resource** = property-value pairs with a Subject URL, backed by a Loro CRDT document.
@@ -139,12 +140,41 @@ Real-time: `useLoroSync` hook → `LORO_SYNC_UPDATE` WebSocket.
 
 Loro OpLog time-travel: `doc.getAllChanges()` → sort → `doc.checkout(frontiers)` per version. Instant, no network round-trips.
 
+## Iroh P2P Sync
+
+Devices sync via [Iroh](https://iroh.computer) QUIC connections. The transport is in `lib/src/sync/`:
+
+- **`peer.rs`** — Iroh endpoint, Router (must stay alive for incoming connections), persistent NodeID (secret key stored in redb), known peers list.
+- **`engine.rs`** — Transport-agnostic sync engine. Compares Loro version vectors, computes diffs, imports snapshots. Used by both WS and Iroh.
+- **`protocol.rs`** — Binary frame encoding: AUTH, SYNC, SYNC_DIFF, SYNC_PUSH, SYNC_OK, GET, UPDATE.
+
+### Sync flow (QR pairing)
+1. Both devices start Iroh (`peer::start()`) → get persistent NodeID, connect to n0 relay
+2. Device A shows QR code containing `did:ad:node:<nodeId>`
+3. Device B scans QR → calls `peer_sync(nodeId)` → `sync_drive_with_peer()`
+4. B→A: AUTH, SYNC (with B's version vectors)
+5. A→B: SYNC_DIFF (what to push/pull), SYNC_PUSH (A's data)
+6. B→A: SYNC_PUSH (B's data for A's pull list)
+7. Both devices now have each other's data
+
+### Key details
+- The `Router` must be kept alive globally (`ROUTER` static) — dropping it stops incoming connections.
+- After sending the final SYNC_PUSH, call `send.finish()` + short delay so the server processes it before the connection drops.
+- Loro snapshots are stored in `Tree::LoroSnapshots` keyed by `Subject::pure_id()` (strips query params/drive hints).
+- `collect_drive_subjects()` and `build_drive_vvs()` must use `pure_id()` consistently to match snapshot keys.
+
+### Node identity
+- `did:ad:node:<hex>` — URI format for Iroh NodeIDs, used in QR codes and UI.
+- NodeIDs are persistent — derived from a secret key stored in redb (`Tree::PluginMeta`).
+- Known peers are also stored in `Tree::PluginMeta` as a JSON array.
+
 ## Testing
 
 ```
 cargo test -p atomic_lib --no-default-features  # 76 tests
 cargo test -p atomic-server --lib               # 23 tests
 cargo test -p atomic-server --test sync          # E2E: real server, 2 agents, WS sync
+cargo test -p atomic_lib --features "iroh,discovery,db-redb" --lib -- sync::tests  # Iroh sync tests
 cd browser/lib && pnpm test                      # 29 JS tests
 cd browser && pnpm run -r build                  # Full workspace build
 ```
@@ -157,3 +187,6 @@ cd browser && pnpm run -r build                  # Full workspace build
 4. **CSP**: Includes `'wasm-unsafe-eval'` for Loro WASM in browser.
 5. **`build.rs`**: Watches `lib/src`, `react/src`, `data-browser/src`. Delete `server/assets_tmp` to force JS rebuild.
 6. **`CommitBuilder.push_propval`**: Starts from empty, not from resource's existing value. Load current value first if appending.
+7. **`flutter/rust/`** is excluded from the Cargo workspace (`Cargo.toml` `exclude`). It has its own `Cargo.toml` with a path dep to `../../lib`.
+8. **NDK 27 `llvm-strip`** corrupts large `.so` files. Debug builds use `doNotStrip '**/*.so'` in both `app/build.gradle` and `rust_builder/android/build.gradle`.
+9. **Iroh Router lifetime**: `peer::start()` returns a Router that must be kept alive. Dropping it silently stops incoming connections — no error, just "failed connecting to remote endpoint" on the other side.

Cargo.lock

@@ -9,5 +9,8 @@ members = [
   "plugin-examples/test-plugin",
   "atomic-plugin",
 ]
+exclude = [
+  "flutter/rust",
+]
 
 # Tauri build is deprecated, see

Cargo.toml

@@ -419,6 +419,7 @@ msgstr "Kein Code-Verifizierer gefunden"
 
 #: src/components/forms/SearchBox/SearchBox.tsx
 #: src/routes/LinkOpenRouter.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Error"
 msgstr "Fehler"
 
@@ -765,10 +766,14 @@ msgstr "Akzeptieren als {0}"
 
 #. placeholder {0}: JSON.stringify(error)
 #. placeholder {0}: searchError.message
+#. placeholder {0}: data.error
+#. placeholder {0}: e
 #. placeholder {0}: resource.error.message
 #. placeholder {0}: resource.error.message
 #: src/components/forms/Field.tsx
 #: src/components/forms/SearchBox/SearchBoxWindow.tsx
+#: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 #: src/views/ResourceLine.tsx
 #: src/views/ResourceRow.tsx
 msgid "Error: {0}"
@@ -4749,6 +4754,8 @@ msgstr ""
 
 #: src/components/SideBar/SyncMenuItem.tsx
 #: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Sync"
 msgstr ""
 
@@ -4853,6 +4860,7 @@ msgstr ""
 
 #: src/components/SideBar/SyncMenuItem.tsx
 #: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Syncing..."
 msgstr ""
 
@@ -5040,6 +5048,7 @@ msgstr ""
 msgid "Connected to {0}"
 msgstr ""
 
+#: src/routes/SyncRoute.tsx
 #: src/routes/SyncRoute.tsx
 msgid "+ Add"
 msgstr ""
@@ -5057,6 +5066,41 @@ msgstr ""
 msgid "How to run your own server"
 msgstr ""
 
+#~ msgid "Peer ID"
+#~ msgstr ""
+
+#. placeholder {0}: data.count
+#. placeholder {1}: data.count !== 1 ? 's' : ''
+#: src/routes/SyncRoute.tsx
+msgid "Synced {0} resource{1}"
+msgstr ""
+
+#~ msgid "Peer sync"
+#~ msgstr ""
+
+#~ msgid "Paste device ID to sync with"
+#~ msgstr ""
+
+#: src/routes/SyncRoute.tsx
+msgid "Peers"
+msgstr ""
+
+#: src/routes/SyncRoute.tsx
+msgid "No peers connected"
+msgstr ""
+
+#~ msgid "Paste device ID"
+#~ msgstr ""
+
+#: src/routes/SyncRoute.tsx
+msgid "Node DID"
+msgstr ""
+
+#. placeholder {0}: irohNodeId.slice(0, 12)
+#: src/routes/SyncRoute.tsx
+msgid "did:ad:node:{0}..."
+msgstr ""
+
 #: src/routes/SyncRoute.tsx
-msgid "Peer ID"
+msgid "Paste did:ad:node:..."
 msgstr ""

browser/data-browser/src/locales/de.po

@@ -23,10 +23,14 @@ msgstr "Upload"
 
 #. placeholder {0}: JSON.stringify(error)
 #. placeholder {0}: searchError.message
+#. placeholder {0}: data.error
+#. placeholder {0}: e
 #. placeholder {0}: resource.error.message
 #. placeholder {0}: resource.error.message
 #: src/components/forms/Field.tsx
 #: src/components/forms/SearchBox/SearchBoxWindow.tsx
+#: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 #: src/views/ResourceLine.tsx
 #: src/views/ResourceRow.tsx
 msgid "Error: {0}"
@@ -1895,6 +1899,7 @@ msgstr "No code verifier found"
 
 #: src/components/forms/SearchBox/SearchBox.tsx
 #: src/routes/LinkOpenRouter.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Error"
 msgstr "Error"
 
@@ -4709,6 +4714,8 @@ msgstr "Failed to decode Loro snapshot"
 
 #: src/components/SideBar/SyncMenuItem.tsx
 #: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Sync"
 msgstr "Sync"
 
@@ -4815,6 +4822,7 @@ msgstr "In sync"
 
 #: src/components/SideBar/SyncMenuItem.tsx
 #: src/routes/SyncRoute.tsx
+#: src/routes/SyncRoute.tsx
 msgid "Syncing..."
 msgstr "Syncing..."
 
@@ -5004,6 +5012,7 @@ msgstr "Running in offline mode. Reconnect in the sync menu."
 msgid "Connected to {0}"
 msgstr "Connected to {0}"
 
+#: src/routes/SyncRoute.tsx
 #: src/routes/SyncRoute.tsx
 msgid "+ Add"
 msgstr "+ Add"
@@ -5021,6 +5030,41 @@ msgstr "Server settings have moved to the{0} <0>Sync page</0>."
 msgid "How to run your own server"
 msgstr "How to run your own server"
 
+#~ msgid "Peer ID"
+#~ msgstr "Peer ID"
+
+#. placeholder {0}: data.count
+#. placeholder {1}: data.count !== 1 ? 's' : ''
+#: src/routes/SyncRoute.tsx
+msgid "Synced {0} resource{1}"
+msgstr "Synced {0} resource{1}"
+
+#~ msgid "Peer sync"
+#~ msgstr "Peer sync"
+
+#~ msgid "Paste device ID to sync with"
+#~ msgstr "Paste device ID to sync with"
+
+#: src/routes/SyncRoute.tsx
+msgid "Peers"
+msgstr "Peers"
+
+#: src/routes/SyncRoute.tsx
+msgid "No peers connected"
+msgstr "No peers connected"
+
+#~ msgid "Paste device ID"
+#~ msgstr "Paste device ID"
+
+#: src/routes/SyncRoute.tsx
+msgid "Node DID"
+msgstr "Node DID"
+
+#. placeholder {0}: irohNodeId.slice(0, 12)
+#: src/routes/SyncRoute.tsx
+msgid "did:ad:node:{0}..."
+msgstr "did:ad:node:{0}..."
+
 #: src/routes/SyncRoute.tsx
-msgid "Peer ID"
-msgstr "Peer ID"
+msgid "Paste did:ad:node:..."
+msgstr "Paste did:ad:node:..."