added reverse image search

Author: bcherb2

README.md

@@ -19,29 +19,34 @@ https://github.com/user-attachments/assets/880c4caf-1367-4836-bc36-ef35524e1b2d
 https://github.com/user-attachments/assets/5ab6b907-5b08-4ef9-8ead-4b1b25510ce0
 
 
+#### reverse image search — upload a photo to find similar pages
+
 #### can use OPT files to speed up indexing
 #### can export your index files to backup or share with others
  - on the web gui with admin mode enabled, or via cli
 
 ## Quick Start
 
 1. Install [Docker](https://docs.docker.com/get-docker/)
-2. Configure:
+2. Clone the repo:
+   ```bash
+   git clone https://github.com/pdfiles/pdfiles.git
+   cd pdfiles
+   ```
+3. Configure:
    ```
    cp .env.example .env
    # Edit .env — set DATA_PATH to your documents folder
    ```
-3. Start:
+4. Start:
    ```bash
-   # Pull pre-built images (fastest)
-   docker compose pull && docker compose up -d
-
-   # Or build from source
-   ./pdfiles.sh deploy
+   ./pdfiles.sh up
    ```
-   On Windows: `pdfiles.bat deploy`
+   On Windows: `pdfiles.bat up`
+
+5. Open http://localhost
 
-4. Open http://localhost
+To stop: `./pdfiles.sh down` (or `pdfiles.bat down` on Windows)
 
 First startup downloads the search model (~4 GB) and takes 2-3 minutes.
 
@@ -50,12 +55,22 @@ First startup downloads the search model (~4 GB) and takes 2-3 minutes.
 | Command | Description |
 |---------|-------------|
 | `./pdfiles.sh up` | Start services |
+| `./pdfiles.sh update` | Pull latest images and restart |
 | `./pdfiles.sh down` | Stop services |
 | `./pdfiles.sh logs` | View logs |
 | `./pdfiles.sh status` | Health check |
-| `./pdfiles.sh backup` | Backup data |
+| `./pdfiles.sh backup` | Backup index to sqlite |
+| `./pdfiles.sh restore DIR` | Restore from a backup |
 | `./pdfiles.sh --help` | All commands |
 
+### A note on updates
+
+Qdrant data is stored in a docker volume that will persist `down` and `up`, reboots, etc. 
+Updates should not affect this, which you can do with `pdfiles update`, but if you have spent a long time indexing files, it is always safest to `pdfiles backup` first.
+
+**You do not need to reindex files to update**
+
+
 ## Requirements
 
 - Docker and Docker Compose
@@ -70,22 +85,13 @@ First startup downloads the search model (~4 GB) and takes 2-3 minutes.
 pdfiles scans through all of the pdfs that you mount in `DATA_PATH` and saves each page's essence in the form of vectors.  when you type something in search, these words get turned into vectors; and then the two sets of vectors get compared.  the resulting files are an ordered list of images that are closest to what you search.  this is what enables "find similar photos" as well.
 
 
-## Architecture Overview
+## Architecture
 
-```
-PDF Pages (517K)
-  |
-  v
-[Bouncer] ---> VISUAL / TEXT_ONLY / UNCERTAIN
-  |                (Tier 1: PyMuPDF text_ratio)
-  |                (Tier 2: Surya layout detection)
-  v
-[Indexer] ---> ColQwen2.5 embed ---> 2D block pool ---> Qdrant (262x128d multi-vectors)
-  |
-  v
-[Librarian] ---> mean-pool to 1x128d ---> K-Means ---> auto-label clusters
-  |
-  v
-[Search UI] ---> text query ---> ColQwen2.5 ---> MaxSim ---> ranked results
-                 cluster browse ---> gallery of representative pages
+```mermaid
+flowchart TD
+    PDF[PDF Pages] --> Bouncer --> Indexer
+    Indexer --> ColQwen2.5 --> Qdrant[(Qdrant)]
+
+    Query[Text / Image / Similar] --> ColQwen2.5
+    Qdrant -->|MaxSim| Results[Ranked Results]
 ```

pdfiles.bat

@@ -9,6 +9,7 @@ if "%COMMAND%"=="" set "COMMAND=help"
 
 if "%COMMAND%"=="deploy" goto :deploy
 if "%COMMAND%"=="up" goto :up
+if "%COMMAND%"=="update" goto :update
 if "%COMMAND%"=="down" goto :down
 if "%COMMAND%"=="logs" goto :logs
 if "%COMMAND%"=="status" goto :status
@@ -57,6 +58,15 @@ echo.
 echo Frontend: http://localhost:80
 goto :eof
 
+:update
+echo Pulling latest images...
+docker compose pull
+echo Restarting services...
+docker compose up -d
+echo.
+echo Frontend: http://localhost:80
+goto :eof
+
 :down
 echo Stopping services...
 docker compose down
@@ -83,6 +93,7 @@ echo.
 echo Commands:
 echo   deploy DATA_PATH    First-time setup (creates .env, builds, starts)
 echo   up                  Start services
+echo   update              Pull latest images and restart
 echo   down                Stop services
 echo   logs [SERVICE]      View logs
 echo   status              Health check

pdfiles.sh

@@ -128,6 +128,24 @@ cmd_up() {
     print_urls
 }
 
+cmd_update() {
+    local force_mode=""
+    for arg in "$@"; do
+        case "$arg" in
+            --cpu) force_mode="cpu" ;;
+            --gpu) force_mode="gpu" ;;
+        esac
+    done
+
+    local compose_file
+    compose_file=$(detect_compose_file "$force_mode")
+    echo "Pulling latest images..."
+    docker compose -f "$compose_file" pull
+    echo "Restarting services..."
+    docker compose -f "$compose_file" up -d
+    print_urls
+}
+
 cmd_down() {
     local clean=false
     for arg in "$@"; do
@@ -569,6 +587,7 @@ Usage: ./pdfiles.sh <command> [options]
 Commands:
   deploy [DATA_PATH]        First-time setup (creates .env, builds, starts)
   up [--build] [--cpu]      Start services
+  update [--cpu]            Pull latest images and restart
   down [--clean]            Stop services (--clean removes volumes)
   logs [SERVICE]            Tail logs
   status                    Health dashboard
@@ -598,6 +617,7 @@ shift 2>/dev/null || true
 case "$COMMAND" in
     deploy)  cmd_deploy "$@" ;;
     up)      cmd_up "$@" ;;
+    update)  cmd_update "$@" ;;
     down)    cmd_down "$@" ;;
     logs)    cmd_logs "$@" ;;
     status)  cmd_status "$@" ;;

src/pdfiles/api.py

@@ -1,3 +1,4 @@
+import asyncio
 import io
 import logging
 import threading
@@ -6,7 +7,7 @@
 from pathlib import Path
 
 import httpx
-from fastapi import FastAPI, HTTPException, Query
+from fastapi import FastAPI, HTTPException, Query, UploadFile, File
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import StreamingResponse
 
@@ -179,6 +180,40 @@ def search(
     ]
 
 
+@app.post("/api/search/image")
+async def search_by_image(
+    file: UploadFile = File(...),
+    top_k: int = Query(10, ge=1, le=500),
+):
+    ALLOWED_TYPES = {"image/jpeg", "image/png"}
+    if file.content_type not in ALLOWED_TYPES:
+        raise HTTPException(status_code=415, detail=f"Unsupported file type: {file.content_type}. Use JPEG or PNG.")
+
+    MAX_SIZE = 10 * 1024 * 1024  # 10 MB
+    data = await file.read()
+    if len(data) > MAX_SIZE:
+        raise HTTPException(status_code=413, detail="File too large (max 10 MB)")
+
+    from PIL import Image
+
+    image = Image.open(io.BytesIO(data)).convert("RGB")
+    results = await asyncio.to_thread(_searcher.search_by_image, image, top_k=top_k)
+    return [
+        {
+            "page_id": r.page_id,
+            "pdf_path": r.pdf_path,
+            "source_path": str(_resolve_pdf_path(r.pdf_path)),
+            "pdf_id": r.pdf_id,
+            "page_index": r.page_index,
+            "total_pages": r.total_pages,
+            "volume": r.volume,
+            "score": r.score,
+            "point_id": r.point_id,
+        }
+        for r in results
+    ]
+
+
 @app.get("/api/search/similar")
 def search_similar(
     point_id: int = Query(..., description="Qdrant integer point ID"),

src/pdfiles/searcher.py

@@ -39,6 +39,12 @@ def expand_query(query: str) -> str:
             return f"Photograph or image showing: {query}"
         return query
 
+    def search_by_image(self, image: "PIL.Image.Image", top_k: int = 10) -> list[SearchResult]:
+        """Search for pages visually similar to an uploaded image."""
+        embeddings = self.embedder.embed_images([image])
+        query_vectors = embeddings[0].tolist()
+        return self.store.search(query_vectors, top_k=top_k)
+
     def search(self, query: str, top_k: int = 10) -> list[SearchResult]:
         """Search for pages matching a text query (single-query with expansion)."""
         expanded = self.expand_query(query)

web/src/App.jsx

@@ -1,6 +1,6 @@
 import React, { useState, useEffect, useCallback, useRef } from 'react';
-import { Search, ArrowLeft, X, LayoutGrid, Loader2, Database, Download, ChevronLeft, ChevronRight, Scan } from 'lucide-react';
-import { searchPages, searchSimilar, getClusters, getClusterDetail, getStatus, startIndexing, getIndexStatus, pageImageUrl, exportSnapshot } from './api';
+import { Search, ArrowLeft, X, LayoutGrid, Loader2, Database, Download, ChevronLeft, ChevronRight, Scan, ImagePlus } from 'lucide-react';
+import { searchPages, searchSimilar, searchByImage, getClusters, getClusterDetail, getStatus, startIndexing, getIndexStatus, pageImageUrl, exportSnapshot } from './api';
 
 function App() {
   const [query, setQuery] = useState('');
@@ -36,6 +36,7 @@ function App() {
   });
   const [indexError, setIndexError] = useState(null);
   const pollRef = useRef(null);
+  const fileInputRef = useRef(null);
 
   // Connection status
   const [backendReady, setBackendReady] = useState(false);
@@ -156,6 +157,34 @@ function App() {
     searchContextRef.current = null;
   };
 
+  const handleImageSearch = useCallback(async (file) => {
+    setLoading(true);
+    setError(null);
+    setLightboxIndex(null);
+    setCurrentPage(1);
+    setActiveClusterId(null);
+    setActiveCluster(null);
+    setClusterPages([]);
+    searchContextRef.current = { type: 'image' };
+    setQuery(`Image: ${file.name}`);
+    try {
+      const results = await searchByImage(file, FETCH_BATCH);
+      setSearchResults(results);
+      setFetchedTopK(FETCH_MAX); // disable load-more — file is ephemeral
+    } catch (e) {
+      setError(e.message);
+      setSearchResults([]);
+    } finally {
+      setLoading(false);
+    }
+  }, []);
+
+  const handleFileChange = (e) => {
+    const file = e.target.files?.[0];
+    if (file) handleImageSearch(file);
+    e.target.value = '';
+  };
+
   const handleExport = async () => {
     setExporting(true);
     try {
@@ -229,7 +258,7 @@ function App() {
     }
     // Need more results — fetch next batch and advance
     const ctx = searchContextRef.current;
-    if (!ctx || fetchedTopK >= FETCH_MAX) return;
+    if (!ctx || ctx.type === 'image' || fetchedTopK >= FETCH_MAX) return;
     const nextTopK = Math.min(fetchedTopK + FETCH_BATCH, FETCH_MAX);
     setLoadingMore(true);
     try {
@@ -286,20 +315,36 @@ function App() {
               </div>
               <input
                 type="text"
-                className="block w-full pl-4 pr-12 py-4 bg-transparent border-none text-white placeholder-gray-600 focus:outline-none focus:ring-0 text-lg"
+                className="block w-full pl-4 pr-20 py-4 bg-transparent border-none text-white placeholder-gray-600 focus:outline-none focus:ring-0 text-lg"
                 placeholder="Search documents..."
                 value={query}
                 onChange={(e) => setQuery(e.target.value)}
                 onKeyDown={handleKeyDown}
               />
-              {query && (
+              <div className="absolute inset-y-0 right-0 pr-4 flex items-center gap-2">
+                {query && (
+                  <button
+                    onClick={clearSearch}
+                    className="cursor-pointer text-gray-500 hover:text-gray-300"
+                  >
+                    <X className="h-5 w-5" />
+                  </button>
+                )}
                 <button
-                  onClick={clearSearch}
-                  className="absolute inset-y-0 right-0 pr-4 flex items-center cursor-pointer text-gray-500 hover:text-gray-300"
+                  onClick={() => fileInputRef.current?.click()}
+                  className="cursor-pointer text-gray-500 hover:text-indigo-400 transition-colors"
+                  title="Search by image"
                 >
-                  <X className="h-5 w-5" />
+                  <ImagePlus className="h-5 w-5" />
                 </button>
-              )}
+                <input
+                  ref={fileInputRef}
+                  type="file"
+                  accept="image/jpeg,image/png"
+                  className="hidden"
+                  onChange={handleFileChange}
+                />
+              </div>
             </div>
           </div>
 

web/src/api.js

@@ -45,6 +45,17 @@ export async function searchSimilar(pointId, topK = 10) {
   return res.json();
 }
 
+export async function searchByImage(file, topK = 10) {
+  const form = new FormData();
+  form.append('file', file);
+  const res = await fetch(`${API_BASE}/search/image?top_k=${topK}`, {
+    method: 'POST',
+    body: form,
+  });
+  if (!res.ok) throw new Error(`Image search failed: ${res.status}`);
+  return res.json();
+}
+
 export function pageImageUrl(pointId) {
   return `${API_BASE}/page/${pointId}/image`;
 }