[❇️] [TZF-260079]: : Add LanceDB table explorer for administrators

[recursivezero]

Description

Summary

Add an admin-only page to inspect LanceDB tables and their contents directly from the application. This will simplify debugging embeddings, validating image metadata, and troubleshooting search issues without requiring direct access to the server.

Motivation

During debugging we had to:

Verify table metadata

Inspect stored vectors and image URIs

Validate tags and hashes

Confirm embedding metadata exists

Check migrated records after URL updates

This currently requires Python scripts and direct filesystem access. An internal admin UI would make these tasks significantly easier.

Scope

Backend (Python)

Create secure admin endpoints for LanceDB inspection.

Suggested endpoints:

• GET /admin/lancedb/tables

List all LanceDB tables.

• GET /admin/lancedb/{table}

Return table information:

row count

schema

metadata

embedding function metadata

vector dimension

• GET /admin/lancedb/{table}/rows

Paginated rows.

Support:

page

pageSize

sort

filtering

• GET /admin/lancedb/{table}/row/{id} (optional)

View complete record.

Frontend (React)

Create an Admin → LanceDB Explorer page.

Features:

1. Table selector
2. Row count
3. Schema viewer
4. Metadata viewer
5. Paginated data grid
6. Search/filter
7. Sort columns
8. Copy row JSON
9. Refresh table

Table columns

Display:

1. image_uri
2. tag
3. hash
4. mtime
5. vector (collapsed by default)

Large vectors should not be expanded automatically.

Instead show:

vector length

expandable JSON

copy button

Metadata section

Display:

embedding function metadata

vector dimension

source column

vector column

Example:

{
  "embedding_functions": [
    {
      "name": "siglip",
      "source_column": "image_uri",
      "vector_column": "vector"
    }
  ]
}

Security

Admin users only.

Reuse existing authentication/authorization.

Never expose this page publicly.

UX

Server-side pagination

Loading state

Empty state

Error state

Copy-to-clipboard actions

Responsive layout

Future Enhancements

Delete row

Recompute embeddings

Edit metadata

Export rows as CSV/JSON

Compare two LanceDB tables

View vector statistics

Similarity search directly from the UI

Trigger table maintenance (optimize, vacuum, rebuild indexes)

Acceptance Criteria

[ ] Admin-only access

[ ] List all LanceDB tables

[ ] View schema and metadata

[ ] View paginated table rows

[ ] Filter by tag

[ ] Sort by columns

[ ] Collapse large vector fields

[ ] Copy row JSON

[ ] Display embedding function metadata

[ ] Works with large tables without loading everything into memory

[github-actions]

👋 Thank you for opening this issue! We will look into it as soon as possible.

[recursivezero]

Architecture Diagram

Below are architecture diagram to understand the flow without reading the backend code.

Architecture

flowchart LR

    A[Admin User] --> B[React Admin UI]

    B --> C[Admin Auth Guard]

    C -->|Authenticated Admin| D[Python Backend API]

    D --> E["GET /admin/lancedb/tables"]
    D --> F["GET /admin/lancedb/{table}"]
    D --> G["GET /admin/lancedb/{table}/rows"]

    E --> H[(LanceDB)]
    F --> H
    G --> H

    H --> I[Schema]
    H --> J[Metadata]
    H --> K[Rows]
    H --> L[Vectors]

    I --> B
    J --> B
    K --> B
    L --> B

Loading

---

Component Diagram

graph TD

    subgraph Frontend
        A1[Admin Page]
        A2[Table Selector]
        A3[Schema Panel]
        A4[Metadata Panel]
        A5[Data Grid]
        A6[Pagination]
    end

    subgraph Backend
        B1[Admin Router]
        B2[LanceDB Service]
        B3[Authentication]
    end

    subgraph Database
        C1[(LanceDB)]
    end

    A1 --> A2
    A1 --> A3
    A1 --> A4
    A1 --> A5
    A5 --> A6

    A1 --> B1
    B1 --> B3
    B1 --> B2
    B2 --> C1

Loading

---

Sequence Diagram

sequenceDiagram

    participant Admin
    participant React
    participant API
    participant LanceDB

    Admin->>React: Open LanceDB Explorer

    React->>API: GET /admin/lancedb/tables
    API->>LanceDB: List tables
    LanceDB-->>API: Tables
    API-->>React: Tables

    Admin->>React: Select table

    React->>API: GET /admin/lancedb/{table}
    API->>LanceDB: Read schema + metadata
    LanceDB-->>API: Schema + Metadata
    API-->>React: Schema + Metadata

    React->>API: GET /admin/lancedb/{table}/rows?page=1
    API->>LanceDB: Fetch rows
    LanceDB-->>API: Paginated rows
    API-->>React: Rows

    React->>Admin: Render data grid

Loading

---

Suggested implementation structure

backend/
└── image_search/
    ├── api/
    │   └── admin/
    │       └── lancedb.py        # Admin endpoints
    │
    ├── services/
    │   └── lancedb_service.py    # LanceDB read operations
    │
    └── auth/
        └── admin_guard.py

frontend/
└── src/
    ├── pages/
    │   └── admin/
    │       └── LanceDBExplorer.tsx
    │
    ├── components/
    │   ├── LanceTableSelector.tsx
    │   ├── LanceSchemaPanel.tsx
    │   ├── LanceMetadataPanel.tsx
    │   ├── LanceRowGrid.tsx
    │   └── VectorViewer.tsx
    │
    └── api/
        └── lancedb.ts

[github-actions]

🚀 Branch Created: feature/TZF-260079 by @DhwaniKhandelwal-tech