chore: async filtering rows

[appflowy]

Description

---

Checklist

General

• I've included relevant documentation or comments for the changes introduced.
• I've tested the changes in multiple environments (e.g., different browsers, operating systems).

Testing

• I've added or updated tests to validate the changes introduced for AppFlowy Web.

Feature-Specific

• For feature additions, I've added a preview (video, screenshot, or demo) in the "Feature Preview" section.
• I've verified that this feature integrates seamlessly with existing functionality.

Summary by Sourcery

Optimize database row loading and condition-based sorting/filtering by asynchronously preloading seeded row docs and progressively applying filters, while adjusting publish UI elements for database views.

New Features:

• Introduce a batch preload mechanism that uses blob diff seeds to eagerly hydrate priority row documents before user access.
• Add a callback hook to blob prefetch that signals when row doc seeds are ready for consumption by other subsystems.

Enhancements:

• Gate per-row loading on completion of batch preload to reduce redundant work and leverage prehydrated row documents.
• Update the background row document loader to prefer fast in-memory seed application and only fall back to IndexedDB when necessary.
• Change row ordering logic to progressively apply sorts/filters only to rows whose documents are loaded, avoiding full-data blocking behavior.
• Prevent publish icons from showing or toggling publish state on database views within the outline and breadcrumb components.

[sourcery-ai]

Reviewer's Guide

Introduces async, seed-aware preloading for database rows to make sorting/filtering operate on progressively loaded row docs while keeping the UI responsive, wires blob diff seeds into background loading and ensureRow, and hides publish controls for database views in outline and breadcrumb components.

Sequence diagram for async seed-aware row loading and ensureRow gating

sequenceDiagram
  actor User
  participant GridVirtualRow
  participant useRowOrdersSelector
  participant useBackgroundRowDocLoader
  participant Database
  participant prefetchDatabaseBlobDiff
  participant IndexedDB

  User->>GridVirtualRow: scrolls / opens database view
  GridVirtualRow->>Database: ensureBlobPrefetch()
  alt readOnly or missing workspaceId/databaseId
    Database-->>Database: seedsGateRef.resolve()
    Database-->>GridVirtualRow: return
  else normal prefetch
    Database->>prefetchDatabaseBlobDiff: prefetchDatabaseBlobDiff(workspaceId, databaseId, { priorityRowIds, onSeedsReady })
    prefetchDatabaseBlobDiff-->>Database: cache seeds
    prefetchDatabaseBlobDiff->>Database: options.onSeedsReady()
    Database->>Database: runBatchPreload()
    Database-->>Database: batchPreloadDoneRef = true
    Database-->>Database: collect seeds for priorityRowIds
    Database->>Database: createRowFast(rowKey, seed) for each row
    Database-->>Database: setRowMap with preloaded rows
    Database->>Database: requestAnimationFrame(registerRowSync)
    Database-->>Database: seedsGateRef.resolve()
    prefetchDatabaseBlobDiff->>IndexedDB: persist blobs and seeds
    prefetchDatabaseBlobDiff-->>Database: promise resolved
    Database-->>GridVirtualRow: blobPrefetchComplete = true
  end

  User->>GridVirtualRow: focuses or edits row
  GridVirtualRow->>Database: ensureRow(rowId)
  alt row already in rowMapRef
    Database-->>Database: getDatabaseId, getRowKey
    Database->>Database: registerRowSync(rowKey)
    Database-->>GridVirtualRow: existing rowDoc
  else row missing
    Database->>Database: await seedsGateRef.promise
    alt row added by batch preload
      Database-->>Database: existingAfterGate in rowMapRef
      Database->>Database: registerRowSync(rowKey)
      Database-->>GridVirtualRow: existingAfterGate rowDoc
    else still missing after gate
      Database-->>Database: check pendingRowDocsRef
      opt no pending
        Database->>Database: createRow(rowKey)
        Database-->>Database: pendingRowDocsRef.set(rowId, promise)
        Database-->>Database: registerRowSync(rowKey)
      end
      Database-->>GridVirtualRow: awaited rowDoc
    end
  end

  useRowOrdersSelector->>useBackgroundRowDocLoader: hasConditions = true
  useBackgroundRowDocLoader-->>useBackgroundRowDocLoader: wait until blobPrefetchComplete
  useBackgroundRowDocLoader->>Database: rows = useRowMap()
  loop for each rowId in row_orders
    alt rowDoc missing in rows and cachedRowDocsRef
      useBackgroundRowDocLoader->>Database: populateRowFromCache(rowId)
      alt seed exists
        Database-->>useBackgroundRowDocLoader: promise<YDoc>
        useBackgroundRowDocLoader-->>Database: await promise
        Database-->>Database: rowMap updated via populateRowFromCache
        useBackgroundRowDocLoader-->>useBackgroundRowDocLoader: doc found, skip IndexedDB
      else no seed or failure
        useBackgroundRowDocLoader->>IndexedDB: openCollabDBWithProvider(rowKey, { skipCache: true })
        IndexedDB-->>useBackgroundRowDocLoader: doc
        useBackgroundRowDocLoader-->>Database: provider.destroy()
        useBackgroundRowDocLoader-->>useBackgroundRowDocLoader: cachedRowDocsRef updated
      end
    end
  end

  useBackgroundRowDocLoader-->>useRowOrdersSelector: updated rowDocsForConditions
  useRowOrdersSelector-->>useRowOrdersSelector: recompute sorted/filtered rowOrders progressively

Loading

Class diagram for updated Database and background row loading hooks

classDiagram
  class DatabaseComponent {
    +workspaceId string
    +doc YDoc
    +readOnly boolean
    +rowMap Record~string, YDoc~
    +rowMapRef MutableRefObject~Record~string, YDoc~~
    +prefetchPromisesRef MutableRefObject~Map~string, Promise~void~~
    +blobPrefetchPromiseRef MutableRefObject~Promise~void~ or null~
    +localCachePrimedRef MutableRefObject~boolean~
    +syncedRowKeysRef MutableRefObject~Set~string~~
    +batchPreloadDoneRef MutableRefObject~boolean~
    +seedsGateRef MutableRefObject~DeferredGate~
    +blobPrefetchComplete boolean
    +ensureBlobPrefetch() Promise~void~ or null
    +runBatchPreload() void
    +ensureRow(rowId string) Promise~YDoc or void~
    +resetOnDocChange() void
  }

  class DeferredGate {
    +promise Promise~void~
    +resolve() void
  }

  class PrefetchOptions {
    +priorityRowIds string[]
    +onSeedsReady() void
  }

  class DatabaseBlobModule {
    +prefetchDatabaseBlobDiff(workspaceId string, databaseId string, options PrefetchOptions) Promise~void~
  }

  class UseBackgroundRowDocLoaderHook {
    +hasConditions boolean
    +cachedRowDocs Record~string, YDoc~
    +cachedRowDocsRef MutableRefObject~Record~string, YDoc~~
    +blobPrefetchComplete boolean
    +populateRowFromCache(rowId string) Promise~YDoc or null~
    +useBackgroundRowDocLoader(hasConditions boolean) RowDocLoaderResult
  }

  class RowDocLoaderResult {
    +cachedRowDocs Record~string, YDoc~
    +rowDocsForConditions Record~string, YDoc~
  }

  class UseRowOrdersSelectorHook {
    +rowOrders Row[]
    +originalRowOrders Row[]
    +rowDocsForConditions Record~string, YDoc~
    +filtersAppliedRef MutableRefObject~boolean~
    +useRowOrdersSelector() RowOrdersResult
  }

  class RowOrdersResult {
    +rowOrders Row[]
    +rollupWatchVersion number
  }

  DatabaseComponent --> DeferredGate : uses via seedsGateRef
  DatabaseComponent --> DatabaseBlobModule : calls prefetchDatabaseBlobDiff
  DatabaseBlobModule --> PrefetchOptions : consumes
  UseBackgroundRowDocLoaderHook --> DatabaseComponent : uses blobPrefetchComplete, populateRowFromCache
  UseBackgroundRowDocLoaderHook --> RowDocLoaderResult : returns
  UseRowOrdersSelectorHook --> UseBackgroundRowDocLoaderHook : uses rowDocsForConditions
  UseRowOrdersSelectorHook --> RowOrdersResult : returns

Loading

File-Level Changes

Change	Details	Files
Add a deferred gate and batch seed-based preload pipeline so ensureRow waits for initial seed hydration instead of individually opening row docs.	Introduce createDeferredGate and seedsGateRef to coordinate between blob prefetch and ensureRow calls. Track batchPreloadDoneRef and implement runBatchPreload to hydrate the first batch of priority rows from blob diff seeds using createRowFast. Wire runBatchPreload into ensureBlobPrefetch via prefetchDatabaseBlobDiff onSeedsReady, resolving the gate in read-only or error paths. Update ensureRow to first check rowMap, then await the gate and re-check before falling back to per-row loading, and reset gate/batch flags when the database doc guid changes.	src/components/database/Database.tsx
Make background row doc loading for sorts/filters seed-aware and dependent on blob prefetch completion.	Extend database context usage to include populateRowFromCache and blobPrefetchComplete. Gate background loading by hasConditions and blobPrefetchComplete so seeds are available first. Try populateRowFromCache for each rowId, short-circuiting when it succeeds before falling back to IndexedDB with openCollabDBWithProvider. Expand effect dependencies to include blobPrefetchComplete and populateRowFromCache.	src/application/database-yjs/hooks/useBackgroundRowDocLoader.ts
Change row order computation to progressively apply filters/sorts only to rows whose docs are loaded, avoiding flicker back to unfiltered results.	Add filtersAppliedRef to track when filtered/sorted results have been applied at least once. Compute rowsWithDocs from originalRowOrders using rowDocsForConditions and early-return while preserving state if no docs are loaded yet. Run sortBy and filterBy on rowsWithDocs instead of the full set, and set rowOrders to computedRowOrders or rowsWithDocs. Reset filtersAppliedRef and rowOrders to originalRowOrders when there are no active conditions.	src/application/database-yjs/selector.ts
Expose an onSeedsReady callback from the blob diff prefetcher so callers can start using seeds before IndexedDB persistence completes.	Extend PrefetchOptions with an optional onSeedsReady callback. Invoke options.onSeedsReady immediately after building the rowDocSeedLookup and before starting IndexedDB persistence.	src/application/database-blob/index.ts
Hide publish controls for database views in outline and breadcrumb UI to avoid publishing nested database views incorrectly.	Determine isDatabaseView in OutlineItemContent outside the click handler and prevent showing PublishIcon for database views in publish variant. Guard BreadcrumbItem PublishIcon rendering with a check that the crumb is not a database view (database_id present and not a database container).	src/components/_shared/outline/OutlineItemContent.tsx src/components/_shared/breadcrumb/BreadcrumbItem.tsx
Minor grid row component cleanup related to sorts state.	Import and use useSortsSelector in GridVirtualRow and add local state for openClearSortsConfirmed, preparing for UI around clearing sorts.	src/components/database/components/grid/grid-row/GridVirtualRow.tsx

Possibly linked issues

• #unknown: PR restructures async row loading/filtering to avoid undefined rows in filterBy, addressing the grid 'id' error.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}